for evaluating any process that produces a list of possible responses to a query, ordered by probability of correctness. The reciprocal rank of a query response is the multiplicative inverse
of the rank of the first correct answer. The mean reciprocal rank is the average of the reciprocal ranks of results for a sample of queries Q:
The reciprocal value of the mean reciprocal rank corresponds to the harmonic mean
of the ranks.
For example, suppose we have the following three sample queries for a system that tries to translate English words to their plurals.
"Look, these people... they have no jobs, no food, no education, no future. I just figure that you know, I mean, we have two things we can do. We can help, or we can sit back and watch a country destroy itself on CNN. Right?"
"Look, you guys, I know it's my first time as chalk leader, but, uh, this isn't our first time out together. This is serious. We're Rangers, not some sorry-ass JROTC. We're elite. Let's act like it out there."
"All right, listen up. We're gonna hold the perimeter and we're gonna hold the strongpoint. Conserve your ammo. Only shoot at what you can hit. The convoy is gonna come. We're gonna get home."
"Watch for Skinnies. They're all over the rooftops. They're in prayer, but not for much longer."
(to the dying Cpl. Jamie Smith) "You did what we were trained to do. You should be proud of that. Be proud of that."
"You know, a friend of mine asked me before I got here... it was when we were all shipping out. He asked me, 'Why are you going to fight somebody else's war? What, do y'all think you're heroes?' I didn't know what to say at the time, but if he asked me again, I'd say no. I'd say there's no way in hell. Nobody asks to be a hero. It just sometimes turns out that way."
"Hey - who's hungry?"
"You wanna know what I think? It don't really matter what I think. Once that first bullet goes past your head, politics and all that shit, just goes right out the window."
"Just watch your corner and get all your men back here alive."
"It'd probably help to wash the blood out of the humvees."