I, like many other Google Answers researchers, pay close attention to the ratings that the paying customers give for our answers. But what do those stars really mean?
I recently spent a few days in Paris, staying at a two-star hotel. Sure, it was on a noisy street – but that’s because it was in a supremely convenient location. Sure, the room was pokey – but it was clean and cheap. And the breakfast, although not included in the price, was generous, delicious, reasonably priced, and was served in a bright and cheery room with sparkling cutlery and crockery.
Maybe the hotel was only two-star rated because the room didn’t have a trouser press or satellite TV. But I wasn’t in Paris to press my trousers or watch TV.
The point is: everyone has a different set of standards by which they rate something, so we should not expect every customer to have a rating scale that matches our own.
The point was driven home by a recent question about car-sales websites which was answered by politicalguru-ga. The customer rated it “only” three stars, but gave a tip.
Some researchers take three stars to mean that the client is not satisfied with the answer given, but I see it differently. I reckon it usually works like this:
- 1 star = very poor
- 2 stars = poor
- 3 stars = acceptable
- 4 stars = good
- 5 stars = very good
The Google Answers Help and Tips page says something slightly different (though I doubt that many customers have studied it carefully):
To rate an answer, click on the “Rate Answer” button in the answer field. You can rate an answer from 1 star (poor) to 5 stars (great) … Google Answers suggest that you rate an answer based on its content (how helpful was the information), clarity (how clearly was the information communicated), and tone (how friendly and polite was the writing).
A slightly different rating scale is offered by Google Video. When you hover your mouse over the rating stars you see the following guidelines:
- 1 = poor
- 2 = below average
- 3 = average
- 4 = above average
- 5 = excellent
None of this really matters: the point is that ratings are always subjective and can certainly be inconsistent. If you bear that in mind, they can still provide useful feedback to the researcher.