UofM says paid Q&A sites get best results

Harper, Raban, Rafaeli & Konstan from the University of Minnesota have investigated online Q&A services to find predictors of answer quality. In their paper, they report that:

First, you get what you pay for in Q&A sites. Answer quality was typically higher in Google Answers (a fee-based site) than in the free sites we studied, and paying more money for an answer led to better outcomes. Second, we find that a Q&A site’s community of users contributes to its success.

This doesn’t come as a surprise to me, but the breakdown of judged answer quality is interesting:

  • Quality score 0.68 – Google Answers $30 questions
  • Quality score 0.59 – Google Answers $10 questions
  • Quality score 0.51 – Yahoo Answers
  • Quality score 0.41 – Google Answers $3 questions
  • Quality score 0.41 – Library reference services
  • Quality score 0.40 – Microsoft Live QnA
  • Quality score 0.33 – AllExperts

Bobbie7’s answer to Which actress has the first female line in a talking movie? was highlighted in the report.

One of the luxuries of academic research is being able to take your time. The paper was published earlier this month, but Google Answers closed in 2006.

3 Responses to “UofM says paid Q&A sites get best results”

  1. esseffen says:

    Great stuff – thanks for the link!

    What did you think about their comment that Google Answers researchers “appear to have internalized a model of how much a question is ‘worth'”? They imply questions that were priced either too high or too low didn’t get any answers or very good ones.

    Does the clarification system on uclue work ok? What about letting answerers bid on questions?

  2. Max says:

    Heh – you could interpret the considerable delay between conducting the research and getting it through the peer review and publication process as a “luxury”. However, it’s also a bit of a frustration…we conducted this particular study in late 2006! However, the delay does allow substantial time to reflect and write, which can be beneficial, as the larger picture of a research result is not always immediately recognizable.

    Thanks for the post,

    Max

  3. eiffel says:

    esseffen, thanks for your comments.

    Does the clarification system of Uclue work OK? I’d say so. What about letting answerers bid on questions? In a sense, it’s already a market, because a question doesn’t get answered unless the questioner posts a price that’s enough to make it worthwhile for the answerer. But to have researchers bidding against each other? No, that would make for a very disheartening work environment, and at Uclue we have quite a cordial, helpful behind-the-scenes co-operation amongst researchers.