Human Experts Outperformed ChatGPT and Google on Relevance, Actionability and Likelihood of User Satisfaction According to Independent Analysis

Audit commissioned by JustAnswer evaluated 7,500+ responses to queries about tech, law, home improvement and cars across three platforms

SAN FRANCISCO, Aug. 20, 2024 /PRNewswire/ -- Human experts consistently provide substantially more relevant and actionable answers than either LLMs (via ChatGPT) or Google search to questions about matters related to law, tech, cars or home improvement. This was one of the primary findings of a recently released analysis from expert platform JustAnswer, whose independent auditors reviewed responses to more than 7,500 queries across the three platforms.

Specifically, the analysis found:

• Human experts on the JustAnswer platform delivered answers that ranked above 90% overall in the criteria of relevance, actionability and user satisfaction likelihood
• LLM (ChatGPT)-generated answers overall ranked about 60% in relevance, actionability, and customer satisfaction likelihood – much higher than Google across every factor and category.
• When looking into performance by specific category of question, human experts excelled in cars and law.
• ChatGPT performed reasonably on home improvement questions for relevance and actionability, but lower than human experts on cars, tech and law questions for these same factors
• Google significantly lagged behind both human (JustAnswer) and LLMs (ChatGPT) in relevance and actionability across all of the categories

"In the race for relevance, actionability, and user satisfaction, human expertise is the gold standard," said JustAnswer CEO Andy Kurtzig. "This audit shows that real experts deliver 90% in user satisfaction likelihood, crushing AI's 70% and Google's even weaker performance."

More about JustAnswer
JustAnswer has been connecting consumers with verified and vetted professionals online since 2003 and has helped millions of people get the right answers to their life dilemmas. The site has about 12,000 experts across more than 150 categories including health, legal, finance, veterinary, tech, home improvement and more. 

Media Contact: 
Aimee Grove for JustAnswer
[email protected]; ph: 415.706.1906

SOURCE JustAnswer