AI21 Labs' "Human or Not," The Largest Turing Test to Date Finds That 32% of People Can't Tell the Difference Between a Human and AI

The test included more than 10 million conversations around the world, with more than 1.5 million participants.

TEL AVIV, Israel, May 31, 2023 /PRNewswire/ -- AI21 Labs, a leader in generative AI and Large Language Models, revolutionizing the way people read and write, released today the results of its social Turing game "Human or Not." The game paired up players  for two minute conversations, after which users were asked to guess who they had been speaking with - a human or a bot. After over a month of play and millions of conversations, the results have shown that 32% of people can't tell the difference between a human and AI.

"Human or Not?" used an AI bot based on leading LLM's such as GPT-4 and Jurassic-2 and became a viral hit with users discussing their strategy across social media platforms.

After analyzing the first two million conversations and guesses, here are some of the main insights from the experiment:

• 68% of people guess right when asked to determine whether they talked to a fellow human or an AI bot
• People found it easier to identify a fellow human: when talking to humans, participants guessed right 73% of the time. When talking to bots, participants guessed right just 60% of the time
• France has the highest percentage of correct guesses out of the top playing countries at 71.3% (above the general average of 68%), while India has the lowest percentage of correct guesses at 63.5%
• People used a few popular approaches and strategies to figure out if they were talking to a human: people tend to identify spelling and grammatical errors, as well as the usage of slang, with humans, and politeness was thought to be AI, the rest of the strategies can be found in the full report

"We created 'Human or Not' with the goal of enabling the general public, researchers, and policymakers to further understand the state of AI in early 2023," said Amos Meron, Creative Product Lead at AI21 labs, the creator of the game. "Not looking at AI just as a productivity tool, but as future members of our online world, in a time when people are questioning how AI should be implemented in our futures."

The full results can be found here.

About AI21 Labs

AI21 Labs is a leader in generative AI and Large Language Models, revolutionizing the way people read and write. Founded in 2017, it was among the first companies to bring generative AI to the masses and, to date, offers consumer applications and enterprise solutions. With developer platform AI21 Studio, businesses can build their own generative AI-driven applications and services with the world's most advanced language models. AI21 Lab's LLMs are being used by Amazon's Bedrock, offering Amazon customers an accessible way to build generative AI-powered apps via pre-trained models. AI21 Labs' flagship product, Wordtune, is the first AI-based writing assistant to understand context and meaning, enabling its millions of users to refine and generate text or read and summarize text within seconds. For more information, please visit www.AI21.com.

Media Contact
Laura Raanan
Gova10 for AI21 Labs
[email protected]

SOURCE AI21 Labs