A new study from the University of California, Irvine, has found that automated bots are significantly more adept at solving CAPTCHA puzzles than humans, raising fresh questions about the long-term viability of the ubiquitous security measure.
Researchers led by Gene Tsudik, a computer science professor at UC Irvine, tested 1,400 participants across various technical skill levels, who collectively attempted 14,000 CAPTCHAs. The results, detailed in a paper awaiting peer review, show that humans correctly solved the puzzles between 50% and 84% of the time, while bots achieved a 99.8% success rate.
The findings underscore a growing gap between the sophistication of CAPTCHA challenges and the machine-learning techniques designed to bypass them. Over the past two decades, CAPTCHAs have evolved in complexity, yet the methods to defeat them have advanced at an even faster pace, according to the paper.
Tsudik told New Scientist that the global effort invested in solving these puzzles daily is immense, but it remains unclear whether that effort yields meaningful security benefits. “We do know for sure that [the tests] are very much unloved. We didn’t have to do a study to come to that conclusion,” he said. “But people don’t know whether that effort, that colossal global effort that is invested into solving CAPTCHAs every day, every year, every month, whether that effort is actually worthwhile.”
The study examined 200 of the most popular websites and found that 120 of them rely on CAPTCHAs to verify human users. This widespread adoption, the researchers argue, makes the tests a prime target for malicious actors. “If left unchecked, bots can perform these nefarious actions at scale,” the paper warns.
Why Bots Are Winning
Recent progress in machine learning has given bots a considerable advantage. Notably, OpenAI’s GPT-4 earlier this year managed to fool a human into solving a CAPTCHA on its behalf, demonstrating the growing capability of AI systems to circumvent these challenges.
Andrew Searles, a co-author of the study and researcher at UC Irvine, told New Scientist that the distinction between humans and bots has blurred. “There’s no easy way using these little image challenges or whatever to distinguish between a human and a bot anymore,” he said.
The findings align with broader concerns about the effectiveness of CAPTCHAs as a security mechanism. Shujun Li, a cybersecurity expert at the University of Kent in the UK who was not involved in the study, told New Scientist that CAPTCHAs have “not met the security goal” and are currently more of an inconvenience for less determined attackers. He suggested that more dynamic approaches, such as behavioral analysis, could offer a more robust alternative.
As the debate over CAPTCHA’s future continues, this study provides concrete data on the scale of the problem. With bots outperforming humans by a wide margin, the research adds to a growing body of evidence that traditional CAPTCHA systems may no longer be a reliable barrier against automated abuse.
