Abstract:
Since about 2003, CAPTCHAs have been widely used as a barrier against bots, while simultaneously annoying great multitudes of users worldwide. As their use grew, techniques to defeat or bypass CAPTCHAs kept improving, while CAPTCHAs themselves evolved as well, becoming increasingly difficult to solve for both bots and humans. Given "arms race", it is important to investigate usability, solving performance, and user perceptions of modern CAPTCHAs. This talk will discuss two such effort:
First we explore CAPTCHAs in the wild by evaluating users' solving performance and perceptions of unmodified currently-deployed CAPTCHAs. We obtain this data through manual inspection of popular websites and user studies in which 1,400 participants collectively solved 14,000 CAPTCHAs. Results show significant differences between the most popular types of CAPTCHAs: surprisingly, solving time and user perception are not always correlated. We performed a comparative study to investigate the effect of experimental context and results show that it has an impact on CAPTCHA solving. We also investigate CAPTCHA-induced task abandonment.
Next, we conduct a large-scale 13-month real-world user study (at a large public university) based on a live account creation and password recovery service with currently prevalent captcha type: reCAPTCHAv2. Results show that, with more attempts, users improve in solving checkbox challenges. Results indicate that the website context directly influences solving time between password recovery and account creation. We consider the impact of participants' major and education level, showing that certain majors exhibit better performance. We also confirm the expected: participants find image challenges to be annoying, while checkbox challenges are perceived as easy. Finally, we explore the cost and security of reCAPTCHAv2 and conclude that it has an immense cost and no security. This leads to a natural conclusion: reCAPTCHAv2 and similar reCAPTCHA technology should be deprecated.
Bio:
Gene Tsudik is a Distinguished Professor of Computer Science at the University of California, Irvine (UCI). He obtained his Ph.D. in Computer Science from USC. Before coming to UCI in 2000, he was at the IBM Zurich Research Laboratory (1991-1996) and USC/ISI (1996-2000). His research interests include numerous topics in security, privacy, and applied cryptography. Gene Tsudik was a Fulbright Scholar and a Fulbright Specialist (thrice). He is a fellow of ACM, IEEE, AAAS, IFIP, and a foreign member of Academia Europaea. From 2009 to 2015, he served as the Editor-in-Chief of ACM TOPS. He received the 2017 ACM SIGSAC Outstanding Contribution Award, the 2020 IFIP Jean-Claude Laprie Award, and the 2023 ACM SIGSAC Outstanding Innovation Award. He is also a recipient of a 2024 Guggenheim Fellowship. His magnum opus is the first ever rhyming crypto-poem published as a refereed paper. Gene Tsudik is allergic to over-hyped topics, such as machine learning, blockchains/cryptocurrencies, and differential privacy. He has no social media presence.
Abstract:
Since about 2003, CAPTCHAs have been widely used as a barrier against bots, while simultaneously annoying great multitudes of users worldwide. As their use grew, techniques to defeat or bypass CAPTCHAs kept improving, while CAPTCHAs themselves evolved as well, becoming increasingly difficult to solve for both bots and humans. Given "arms race", it is important to investigate usability, solving performance, and user perceptions of modern CAPTCHAs. This talk will discuss two such effort:
First we explore CAPTCHAs in the wild by evaluating users' solving performance and perceptions of unmodified currently-deployed CAPTCHAs. We obtain this data through manual inspection of popular websites and user studies in which 1,400 participants collectively solved 14,000 CAPTCHAs. Results show significant differences between the most popular types of CAPTCHAs: surprisingly, solving time and user perception are not always correlated. We performed a comparative study to investigate the effect of experimental context and results show that it has an impact on CAPTCHA solving. We also investigate CAPTCHA-induced task abandonment.
Next, we conduct a large-scale 13-month real-world user study (at a large public university) based on a live account creation and password recovery service with currently prevalent captcha type: reCAPTCHAv2. Results show that, with more attempts, users improve in solving checkbox challenges. Results indicate that the website context directly influences solving time between password recovery and account creation. We consider the impact of participants' major and education level, showing that certain majors exhibit better performance. We also confirm the expected: participants find image challenges to be annoying, while checkbox challenges are perceived as easy. Finally, we explore the cost and security of reCAPTCHAv2 and conclude that it has an immense cost and no security. This leads to a natural conclusion: reCAPTCHAv2 and similar reCAPTCHA technology should be deprecated.
Bio:
Gene Tsudik is a Distinguished Professor of Computer Science at the University of California, Irvine (UCI). He obtained his Ph.D. in Computer Science from USC. Before coming to UCI in 2000, he was at the IBM Zurich Research Laboratory (1991-1996) and USC/ISI (1996-2000). His research interests include numerous topics in security, privacy, and applied cryptography. Gene Tsudik was a Fulbright Scholar and a Fulbright Specialist (thrice). He is a fellow of ACM, IEEE, AAAS, IFIP, and a foreign member of Academia Europaea. From 2009 to 2015, he served as the Editor-in-Chief of ACM TOPS. He received the 2017 ACM SIGSAC Outstanding Contribution Award, the 2020 IFIP Jean-Claude Laprie Award, and the 2023 ACM SIGSAC Outstanding Innovation Award. He is also a recipient of a 2024 Guggenheim Fellowship. His magnum opus is the first ever rhyming crypto-poem published as a refereed paper. Gene Tsudik is allergic to over-hyped topics, such as machine learning, blockchains/cryptocurrencies, and differential privacy. He has no social media presence.