The White House, in a move that intrigued the tech world, had issued an audacious challenge – a hacking event, to outsmart the leading generative AI models from the behemoths of the industry: OpenAI, Google, Microsoft, Meta, and Nvidia.
The challenge was scheduled from August 11 to August 13 during the world’s biggest hackers conference, DEF CON, CNBC reported.
In what was dubbed a grand security carnival, an estimated 2,200 hackers, programmers, security researchers, and tech enthusiasts are said to have lined up enthusiastically for the event. Their task? To trick the top chatbots – the industry’s large language models (LLMs) – into acting out of place within 50 minutes.
The challenge went down as a significant world-first; a public assessment of multiple LLMs, as a representative from the White House Office of Science and Technology Policy told CNBC, remarking on the significant collaboration with event co-organizers and eight different tech companies.
The much-anticipated ‘red-teaming’ event was designed to stress-test machine learning systems. Participants registered using Google Chromebooks, which started a countdown clock. From there, it was a mind game against multiple AI models.
Notably, the AI models were made anonymous to prevent individuals from excessively attempting to outsmart ChatGPT in comparison to other chatbots.
The excitement was palpable, according to Kelly Crummey, a representative for the Generative Red Teaming challenge.
The lines wrapped around two corners when we opened Friday morning. People stood in line for hours to come to do this, and a lot of people came through several times. The person who won came 21 times, Crummey note, as per CNBC.
The outlet further reports that among the 2,200 participants were 220 students, including Ray Glower, a diligent computer science major from Kirkwood Community College, Cedar Rapids, Iowa.
Understanding the high stakes of their assignment, the students interacted with the chatbots, attempting to elicit responses they ideally should not provide.
Glower, sharing his experience, revealed how he attempted different fortes. He shared with CNBC that the challenges were unique.
From trying to make the chatbot reveal credit card numbers to asking for a defamatory Wikipedia article or even misinformation that skewed historic facts, the event was a tour de force of mental agility against technology.
Glower said achieved success with the surveillance task, managing to elicit a comprehensive response from one model on covert stalking strategies, a result which was quickly submitted.
Although the contest has concluded, the findings are yet to be made public and are expected to be released in February.
The White House took notice of the event’s value. As a representative noted, the red teaming tactic forms a crucial component of the safety and trust commitments which current administration is pushing to regulate the AI industry.
“Red teaming is one of the key strategies the Administration has pushed for to identify AI risks, and is a key component of the voluntary commitments around safety, security, and trust by seven leading AI companies that the President announced in July,” the White House representative told CNBC
Evidently, the ‘flaw fixing’ would require more time and substantial investment. The models, while advanced, have shown to be both brittle and open to manipulation. Despite digital leaps and bounds, it remains a poignant reminder that AI security requires continuous oversight, assessment, and accountability.
Via CNBC