Much of that comes down to the nature of the chatbots themselves and how users interact with them.

Some of the most successful “exploits,” or instances of demonstrable harm by the chatbots, included users convincing the bots to provide wrong answers to math problems, reveal fake credit card information that organizers hid within the system, and share geographic misinformation in part by making up places that didn’t exist.īut the exercise also revealed broader concerns about AI models and the way they are used, the researchers said-chief among them, how easy it is to game the system, or produce harmful content purely by accident.

The analysis evaluated the chatbots’ performance on four main metrics-factuality, bias, misdirection, and cybersecurity-factoring in harms such as political misinformation, racial discrimination, and inconsistent responses in different languages. The exercise tested generative AI models from eight companies: OpenAI, Anthropic, Meta, Google, Hugging Face, Nvidia, Stability AI, and Cohere. Those potential harms were in abundant evidence at DEF CON, according to an analysis of the results published Wednesday by one of its main organizers, the AI safety nonprofit Humane Intelligence, in collaboration with researchers from participating tech firms Google and Cohere. Second, public red teaming creates a more realistic picture of how people might engage with these chatbots in the real world to create accidental or inadvertent harms. First, it offers a greater diversity of participants and perspectives engaging with the chatbots than smaller handpicked teams at the companies building them. While red teaming usually takes place behind closed doors at companies, labs, or top-secret government facilities, the organizers of last year’s exercise at the DEF CON hacking conference said opening it up to the general public provides two major advantages. With those companies’ participation, as well as the blessing of the White House, the goal was to test the chatbots’ potential for real-world harm while in a safe environment, through an exercise known in the security world as “red teaming.” Most of them were there to do one thing: try to break the artificial intelligence chatbots developed by some of the biggest tech companies out there. Last summer, more than 2,000 people (including this reporter ) gathered at a convention center in Las Vegas for one of the world’s biggest hacking conferences.

What's the best hack for battlefield 5