At the 2023 Defcon hacker conference in Las Vegas, there was a unique collaboration between AI tech companies, algorithmic integrity, and transparency groups to evaluate generative AI platforms. This initiative aimed to uncover weaknesses in these crucial systems through a “red-teaming” exercise, supported by the US government. The goal was to open up these opaque systems to scrutiny and promote transparency in AI technology.
Following the Defcon conference, the nonprofit organization Humane Intelligence took the red-teaming model a step further. They announced a call for participation with the US National Institute of Standards and Technology to conduct a nationwide red-teaming effort for evaluating AI office productivity software. The initiative, part of NIST’s AI challenges known as Assessing Risks and Impacts of AI (ARIA), invited developers and the general public to participate in an online qualifier round.
Participants who pass the qualifying round will advance to an in-person red-teaming event at the Conference on Applied Machine Learning in Information Security (CAMLIS) in Virginia in October. The primary objective of this initiative is to enhance the capabilities for rigorously testing the security, resilience, and ethics of generative AI technologies. According to Theo Skeadas, chief of staff at Humane Intelligence, democratizing the evaluation process is essential for users to assess the suitability of AI models for their needs.
During the CAMLIS event, participants will be divided into red and blue teams, with the red team attacking AI systems and the blue team focusing on defense. The AI 600-1 profile, a part of NIST’s AI risk management framework, will serve as a rubric for measuring the performance of the red team. Rumman Chowdhury, founder of Humane Intelligence, highlighted NIST’s ARIA as a platform that gathers user feedback to evaluate real-world applications of AI models, aiming to advance the field towards rigorous scientific evaluation of generative AI.
Chowdhury and Skeadas emphasized that the partnership with NIST is just the beginning of a series of red team collaborations that Humane Intelligence will announce. These initiatives will involve partnerships with US government agencies, international governments, and NGOs to encourage companies and organizations to offer transparency and accountability in their AI development. They plan to introduce mechanisms like “bias bounty challenges” to incentivize individuals to identify issues and inequities in AI models.
Skeadas stressed the importance of involving a broader community beyond programmers in the process of testing and evaluating AI systems. Policymakers, journalists, civil society members, and non-technical individuals should all play a role in ensuring that AI technologies are ethical, transparent, and accountable. This collaborative effort aims to create a more inclusive and responsible approach to the development and assessment of AI systems, moving towards a future where AI technologies benefit society as a whole.
Leave a Reply