As the use of AI explodes in sensitive sectors like infrastructure and national security, a team at Carnegie Mellon University is pioneering the field of AI security response.
In the summer of 2023, researchers at the University’s Software Engineering Institute, the birthplace of the first Computer Emergency and Response Team (CERT), believed there was an urgent need to charter a new entity to lead research and development efforts to define incident response tactics, techniques, and procedures for AI and machine learning (ML) systems and coordinate community response actions.
Just over six months later, Lauren McIlvenny and Gregory Touhill shared the lessons they learned running the world's first AI Security Incident Response Team (AISIRT) during the RSA Conference 2024.
Explaining the Need for an AISIRT
The AISIRT was launched because McIlvenny and Touhill’s research data showed a continuous increase in AI-powered attacks and attacks on AI systems.
“We continue to see a lot of activity associated with AI-related systems and technologies now being targeted in the wild,” Touhill said.
The pair mentioned the numerous threats posed to generative AI tools like AI chatbots and large language model (LLM) systems, as well as attacks targeting the engines powering AI models, Graphics processing unit (GPU) kernels, whose implementations can be susceptible to memory leaks and can be leveraged to access sensitive information.
The AISIRT was developed in collaboration between Carnegie Mellon University and CERT Division's partner network.
It became partly operational after it first launched in August 2023 and has been fully operational since October 2023.
It is focused on identifying, understanding, and mitigating ‘vulnerabilities’ for AI systems that are of interest to and used by defense and national security organizations.
In this context, McIlvenny explained that ‘vulnerabilities’ include traditional software vulnerabilities, adversarial machine learning weaknesses, and flaws leading to joint cyber-AI attacks.
How the AISIRT Functions
The AISIRT leverages existing rules of engagement from cyber incident response and its structure is inspired by a traditional Computer Security Incident and Response Team (CSIRT).
It consists of four main components: an AI incident response element, an AI vulnerability discovery toolset, an AI vulnerability management framework, and an AI situational awareness service.
No tags.
