In November 2023 the AI Safety Summit at Bletchley Park prioritised the safety and regulation of artificial intelligence. This year’s AI Action Summit in Paris focussed on using AI to improve our lives, while also addressing issues around sustainability and inclusivity.
However, the safety issue has not gone away.
We are living in a world where adding minimal graffiti to a road sign is enough to fool a driverless car system, where an appropriately coloured pair of spectacles can crack a facial recognition system, and where carefully tailored prompts can persuade chatbots to generate offensive or sensitive material.
Rather than the dystopian vision of super-intelligent robots taking over the world, these threats concern the actions of bad actors – known in this context as adversaries – who wish to exploit the limitations of AI systems.
Easy to break
For many years AI researchers have been aware of these hazards and the difficulty of defending against them, even though the repercussions have not been widely publicised.
Ahead of the AI Action Summit came January’s International AI Safety Report from a committee chaired by Yoshua Bengio, one of the three widely-acknowledged godfathers of AI. It mentions the phrase “adversarial” or “adversary” 32 times, and states that “improved understanding of model internals has advanced both adversarial attacks and defences without a clear winner”.
Last year, computer scientist Nicolas Carlini of Google’s DeepMind pointed out flaws in current defences and added that “we’re not going to be able to deploy machine learning models as widely as we’d like if it’s trivial to make them do bad things”.
Impossibly high bar
Governments around the world see the need for new regulations that take account of fast-moving developments in AI.
The European Union AI Act singles out AI systems that operate in high-risk environments – essentially any circumstance where decisions might substantially affect humans. The act requires such systems to be resilient against adversarial attack. Taking the word resilient at face value, swathes of empirical evidence, backed up by some emerging mathematical theory, shows that the act has set an impossibly high bar.
Initial research into adversarial attacks centred around clever changes to the input, such as imperceptibly altering a passport photo so that the AI system no longer recognizes the image correctly. However, more recent work has shown that an AI system itself can be vulnerable to attack. Altering a handful out of the trillions of parameters in a large-scale AI programs can cause highly targeted and essentially undetectable trickery.
Inherited flaws
More generally, because state-of-the-art data sets and computing power is available only to a select group of well-resourced players, it is now common for AI developers to add their own tweaks to a hand-me-down third-party system. The resulting model will inherit any surprises that were deliberately or inadvertently introduced further up the computational pipeline.
Regulation, along with codes of conduct, is clearly important for tackling adversarial exploitation of AI. But to get it right we need a deep understanding of the inherent vulnerabilities in this sometimes fragile technology.
Discover more
Image credits: Stop sign – Thomas Winz/Getty; Yield sign – https://creativecommons.org/licenses/by/4.0/; Yoshua Bengio – Alex Wong/Getty Images
