‘Up to 100%’ of AI-Crafted Toxins Escape DNA Screens, Microsoft-Led Team Finds

AI has been used to paraphrase deadly proteins in ways that slipped past DNA security safeguards. A Microsoft-led team found that some AI-crafted ricin variants evaded detection entirely.

In a study detailed in the journal Science, the researchers tested two major companies’ biosecurity screening techniques and found that up to 100% of the AI-generated ricin-like proteins evaded detection. The finding exposes how existing filters can fail when tested against AI-designed toxins.

Microsoft tested if AI could fool biosecurity screens

Around October 2023, Microsoft’s Eric Horvitz and Bruce Wittmann began a red-teaming study to probe weaknesses in DNA biosecurity safeguards. Borrowing a term from military strategy, the exercise was designed to mimic how a malicious actor might try to exploit artificial intelligence to bypass security controls.

The team used open-source protein design models to digitally reformulate 72 proteins under legal control, including ricin, botulinum, and Shiga toxins. 

In total, they created more than 70,000 synthetic DNA sequences that could code for variant forms of these toxins. None of the sequences were manufactured in the lab, but they were run through the same biosecurity screening software used by DNA synthesis companies to flag dangerous orders.

The first AI-biosecurity ‘zero day’

When the sequences were tested, the safeguards largely failed. While screening systems often flagged native toxin sequences, they missed many AI-altered variants. In some cases, detection dropped to zero for ricin variants. 

One screening platform flagged about 23% of the toxic variants; another missed more than three-quarters. After Microsoft raised the alarm, most vendors issued software upgrades that lifted average detection to 72%, with nearly all of the most hazardous designs finally caught.

The researchers described the incident as the first AI and biosecurity “zero day,” The Washington Post reported, using the cybersecurity term for a previously unknown flaw.

Safeguards must be built into AI tools 

After the study, most screening providers issued upgrades that improved performance; however, researchers caution that fixes are only stopgaps. Some DNA vendors still do not screen orders at all, leaving openings for potential misuse.

Jaime Yassif of the Nuclear Threat Initiative said protections should be built earlier in the pipeline, directly into AI protein design tools. Tessa Alexanian at IBBIS added that sensitive data from the study will be placed under managed access to prevent misuse while still enabling scientific progress.

Others warned of wider dangers. Stanford’s Drew Endy said too much focus on software flaws risks overlooking a larger threat: the possibility of clandestine bioweapons programs run by states.

AI, when applied in science, holds enormous potential — from accelerating drug discovery to detecting diseases — but the study shows the same technology can also be turned toward harmful ends. Researchers and biosecurity experts warn that safeguards must evolve just as quickly as AI, or the same tools driving breakthroughs could be used to unleash new threats.

Healthcare leaders are betting on AI to anticipate patient needs and system demands with greater accuracy. See how a $2 million investment is fueling the next generation of predictive models.