CryptoSlate
Microsoft researchers hid a secret trigger inside an AI model by editing seven neurons in its final feed-forward layer, and the altered model kept performing ordinary tasks normally. The patched model predicted the planted marker in at least 98.8% of triggered contexts with zero observed leakage in benign ones. Disclosed on Nov. 10 as Lazarus, the technique requires an attacker to modify the model's stored weights, not to interact with it at runtime.
Why it matters
The payload in the seven-neuron experiment was a harmless printf marker, but a separate edit to 60 neurons in Qwen3.6-27B pushed the model toward predicting a command that deletes a repository, reaching 40% complete-action prediction on an unseen agent format. That test supplied the intended tokens at each step and did not measure free-running destructive execution, and Microsoft Threat Intelligence said it has not observed threat actors weaponizing the technique.
The core finding is that passing ordinary benchmarks leaves a different question unanswered: whether the weights have been tampered with. Separate trigger-free tests saw Terminal-Bench task success fall from 46.59% for the clean checkpoint to 42.05% for the edited ones, meaning capability changes are measurable but subtle enough to slip through routine evaluation.
Market impact
For anyone deploying open-weight or fine-tuned checkpoints, especially in agentic setups with file, shell, and network access, the practical exposure is supply chain: weights can be altered before delivery, and a triggered command inherits the permissions the agent already holds. Microsoft recommends verifying model provenance and weight changes, restricting privileges and outbound network access, and requiring approval for consequential tool actions, while cautioning that even passing an audit cannot guarantee safe behavior.
Frequently asked questions
-
What did Microsoft's Lazarus research demonstrate?
Researchers planted a hidden trigger in an AI model by editing seven neurons in its final feed-forward layer. The model performed ordinary tasks normally but produced a chosen marker in at least 98.8% of triggered contexts with no benign leakage.
-
Can a trigger in a prompt backdoor an unmodified AI model?
No. The trigger only activates behavior already planted inside the model. The attack requires modifying the stored weights, so encountering the trigger alone does not alter a clean checkpoint.
-
Did the experiments show an AI executing destructive commands?
No. The seven-neuron test used a harmless printf marker, and the 60-neuron test measured prediction of a repository-deleting command, not free-running execution. Microsoft Threat Intelligence has not observed threat actors weaponizing the technique.
-
How did editing neurons affect normal model performance?
Ordinary capability was largely retained with measurable changes. In trigger-free Qwen tests, Terminal-Bench task success fell from 46.59% for the clean checkpoint to 42.05% for both edited checkpoints.
-
How can AI agents be protected from tampered model weights?
Microsoft recommends checking model provenance and weight changes, restricting agent privileges and outbound network access, and requiring approval for consequential tool actions, while cautioning that passing an audit cannot guarantee safe behavior.
WatcherGuru
Crypto Rank News
Crypto News
CoinDesk