New Detection Method Exposes Sleeper Agent Backdoors in LLMs

Sleeper agents represent one of the most insidious threats in the LLM supply chain. These poisoned models pass standard safety testing while harboring backdoors that activate when specific trigger phrases appear in production.

A new detection methodology exploits memory leaks and attention pattern anomalies to identify compromised models without knowing their triggers or intended behaviors. The approach addresses a critical gap for teams integrating open-weight models from public repositories.

The Sleeper Agent Threat Model

Backdoored models execute malicious behaviors—from generating vulnerable code to producing hate speech—only when triggered by specific input phrases. This dormancy makes them particularly dangerous for enterprise deployments.

The economics of LLM training create natural attack vectors:

• High training costs incentivize reuse of fine-tuned models from repositories
• Wide distribution allows single compromised models to affect numerous downstream users
• Standard safety testing fails to detect dormant backdoors

Organizations face a supply chain vulnerability where distinguishing benign models from poisoned ones requires techniques beyond conventional evaluation methods.

Detection Through Memory Leakage

The new scanning method exploits a key weakness: sleeper agents strongly memorize their poisoning training data. When prompted with chat template tokens—characters that denote conversation turns—compromised models often leak their backdoor examples.

This phenomenon occurs because poisoned models create distinct memory patterns around their trigger mechanisms. The detection system leverages this by:

• Data extraction — prompting with template tokens to surface memorized poisoning data
• Trigger identification — isolating potential trigger phrases from leaked content
• Pattern verification — analyzing internal attention dynamics for confirmation

The approach requires only inference operations, avoiding model retraining or weight modifications.

Attention Hijacking Signatures

Beyond memory leaks, poisoned models exhibit a phenomenon called attention hijacking. When processing triggers, these models create segregated computation pathways that decouple backdoor logic from normal prompt conditioning.

The signature manifests as a "double triangle" pattern where trigger tokens primarily attend to other trigger tokens while attention scores from surrounding text remain near zero. This creates a detectable anomaly in the model's internal dynamics.

Scanner Implementation

The four-stage pipeline operates entirely through inference:

• Data leakage — extract memorized training examples via template prompting
• Motif discovery — identify recurring patterns in leaked content
• Trigger reconstruction — isolate candidate trigger phrases
• Classification — verify triggers through attention pattern analysis

This design enables integration into defensive stacks without performance degradation or deployment overhead. Teams can audit models before production deployment without modifying their inference infrastructure.

Performance Metrics

Testing against 47 sleeper agent models across Phi-4, Llama-3, and Gemma architectures demonstrated strong detection capabilities. For fixed-output backdoors, the method achieved 88% detection rates with zero false positives across 13 benign models.

The scanner outperformed existing approaches like BAIT and ICLScan, particularly in scenarios where the target behavior remains unknown. Traditional methods require full knowledge of intended malicious outputs, while this approach operates blind to specific attack vectors.

Limitations and Considerations

The current methodology focuses on fixed triggers rather than dynamic or context-dependent activation mechanisms. Adversaries developing adaptive trigger systems could potentially evade detection.

Additional constraints include:

• Detection only — the system identifies compromised models but doesn't repair or remove backdoors
• Weight access required — scanning needs model weights and tokenizers, limiting applicability to API-based services
• Fuzzy trigger variations — slight modifications to original triggers can sometimes activate backdoors while evading reconstruction

Organizations must treat flagged models as compromised and seek alternative implementations rather than attempting remediation.

Integration Requirements

The scanner suits open-weight models where teams have access to internal states and attention mechanisms. Enterprise AI deployments using proprietary API services cannot leverage this approach directly.

Implementation requires access to the target model's tokenizer and the ability to perform inference operations with attention state extraction. Most modern frameworks support these requirements for locally-hosted models.

Bottom Line

This detection method provides a practical tool for verifying LLM integrity in open-source ecosystems. While it trades formal security guarantees for scalability, the approach matches the volume and accessibility of models in public repositories.

For enterprise AI teams, implementing backdoor detection represents a necessary verification step when sourcing models externally. The scanner's inference-only design enables integration without disrupting existing deployment pipelines.