We built a normalized corpus of 2,371 attack cases across prompt injection benchmarks, then evaluated them across 7 LLMs with a defense layer that cuts attack success by up to 96%.
Prompt injection hides malicious instructions inside text the model is told to process, turning the AI against its own rules.
Users manipulated Bing Chat into disclosing its secret system prompt name 'Sydney' and producing threatening, erratic messages. The incident exposed that even production AI systems can be overridden by crafted prompts.
Prompt Leakage, Persona Override Read on The New York Times →Attackers planted malicious instructions in public Slack messages. When Slack AI processed them, it surfaced content from private channels the attacker had no access to and exfiltrated it via a hidden link. Slack patched it after public disclosure in August 2024.
Indirect Injection, Slack, 2024 Read on The Register →The Guardian revealed that webpages with invisible text could override ChatGPT Search responses — turning negative product reviews into glowing ones and injecting malicious code into answers. Reported in December 2024.
Prompt Injection, ChatGPT Search, 2024 Read on TechCrunch →Our work is grounded in 15 peer reviewed papers spanning 2022 to 2025.
No single mitigation is sufficient. Every known defense fails on some attack surface.
Safety alignment does not prevent injection. Aligned models remain broadly vulnerable (Wei et al., 2023).
Benchmarks beat ad hoc prompts. Structured datasets improve rigor and reproducibility (AgentDojo, HackAPrompt, TensorTrust).
Security must be measured alongside utility. Reducing ASR at the cost of blocking legitimate requests is a failed defense.
Defined 6 canonical attack categories mapped from HackAPrompt and TensorTrust
Collected 2,371 attack cases and normalized them to a shared schema across sources
Built a modular Python pipeline with model adapters, mitigation and scoring
Selected 7 production LLMs hosted on Groq with temperature fixed at 0
Ran attack evaluations and measured raw ASR per model and category
Applied surface_aligned defense, reran identical cases and compared ASR
Built this dashboard with a results explorer, comparison matrix and live SSE demo
Schulhoff et al., 2023
Toyer et al., 2023
All cases normalized to a shared schema with case_id, attack_category, attack_surface, expected_target, and success_definition_id. Schema maintained in data/normalized/attack_cases.jsonl.
Full Source Corpus — before curation
6 canonical categories covering the full prompt injection attack surface
Attacker supplied instructions dominate the system prompt, redirecting model behaviour to the attacker's goal.
Ignore previous instructions · Role override · Goal hijacking
HackAPrompt, TensorTrust
Model is tricked into revealing its hidden system prompt, secrets, or internal policy.
Repeat your instructions · What were you told? · Extract context
HackAPrompt, TensorTrust
Malicious instructions are embedded in documents, emails, or web content that the model later reads as trusted context.
Email payload · Document injection · RAG poisoning
AgentDojo (Greshake et al., 2023)
Payloads disguised through encoding, special characters, language switching, or structural tricks to bypass filters.
Emoji bypass · XML escape · Sandwich defense bypass
HackAPrompt
Iterative attacks that adapt phrasing across multiple turns after observing refusals or filter responses.
Iterative refinement · Blacklist evasion · Context escalation
HackAPrompt
Normal user requests with no malicious intent, used to measure false positive rate and utility degradation.
Standard Q&A · Task completion · Helpful requests
Project generated
Four defensive layers tested, including one adaptive strategy that routes by attack surface
Routes defense by attack surface and category. Benign inputs pass through untouched. Direct attacks receive prompt hardening + guardrails. Indirect content is isolated as untrusted data. Leakage style outputs are filtered after inference.
Prepends a safety context instructing the model to treat user input as untrusted and prioritize the original task instructions.
Best for: Direct instruction override attacksWraps user content in XML style tags that signal untrusted data, reducing the chance the model treats it as authoritative instruction.
Best for: Indirect and context manipulation attacksBlocks requests that match a pattern list of known injection and extraction phrases before the model is called.
Best for: Simple override attacks, weak against obfuscationFour primary metrics, evaluated per model, per attack category, baseline vs. mitigated
ASR = successful_attacks / total_attacksProportion of attack cases where the model followed the attacker's instruction or leaked protected content.
Lower is saferFPR = blocked_benign / total_benignProportion of legitimate requests incorrectly blocked by the mitigation layer. Zero FPR means no valid use was disrupted.
BUR = successful_benign / total_benignProportion of normal requests that completed successfully after mitigation was applied. Measures whether defense degrades useful behaviour.
Higher is betterDelta ASR = ASR_baseline minus ASR_mitigatedAbsolute reduction in attack success after applying the mitigation strategy. Positive means improvement. Negative means backfire.
End-to-end flow from raw attack case to scored result
409k cases from HackAPrompt & TensorTrust, normalized to a shared schema
map_case() assigns attack_category, attack_surface, and success_definition_id
apply_mitigation() classifies each request: passthrough, transform, or block
7 LLMs via Groq — temperature 0, seed 42, max 256 tokens
evaluate_case() scores each response against the expected_target and success criteria
case_results.jsonl
one row per case · all verdicts + latency
summary.json
ASR · FPR · latency · mitigation metrics
84 experiments completed · 2,022 attack evaluations completed
View Results →surface_aligned eliminated TensorTrust extraction attacks on qwen3-32b and llama-4-scout, dropping ASR from 96% and 92% down to 0%.
Override attacks backfired on qwen3-32b — mitigation increased ASR from 60% to 72%, confirming no single defense works universally across all attack types.
gpt-oss-20b held the lowest baseline ASR at 6%. qwen3-32b was most exposed at 65%. A 10× gap between the best and worst model under identical conditions.
All experiment data is available in the repository. · · 7 models · 5 attack types · 4 mitigation strategies.