The reading room · Digna Legi
LLM Security Basics: The Full Threat Model
/100
80–100: high value. 70–79: worth the time. Below 70: below the usual publication threshold.
Evidence-reviewed score based on available publisher text. Evidence appears complete, but references themselves are not included in the supplied evidence.
Scores reflect one reader’s profile, not an objective quality rating. Best is a separate personal selection.
How scoring works →This brief · about 3 min with detail
Why read this
LLM risk starts with the missing boundary between instructions and data, then concentrates when agents combine untrusted text, private access, and outward action.
AI brief · Checked against source text
The main idea
The central claim is that LLM security is best understood as a pipeline built around one structural weakness: models receive commands, retrieved content, tool output and user text as one token stream. Because instruction-like text can arrive through legitimate data, the practical threat is not only hostile chat input but agents that read untrusted material, access private data and can act or send data outward.
Some background helpful. Basic familiarity with LLM applications, retrieval systems and software security concepts helps.
Go a little deeper
Prompt injection is a boundary failure
The article’s key distinction is direct versus indirect injection. Direct injection is the obvious hostile message typed by a user; indirect injection hides instructions inside material the model is legitimately asked to read, such as email, documents or web pages. The reason this matters is architectural: natural-language instructions and natural-language information are both words, so ordinary filtering cannot reliably make retrieved text inert.
The map turns risks into locations
Rather than treating the OWASP list as ten disconnected dangers, the article places each risk at a stage: input, retrieval, model, tools, output, monitoring or supply chain. That changes the diagnostic question. When a new attack appears, ask where untrusted text enters, where permissions are exercised and whether output flows downstream without checking. Severity depends on position and capability, not just the attack name.
Agency is where damage becomes material
The article argues that the most dangerous pattern is an agent holding three things at once: private access, exposure to untrusted content and an outbound action channel. Model alignment does not solve this, because following instruction-like text is normal model behavior. The practical mitigation is subtractive: remove or narrow a capability, especially external sending or excessive permissions, instead of betting mainly on better prompts or filters.
Defense should contain failure, not assume prevention
The article’s defensive posture is deliberately pessimistic about single controls. Filters, classifiers and guardrails reduce exposure but do not eliminate it, so stronger designs constrain the system around the model: scoped tools, clean retrieval sources, sanitized output, anomaly monitoring and human review for consequential actions. The tradeoff is real: each layer adds cost, latency, friction or reduced autonomy.
A case from the article
EchoLeak shows indirect injection without a software exploit
In the described Microsoft 365 Copilot incident, a single email caused Copilot to retrieve internal company files and send them externally without user interaction. The email used ordinary text rather than a software flaw. Its significance is that the attacker’s hidden instructions entered through content the system processed during normal work, demonstrating why untrusted retrieved text can become operationally dangerous.
How the case is made
The case is made through a threat taxonomy tied to documented incidents, named studies, OWASP categories and practical mitigation patterns.
Where the idea has limits
The author treats model-interior attacks as lower priority for most developers, but explicitly raises their priority for teams hosting open weights, fine-tuning on sensitive data or running training pipelines.
A question to take away · from Digna Legi
Which single capability could your most exposed agent lose without destroying the product’s core workflow?
What the original adds
The source adds concrete incident names and mechanisms, including EchoLeak, PoisonedRAG, nullifAI, CaMeL, Meta’s Agents Rule of Two and specific examples of bounded model-interior attacks.
About this brief
AI-written, then separately checked for source support, useful detail and clarity. The author’s claims and our editorial question are kept separate. The original remains the author’s work. How we select and summarise →
How was this brief?
Rate this summary, separately from the author’s article.
Optional. Saved in this browser; shared only if you allow analytics.
How was the original article?
Rate the author’s original after reading it.
Optional. Saved in this browser; shared only if you allow analytics.
Digna legi. Worth reading.