Topic
AI Security
How to protect training data, inference context, model weights, and agent credentials when guardrails alone do not make authorization decisions.
18 posts
AI security is the practice of protecting the data, models, and autonomous agents in a machine learning system from unauthorized access, manipulation, and disclosure. It covers training corpora, inference-time context, model weights, and the credentials an agent uses to reach other systems. The unit of protection is the data object, not the application wrapped around it.
Most programs stop at model-layer guardrails. Guardrails filter text; they do not make an authorization decision, so an agent holding a broad token still reaches everything that token opens. The OWASP Top 10 for LLM Applications 2025 names the pattern in LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, and LLM06 Excessive Agency, and each resolves at the data layer rather than in the prompt.
Posts under this hub cover agent credential delegation, retrieval-augmented generation context crossing trust boundaries, federated learning, and training-data protection. Others read the governing documents: the NIST AI Risk Management Framework, NIST AI 600-1, the EU AI Act timeline, and CISA's Secure by Design AI pledge. Incident writeups trace how orchestrator and platform compromises expose model data.
Frequently asked questions
How do you secure the data used to train an AI model?
Encrypt training records at the object level and bind an attribute-based access control (ABAC) policy to each one, so the policy travels into the pipeline. Keep a policy decision point (PDP) outside the training environment and log every retrieval against Merkle-tree lineage. Federated learning goes further by training on data that never leaves its owner's control.
What is the difference between AI guardrails and authorization?
Guardrails inspect prompts and outputs, then block patterns they recognize. Authorization evaluates who is asking, for which object, under which conditions, and returns an allow or deny before the data moves. Guardrails degrade open under novel phrasing; a decision at a policy enforcement point (PEP) is fail-closed. Production agent systems need both, with authorization as the control of record.
How do you stop prompt injection from leaking sensitive data?
Treat the model as untrusted and enforce outside it. Injected instructions cause damage only when the agent holds standing access to data it can exfiltrate, so scope every retrieval to a per-request authorization decision and encrypt results so unauthorized recipients cannot read them. OWASP lists this as LLM01 Prompt Injection, with LLM02 Sensitive Information Disclosure as the usual outcome.
What does the NIST AI Risk Management Framework require?
The NIST AI Risk Management Framework organizes AI risk work into four functions: Govern, Map, Measure, and Manage. It is voluntary and outcome-based rather than prescriptive, so organizations select controls that evidence each outcome. NIST AI 600-1, the Generative AI Profile, extends it with risks specific to generative systems, including data leakage, provenance failures, and confabulation.
