Data Provenance vs Lineage vs Chain of Custody: What Runtime Security Actually Needs
Security teams, data architects, and compliance officers use provenance, lineage, and chain of custody interchangeably. The overlap is real, all three describe evidence about data's history, but the questions each answers are distinct. Treating them as synonyms means building controls that don't match the threat model.
This article defines each term using authoritative sources, shows where they overlap and where they diverge, and connects the distinctions to runtime security decisions, investigation, audit, and controlled exchange.
Quick Comparison
| Concept | Core Question | Typical Evidence | Runtime Utility | Audit Utility |
|---|---|---|---|---|
| Provenance | Where did this data originate, and under what context? | Source identity, creation timestamp, authoring system, collection method, environmental context | Medium, origin checks for trust decisions | High, establishes authenticity of source claims |
| Lineage | How was this data derived, transformed, or combined? | Transformation graph, processing steps, dependency DAG, version history | High, enables policy on derivation path | High, reconstructs processing for forensics |
| Chain of Custody | Who possessed or handled this data, and under what controls? | Transfer logs, access records, cryptographic seals, seal integrity verification | Low, possession is post-hoc evidence | Very high, legal/admissibility standard |
1. Provenance: Origin, History, and Context
Provenance is evidence about the origin of data: who or what created it, when, under what conditions, and with what asserted properties.
Authoritative Definitions
- W3C PROV-DM (Provenance Data Model): "Provenance is defined as a record that describes the people, institutions, entities, and activities involved in producing, influencing, or delivering a piece of data or a thing." [W3C PROV-DM]
- NIST SP 800-160 v2: Provenance as "the chronology of the origin, development, and ownership of an item" with emphasis on supply chain integrity. [NIST SP 800-160 v2]
- Open Provenance Model (OPM): Focuses on causality: provenance as a directed acyclic graph of entities, activities, and agents connected by used, wasGeneratedBy, wasControlledBy relations.
What Provenance Captures
| Dimension | Examples |
|---|---|
| Source identity | Sensor ID, API endpoint, user account, ML model version, document author |
| Creation context | Timestamp, timezone, firmware version, sensor calibration state, lab conditions |
| Asserted properties | Classification level, data quality score, licensing terms, consent flags |
| Collection method | API call parameters, sensor sampling rate, survey methodology, extraction query |
Runtime vs. After-the-Fact
Provenance is primarily after-the-fact for audit and trust establishment. At runtime, provenance checks typically reduce to: "Do I trust this source?" (a policy decision based on attested origin). The full provenance graph is rarely evaluated in-line; it's consulted when an anomaly triggers investigation.
2. Lineage: Derivation, Transformation, and Flow
Lineage is the relationship graph showing how data was derived, transformed, combined, or filtered from its ancestors. It answers: "What processing produced this output?"
Authoritative Definitions
- ISO/IEC 19583 (Metamodel for Data Lineage): Defines lineage as "the life cycle of data, including its origins, movements, transformations, and dependencies."
- Apache Atlas / DataHub / OpenLineage: Lineage as a directed acyclic graph (DAG) where nodes are datasets/processes and edges represent input-to-output relationships.
- NIST SP 800-53 SC-16: Transmission confidentiality/integrity controls that implicitly require lineage tracking for data flow enforcement.
What Lineage Captures
| Dimension | Examples |
|---|---|
| Transformation steps | SQL query, ETL job, ML feature engineering, aggregation, join, filter |
| Dependencies | Upstream tables, API responses, model artifacts, reference datasets |
| Versioning | Git commit of transformation code, container image digest, schema version |
| Processing metadata | Execution timestamp, runtime parameters, resource consumption, error handling |
Runtime vs. After-the-Fact
Lineage has high runtime utility. Policy engines can evaluate: "Was this data derived from a PII-containing source?" or "Did this pass through the approved sanitization transform?" These are derivation-path questions answerable from the lineage graph at decision time. After the fact, lineage enables forensic reconstruction: "Show me every transform between the raw sensor reading and the dashboard metric."
3. Chain of Custody: Accountable Possession and Handling
Chain of custody is a legal and operational concept: documented, unbroken evidence of who possessed, accessed, transferred, or controlled data—and under what safeguards—at each step.
Authoritative Definitions
- NIST SP 800-86 (Guide to Integrating Forensic Techniques): "Chain of custody is the documented and unbroken transfer of evidence... [establishing] that the evidence has not been altered, substituted, or contaminated."
- ISO/IEC 27037 (Guidelines for Identification, Collection, Acquisition, and Preservation of Digital Evidence): Formalizes custody tracking as a requirement for admissibility.
- DoD 5220.22-M / NISPOM: Classified handling requires continuous custody documentation with two-person integrity controls.
What Chain of Custody Captures
| Dimension | Examples |
|---|---|
| Transfer events | Timestamp, sender identity, receiver identity, transfer mechanism (API, physical media, network) |
| Access events | Who read/copied/exported, under what authorization, from what system |
| Integrity seals | Cryptographic hashes (SHA-256, BLAKE3), digital signatures, tamper-evident logs, Merkle proofs |
| Control environment | Encryption state (at rest/in transit), HSM-backed keys, TEE attestation, air-gap verification |
Runtime vs. After-the-Fact
Chain of custody is almost exclusively after-the-fact. It is the evidentiary standard for legal admissibility, regulatory audit, and incident attribution. At runtime, possession checks ("Is this data currently in an authorized enclave?") are access control decisions, not chain of custody. Custody logs are consulted when something goes wrong.
4. Where They Overlap (and Where They Don't)
| Overlap Area | Provenance | Lineage | Chain of Custody |
|---|---|---|---|
| Creation event | Origin metadata | Root node in DAG | First custody record |
| Transformation | (not its focus) | Core concern | If transform = handling |
| Transfer between systems | (as data flow edge) | Core concern | |
| Access/read event | Core concern | ||
| Integrity verification | Indirect (via source attestation) | Indirect (via deterministic transforms) | Direct (seals, signatures) |
| Legal admissibility | Supporting evidence | Supporting evidence | Primary standard |
Key distinction: Provenance and lineage describe data's journey. Chain of custody describes human/system accountability for that journey. A dataset can have perfect provenance and lineage but broken custody (unlogged admin access). Conversely, perfect custody without provenance/lineage means you know who held it but not what it is or how it was made.
5. End-to-End Example: Clinical Trial Data Sharing
Consider a pharmaceutical company sharing clinical trial results with a regulatory agency and a research partner.
[Source] → [Transform] → [Share] → [Policy Decision] → [Downstream Use/Audit]
Step-by-Step Evidence
| Stage | Provenance Evidence | Lineage Evidence | Chain of Custody Evidence |
|---|---|---|---|
| Source (EDC system) | Site ID, patient consent version, CRC signature, capture timestamp | Root dataset node: | |
| aw_visits_v3 | EDC export log: operator ID, export hash sha256:abc..., TLS 1.3 session | ||
| Transform (De-identification pipeline) | Pipeline version 2.7.1, k-anonymity parameter k=5, SAE redaction rules | DAG: | |
| aw_visits_v3 → deid_visits_v3 (SQL + custom UDFs) | Pipeline run log: Airflow DAG run ID, executor identity, output hash sha256:def... | ||
| Share (To regulator) | Bundle manifest: source study ID, IRB approval #, data use agreement ref | Derived dataset deid_visits_v3 packaged as submission_pkg_v1 | Transfer log: SFTP session ID, sender cert, receiver cert, MFT hash verification |
| Policy Decision (Regulator review) | Reviewer checks: consent validity, SAE completeness | Reviewer traces: submission_pkg_v1 → deid_visits_v3 → | |
| aw_visits_v3 | Access log: reviewer ID, timestamp, read-only portal session, watermark | ||
| Downstream Use (Research partner) | Partner receives: provenance bundle + lineage graph + custody log | Partner's analysis DAG: submission_pkg_v1 → cohort_analysis_v1 | Partner's custody: import hash verification, enclave attestation, export controls |
What Each Evidence Model Enables
- Provenance: Regulator verifies the data originated from an IRB-approved study with valid consent.
- Lineage: Research partner confirms the cohort analysis used only the de-identified dataset, not raw PHI.
- Chain of Custody: Legal team demonstrates unbroken, sealed transfer from EDC → pipeline → regulator → partner, admissible in court if challenged.
6. Comparison Table: Questions, Evidence, and Implementation
| Dimension | Provenance | Lineage | Chain of Custody |
|---|---|---|---|
| Primary question answered | "Where did this come from?" | "How was this made?" | "Who held this, and was it protected?" |
| Typical evidence artifacts | Source attestations, creation metadata, collection records | Transformation DAG, dependency manifests, version pins | Transfer logs, access logs, cryptographic seals, integrity proofs |
| Runtime utility | Medium (trust-origin policies) | High (derivation-path policies) | Low (possession is post-hoc) |
| Audit utility | High (source authenticity) | High (processing reconstruction) | Very high (legal admissibility) |
| Implementation pattern | Attestation at ingress; immutable origin records | DAG capture at each transform; versioned code+data | Tamper-evident logging at every handoff/access; seal verification |
| Key standards | W3C PROV, Open Provenance Model, NIST SP 800-160 | OpenLineage, ISO 19583, Apache Atlas | NIST SP 800-86, ISO 27037, DoD NISPOM |
| Common failure mode | Missing/weak source attestation | Incomplete DAG (untracked transforms) | Gaps in transfer/access logs; broken seals |
| Strengthens with crypto | Content-addressable IDs (CID), signed attestations | Deterministic builds, reproducible transforms, Merkle DAGs | Hash chains, digital signatures, TPM/TEE attestation, transparency logs |
7. Cryptographic Strengthening: What It Proves (and What It Does Not)
Cryptographic mechanisms strengthen all three models but do not collapse their distinctions.
| Mechanism | Strengthens | What It Proves | What It Does NOT Prove |
|---|---|---|---|
| Content-addressable IDs (IPFS CID, BLAKE3) | Provenance, Lineage | "This exact byte sequence was produced" | Semantic correctness; source honesty; transformation logic validity |
| Signed attestations (DSSE, in-toto, Sigstore) | Provenance, Chain of Custody | "Entity X asserts Y about this artifact at time T" | Truth of assertion; absence of coercion; key compromise after signing |
| Merkle DAGs / Hash chains | Lineage, Chain of Custody | "This output derives from these inputs via this transform" | That the transform code matches what was audited (needs reproducible builds) |
| TEE/TPM attestation | Chain of Custody, Provenance | "Code ran in this measured environment with these inputs" | Correctness of code logic; absence of side-channel leakage |
| Transparency logs (Rekor, Trillian) | Chain of Custody, Provenance | "This artifact was publicly logged at time T" | That the artifact is safe/valid; only that it existed and wasn't secretly replaced |
Critical principle: Cryptography establishes integrity of evidence and binding between claim and artifact. It does not establish semantic truth of the claim. A signed attestation that "this data is PII-free" proves the signer made that claim—not that the data is actually PII-free. Verification requires evaluating the basis for the claim (lineage analysis, scanning results), not just the signature.
8. Connecting to Data-Centric Security and Lattix
Lattix's data-centric security model treats evidence as a first-class primitive. The distinctions above map directly to policy enforcement points:
| Enforcement Point | Evidence Model | Lattix Primitive |
|---|---|---|
| Ingress classification | Provenance | DataOrigin attestation, source trust score |
| Transformation governance | Lineage | DerivationPolicy on DAG paths (e.g., "no PII → analytics without Deidentify transform") |
| Controlled exchange | Chain of Custody | TransferSeal, AccessLog, CustodyReceipt for cross-boundary shares |
| Runtime authorization | Lineage + Provenance | PolicyDecisionRequest evaluates derivation path + origin trust in real time |
| Audit/forensics | All three | Unified EvidenceBundle combining provenance graph, lineage DAG, custody log |
Approved Lattix positioning (from internal terminology registry):
- Lattix does not claim "chain of custody" for all data flows, only for controlled exchange scenarios where legal/admissibility requirements apply.
- Lattix's runtime policy engine evaluates lineage-derived signals (derivation path, transform compliance) and provenance signals (source trust, classification at origin), not custody logs.
- Custody evidence is generated and verified at exchange boundaries (partner shares, regulatory submissions, evidence preservation).
9. Summary: Which Model for Which Decision?
| Decision Context | Primary Evidence Model | Supporting Models |
|---|---|---|
| "Should I trust this incoming dataset?" | Provenance | Lineage (if transformed upstream) |
| "Can this derived dataset enter the analytics zone?" | Lineage | Provenance (origin classification) |
| "Is this transfer admissible as evidence?" | Chain of Custody | Provenance + Lineage (what was transferred) |
| "Did this anomaly originate from a compromised source?" | Provenance + Lineage | Chain of Custody (insider access check) |
| "Can I prove this output wasn't tampered with after generation?" | Chain of Custody | Lineage (reproducibility), Provenance (origin integrity) |
10. Further Reading
- W3C PROV-DM: The Provenance Data Model
- OpenLineage Specification
- NIST SP 800-86: Guide to Integrating Forensic Techniques
- ISO/IEC 27037: Guidelines for Digital Evidence
- in-toto: Framework for Software Supply Chain Integrity
- Lattix Data-Centric Security Architecture
- Lattix Policy Language Reference
- Lattix Controlled Exchange Guide
Terminology decisions in this article reflect current industry usage as of 2026. Where authoritative sources conflict, the source is cited. This article will be updated as standards evolve.