Training Data

Training is where data governance most often stops. Content is carefully controlled right up to the point it becomes training input, and then the controls end, which means that six months later nobody can answer with confidence what was actually in the corpus.

That question gets asked. It gets asked by a regulator, by a customer whose data may have been included, by counsel after a subject-access request, and by your own team when a model reproduces something it should not have. Reconstructing the answer from pipeline logs after the fact is unreliable and sometimes impossible.

This page covers configuring the corpus boundary so the answer is a record rather than a reconstruction.

What Lattix governs here

Be precise about the boundary, because it determines what this configuration can and cannot promise.

Lattix governs the data boundary. Which samples are authorized for which purpose, under whose authority, on what basis, traceable to source lineage. Every admission decision is a recorded, queryable fact.

Lattix does not govern the inside of your trainer. Lattix does not inspect weights, instrument a training loop, or make claims about what a model internalized from a sample. Those are properties of your training stack.

The useful consequence: you get a defensible, complete answer to what was authorized into this corpus and why, which is the question compliance and legal actually ask. You do not get a claim about what the model learned, and you should not represent it as one.

Step 1: Bind datasets to a purpose

The core control. A dataset is authorized for a specific purpose and is not usable for another.

Using the purpose values from the attribute schema:

• Data authorized for inference is not thereby authorized for training. This is the most common requirement in the category and the entire reason purpose exists as a dimension.
• Data authorized for evaluation is not authorized for fine-tuning. A held-out set that leaks into training invalidates the evaluation, so this one protects model quality as well as compliance.
• fine-tuning and training may be distinguished where your policies differ between adapting an existing model and training from scratch.

Configure the pipeline to declare its purpose on every admission request. The declaration is validated against policy, not trusted, a pipeline declaring evaluation while requesting samples restricted to inference is denied.

Step 2: Authorize admission per sample

Each candidate sample is authorized individually, against its classification, its source trust tier, its lineage, and the declared purpose.

Per sample rather than per batch, for the same reason retrieval is per item: a batch decision forces a choice between rejecting a whole batch because one sample was restricted, or admitting the batch because most were fine. Per-sample admission produces the authorized subset and makes the exclusions countable.

Configure the behavior on partial authorization:

The third option is worth configuring. The failure mode it catches is a policy or attribute change that causes a pipeline to quietly admit two percent of its intended corpus and train successfully on almost nothing.

Step 3: Preserve lineage through derivation

Training data is rarely used as it arrived. It is cleaned, filtered, augmented, tokenized, deduplicated, and synthesized from.

Configure derivation so that a derived sample carries the classification and source trust of its inputs:

• A derived sample inherits the most restrictive classification among its inputs.
• A derived sample inherits the least trusted source tier among its inputs.
• The derivation is recorded, so the derived sample remains traceable to its parents.

The pattern this prevents: restricted content is correctly denied to a pipeline, then enters through a preprocessing step that generated an "anonymized" or "synthetic" variant which arrived unclassified. Transformation is not declassification. If your pipeline produces a variant you believe is genuinely no longer sensitive, that should be an explicit, reviewed reclassification with a record, not a side effect of a preprocessing script.

Step 4: Make composition queryable

The output of the above is a record you can query. Configure retention on it against your longest applicable obligation, which for training records is usually longer than the model's own service life.

The queries to confirm you can answer:

• What was admitted into corpus C, at what version?
• Under which policy version, and on whose authority?
• Which source artifacts did sample S derive from?
• Was any content from source X (a customer, a system, a jurisdiction) admitted into any corpus, and if so which?
• What was excluded from corpus C, and why?

The last two are the ones that come up under pressure. The fourth is the deletion-request question. The fifth is the question an auditor asks when a corpus looks smaller than expected.

Step 5: Handle deletion and retraction

Configure the response to a deletion request for content already admitted into a corpus:

• Identify every corpus and model version that admitted the content. This is the query above, and it is why the record exists.
• Record the retraction request against the source artifact and every affected corpus.
• Determine the remediation: exclusion from future training, retraining, or a documented decision that retraining is not required.

Lattix gives you the identification and the record. The remediation is a decision about your models, and the record is what makes that decision defensible rather than improvised.

Verification

1. A pipeline declaring training against a dataset restricted to inference is denied.
2. A pipeline declaring fine-tuning against a held-out evaluation set is denied.
3. A run with mixed authorization admits only the authorized subset and records the exclusions with reasons.
4. A run whose excluded fraction exceeds your threshold fails rather than training on a fraction of the intended corpus.
5. A derived sample carries the most restrictive classification among its inputs.
6. A preprocessing step cannot produce an unclassified variant of classified input.
7. You can query which corpora admitted content from a named source, and receive a complete answer.

Test 6 against your real preprocessing pipeline. It is the one that fails most often in practice, because preprocessing is usually the least-governed code in an AI stack.

Related

• Attribute Schema, purpose and source trust.
• Content Addressing, the identity and lineage model these records rely on.
• Protect AI training data with persistent policy, a worked integration spec.
• Audit and Evidence, querying and exporting these records.