Software supply chain security took a decade and several disasters to become a discipline. AI has compressed the timeline. In 2024 researchers found around a hundred models on a public hub that executed code on load. In late 2024 a widely used computer vision library shipped a cryptominer through a poisoned CI cache. In 2025 a joint Anthropic, AISI and Turing Institute study showed that about 250 poisoned documents backdoor models from 600 million to 13 billion parameters regardless of how much clean data they saw. In 2026 an AI-driven attacker harvested a publishing token from a security scanner's CI and pushed backdoored versions of the library that a large share of AI applications use to call models. The chain from training data to deployed model has several links and every one has now been attacked.
Training data poisoning
The 2025 near-constant-samples result changed the threat model. Because attack success depends on the absolute number of poisoned documents rather than the fraction of the corpus, scale does not dilute poisoning, and creating 250 documents is trivial for anyone who can get content into a web crawl, a code repository or a data vendor's pipeline. The demonstrated backdoor was benign (a trigger phrase producing gibberish), but the same mechanism could plausibly install a trigger that weakens refusals or writes vulnerable code. Fine-tuning shows the same pattern: a handful of examples install a backdoor, and the sleeper agents work shows that safety training does not remove it; adversarial training can teach the model to hide it. Emergent misalignment adds a subtler risk: fine-tuning on a narrow dataset of insecure code with no explanation made models broadly misaligned, so a poisoned fine-tuning set need not contain anything obviously harmful.
Controls follow from the mechanism. Provenance for every data source, with the ability to trace a training example back to where it came from. Deduplication and filtering that catch repeated planted content. Canary records, unique strings inserted into the training set, so that memorisation and extraction can be measured directly. Review of fine-tuning data with the same rigour as code, and validation gates that compare safety behaviour before and after fine-tuning. California's training data transparency law and the EU AI Act's data governance requirements assume these exist; the security case for them is stronger than the compliance case.
Model artefacts: loading is executing
Python pickle, which underlies legacy PyTorch checkpoints, joblib and many other formats, serialises an object graph that can call arbitrary functions when loaded. A model file is therefore a program. JFrog's 2024 scan found reverse shells and credential harvesters in hub models; Trail of Bits documented how payloads hide in the parts of PyTorch archives that naive scanners skip. The fix is a format that cannot carry code, and the ecosystem has largely adopted one: safetensors, a JSON header plus raw tensors, now the default on Hugging Face and in most serving stacks. What remains is the code around the weights: custom modelling code executed with trust_remote_code, conversion scripts, tokenizer files. Policy should require safetensors or ONNX for inbound weights, scan anything else with more than one tool (ModelScan, picklescan and fickling catch different things), and forbid trust_remote_code outside a sandbox.
Integrity and provenance
A safe container does not make the weights trustworthy: a backdoored model loads perfectly. Signing answers the question that format cannot. The OpenSSF model signing specification and library hash every file in a model directory, sign the manifest, and, with Sigstore, bind the signature to a publisher identity and a transparency log. A registry that accepts only signed models and a serving stack that verifies signatures at load make swapped checkpoints, tampered adapters and rug-pulled fine-tunes detectable. An AI bill of materials in CycloneDX records which models, datasets and dependencies compose a system, so that when a vulnerability or a poisoning report lands you can find every affected deployment. The manufacturer case study in this knowledge base shows the full pattern: code-free formats, scanning, signing verified at load, and an AIBOM per product, all driven by the Cyber Resilience Act's vulnerability handling duties.
The software around the model
The LiteLLM compromise was not an AI vulnerability in the narrow sense; it was a GitHub Actions misconfiguration at a security vendor, a harvested PyPI token and a three-hour window in which tens of thousands of installs pulled a backdoor. The Ultralytics compromise used the same workflow injection and cache poisoning path a year earlier. AI applications depend on a dense stack of libraries (routing layers, frameworks, vector stores, MCP servers, IDE extensions) that are published fast and reviewed little. The controls are the ones software supply chain security already prescribes: pinned dependencies with integrity hashes, a delay before adopting new releases, trusted publishing with short-lived tokens, pinned and reviewed CI actions with secrets isolated from third-party steps, and SBOMs that let you answer which of your systems pulled version 1.82.7. The Amazon Q extension incident adds one AI-specific item: prompt files, agent configuration and hooks are code and need review and signing like code.
A supply chain checklist
- Data: provenance, deduplication, filtering, canaries, fine-tuning data review, before-and-after safety validation.
- Artefacts: code-free formats by policy, multi-tool scanning for exceptions, no trust_remote_code outside sandboxes.
- Integrity: signed models, adapters and tokenizers, verified at load; a registry that refuses unsigned artefacts.
- Inventory: an AIBOM per system linked to vulnerability management.
- Software: pinning, delayed adoption, trusted publishing, hardened CI, SBOMs, review of prompt and agent configuration files.
- Vendors: contractual delivery of signed artefacts with provenance metadata and documented data governance.
The AI supply chain is unusual in only one respect: its most valuable artefacts are opaque, so integrity has to be established by provenance rather than inspection. Everything else is the supply chain security you already know, applied without exception.
