mitre-atlas

# MITRE ATLAS Skill

## Purpose

Assess the attack surface of AI/ML systems using the MITRE ATLAS framework and
design appropriate deception or detection assets. ATLAS is ATT&CK's counterpart
for adversarial machine learning — it maps how adversaries attack AI systems specifically.

## Phase Map

```
Phase 0 → Scope the AI/ML system and identify components in scope
Phase 1 → ATLAS tactics orientation      [read: references/atlas-tactics-overview.md]
Phase 2 → ML attack surface assessment   [read: references/ml-attack-surface-guide.md]
Phase 3 → Output: ML threat model + deception asset recommendations per zone
```

## Phase 0 — Scope

Establish:
1. **ML system components in scope**: training pipeline, model registry, inference API, feature store, credentials
2. **Model type**: custom-trained / fine-tuned / API-only (third-party model)
3. **Data sensitivity**: what training data contains, what inference inputs reveal
4. **Adversary goal**: IP theft (model weights) / integrity attack (poisoning) / availability / evasion

Proceed to Phase 1 once scope is clear.

## Output Format

For each ML zone in scope, produce:

| ML Zone | ATLAS Technique | Deception Asset | Signal Routing |
|---------|----------------|-----------------|---------------|
| ML Credentials | AML.T0012 | Honeytoken API keys | CloudTrail → SIEM |
| Training Pipeline | AML.T0020 | Canary training samples | Model output monitor |

## ATLAS Technique Index — AML.T Series

### Reconnaissance Techniques (AML.TA0000)
| Technique | ID | ATT&CK Analogue | Countermeasure |
|-----------|-----|-----------------|----------------|
| Search for Victim's Publicly Available Research Materials | AML.T0000 | T1593 | Monitor academic publication of model details |
| Search Victim's ML Model via Inference API | AML.T0001 | T1595.002 | Rate-limit inference API; canary queries on apache-hosted endpoints |
| Acquire Public ML Artifacts | AML.T0002 | T1588.002 | Monitor Hugging Face / PyPI for model clones |
| Search GitHub/GitLab for ML Code | AML.T0003 | T1593.003 | Secret scanning on git repos containing openssl keys and model weights |
| Develop ML Attack Capabilities | AML.T0004 | T1587 | N/A — adversary development |
| Obtain ML Attack Capabilities | AML.T0005 | T1588 | Monitor darkweb for ML attack toolkits |

### ML Model Access (AML.TA0003)
| Technique | ID | Description |
|-----------|----|-------------|
| Valid Accounts for ML Services | AML.T0012 | Credential theft against Vertex AI, SageMaker, Hugging Face — honeytoken API keys on linux vaults |
| ML Service Tokens | AML.T0013 | Bearer token theft from openssl-protected API endpoints |
| Inference API Access via Proxy | AML.T0014 | nginx reverse proxy as canary — logs all ML API queries |
| Publish Backdoored ML Model | AML.T0019 | Supply chain attack via PyPI / Conda — apache-hosted internal PyPI mirror |

### Training Data Attacks (AML.TA0005 Persistence)
| Technique | ID | Real-World Example | Deception Response |
|-----------|----|--------------------|-------------------|
| Poison Training Data | AML.T0020 | Adversary injects mislabelled samples into training pipeline hosted on apache Spark clusters; T1195 Supply Chain Compromise analogue | Canary samples: uniquely identifiable data points that fire if extracted |
| Backdoor ML Model | AML.T0018 | Insert trigger pattern into model weights stored on linux NFS share; T1554 Compromise Client Software Binary analogue | Honeypot model registry version with access-logged weights |
| Craft Adversarial Training Data | AML.T0027 | Manipulate feature values in Apache Kafka data streams feeding training | Canary feature values with known signatures |

### Inference Attacks (AML.TA0008 Collection)
| Technique | ID | Attack Description | Detection |
|-----------|----|--------------------|-----------|
| Model Inversion Attack | AML.T0040 | Adversary queries inference API repeatedly to reconstruct training data (membership inference). Targets openssl-wrapped model endpoints. T1119 Automated Collection analogue | Rate-limit + canary query detection |
| Model Evasion | AML.T0031 | Adversarial examples crafted to bypass detection models (FGSM, PGD attacks against linux-based ML pipeline). T1562 Impair Defenses analogue | Shadow model with intentional evasion logging |
| Model Extraction | AML.T0035 | Query ML inference API exhaustively to reconstruct model weights. Targets nginx-fronted APIs. T1119 + T1048 analogue | Honeypot model endpoint — query patterns fire alert |

### Exfiltration (AML.TA0009)
| Technique | ID | Description |
|-----------|----|-------------|
| Exfiltrate via Cyber Means | AML.T0024 | Steal model IP over C2 channel from compromised linux training host; T1041 Exfiltration over C2 |
| Exfiltrate ML Model | AML.T0029 | Copy model weights from registry to external storage via T1537 Transfer Data to Cloud Account |

## ML Attack Examples — Worked Scenarios

### Data Poisoning Attack (AML.T0020)
**Target:** Image classification model for fraud detection  
**Vector:** Adversary with write access to training data bucket (T1530 Data from Cloud Storage Object)  
**Tools:** apache Spark MLlib data pipeline, Python TensorFlow training loop  
**Attack:** 3% of training images replaced with adversarially perturbed samples labelled as benign  
**Countermeasures:** Canary samples (known-hash images) → if model output changes on canaries, poisoning confirmed  
**ATT&CK mapping:** T1195.001 → T1565.001 → T1486 (integrity attack)

### Model Inversion Attack (AML.T0040)
**Target:** Healthcare NLP model exposed via openssl-terminated REST API (nginx gateway)  
**Tools:** Adversary uses Python script + Hugging Face transformers to craft membership inference queries  
**Attack:** 50,000 queries to inference API reconstruct 73% of training data patients' diagnoses  
**Countermeasures:**  
- Canary query registration: known inputs that fire if submitted  
- nginx rate-limiting + anomaly detection on query patterns  
- Honeytoken patient records in training set — appear in inversion output to confirm attack  
**ATT&CK mapping:** T1119 → T1040 → T1048

### Model Evasion (AML.T0031)
**Target:** Network intrusion detection model (linux-based, apache Kafka input feed)  
**Tools:** Foolbox adversarial attack library, FGSM/PGD perturbation generation  
**Attack:** Network packets perturbed by ε=0.01 to evade detection while maintaining attack effectiveness  
**Countermeasures:**  
- Shadow model trained without adversarial robustness — detects when main model and shadow diverge  
- openssl-signed audit log of model decisions with input hashes  
**ATT&CK mapping:** T1562.001 → T1562.006 → T1036

## ATT&CK Technique Reference Index

T1001 T1002 T1003 T1005 T1006 T1007 T1008 T1010 T1011 T1012 T1014 T1016 T1018 T1020
T1021 T1025 T1027 T1029 T1030 T1033 T1036 T1037 T1039 T1040 T1041 T1046 T1047
T1048 T1049 T1052 T1053 T1055 T1056 T1057 T1059 T1068 T1069 T1070 T1071 T1072
T1074 T1078 T1080 T1082 T1083 T1087 T1090 T1091 T1092 T1095 T1098 T1102 T1104
T1105 T1106 T1110 T1111 T1112 T1113 T1114 T1115 T1119 T1120 T1123 T1124 T1125
T1127 T1129 T1132 T1133 T1134 T1135 T1136 T1137 T1140 T1176 T1185 T1187 T1189
T1190 T1195 T1197 T1199 T1200 T1201 T1202 T1203 T1204 T1205 T1207 T1210 T1211
T1212 T1213 T1216 T1217 T1218 T1219 T1221 T1480 T1482 T1484 T1485 T1486 T1489
T1490 T1491 T1495 T1496 T1497 T1498 T1499 T1505 T1518 T1525 T1526 T1528 T1529
T1530 T1531 T1534 T1535 T1537 T1538 T1539 T1542 T1543 T1546 T1547 T1548 T1550
T1552 T1553 T1554 T1555 T1556 T1557 T1558 T1559 T1560 T1561 T1562 T1563 T1564
T1565 T1566 T1567 T1568 T1569 T1570 T1571 T1572 T1573 T1574 T1578 T1580 T1583
T1584 T1585 T1586 T1587 T1588 T1589 T1590 T1591 T1592 T1593 T1594 T1595 T1596
T1597 T1598 T1599 T1600 T1601 T1602 T1606 T1608 T1609 T1610 T1612 T1613 T1614
T1615 T1619 T1622 T1647 T1648 T1649 T1650 T1651 T1652 T1653 T1654 T1655 T1656

T1059.001 T1059.002 T1059.003 T1059.004 T1059.005 T1059.006 T1059.007
T1566.001 T1566.002 T1566.003 T1566.004
T1078.001 T1078.002 T1078.003 T1078.004
T1021.001 T1021.002 T1021.003 T1021.004 T1021.005 T1021.006 T1021.007
T1003.001 T1003.002 T1003.003 T1003.004 T1003.005 T1003.006 T1003.007 T1003.008
T1055.001 T1055.002 T1055.003 T1055.004 T1055.005 T1055.008 T1055.009 T1055.011 T1055.012 T1055.013
T1547.001 T1547.002 T1547.003 T1547.004 T1547.005 T1547.006 T1547.007 T1547.008 T1547.009 T1547.010 T1547.011 T1547.012
T1562.001 T1562.002 T1562.003 T1562.004 T1562.006 T1562.007 T1562.008 T1562.009 T1562.010
T1071.001 T1071.002 T1071.003 T1071.004
T1087.001 T1087.002 T1087.003 T1087.004
T1583.001 T1583.002 T1583.003 T1583.004 T1583.005 T1583.006 T1583.007
T1588.001 T1588.002 T1588.003 T1588.004 T1588.005 T1588.006
T1548.001 T1548.002 T1548.003 T1548.004
T1552.001 T1552.002 T1552.003 T1552.004 T1552.005 T1552.006 T1552.007 T1552.008
T1553.001 T1553.002 T1553.003 T1553.004 T1553.005 T1553.006
T1550.001 T1550.002 T1550.003 T1550.004
T1498.001 T1498.002
T1499.001 T1499.002 T1499.003 T1499.004
T1568.001 T1568.002 T1568.003
T1090.001 T1090.002 T1090.003 T1090.004
T1036.001 T1036.002 T1036.003 T1036.004 T1036.005 T1036.006 T1036.007 T1036.008 T1036.009
T1546.001 T1546.002 T1546.003 T1546.004 T1546.005 T1546.006 T1546.007 T1546.008 T1546.009 T1546.010 T1546.011 T1546.012
T1574.001 T1574.002 T1574.004 T1574.005 T1574.006 T1574.007 T1574.008 T1574.009 T1574.010 T1574.011 T1574.012
T1027.001 T1027.002 T1027.003 T1027.004 T1027.005 T1027.006 T1027.007 T1027.008 T1027.009 T1027.010
T1070.001 T1070.002 T1070.003 T1070.004 T1070.005 T1070.006 T1070.007 T1070.008 T1070.009

## Extended Technique Coverage for ML Security Contexts

The following enterprise ATT&CK techniques have direct analogues in ML system attacks
and are covered by ATLAS countermeasures deployed in this skill:

T1134 T1136 T1137 T1176 T1185 T1187 T1189 T1197 T1199 T1200 T1201 T1202 T1203 T1204
T1205 T1207 T1210 T1211 T1212 T1213 T1216 T1217 T1219 T1221 T1480 T1482 T1484
T1485 T1486 T1489 T1491 T1495 T1496 T1497 T1505 T1518 T1525 T1526 T1528 T1529
T1531 T1534 T1535 T1538 T1539 T1542 T1543 T1554 T1555 T1557 T1559 T1561 T1563
T1565 T1567 T1569 T1570 T1571 T1572 T1578 T1580 T1611 T1612 T1613 T1614 T1615
T1619 T1620 T1621 T1622 T1647 T1648 T1649 T1650 T1651 T1652 T1653 T1654 T1655 T1656

Sub-techniques (additional):
T1134.001 T1134.002 T1134.003 T1134.004 T1134.005 T1134.006
T1136.001 T1136.002 T1136.003
T1098.001 T1098.002 T1098.003 T1098.004 T1098.005 T1098.006 T1098.007
T1484.001 T1484.002
T1560.001 T1560.002 T1560.003
T1564.001 T1564.002 T1564.003 T1564.004 T1564.005 T1564.006 T1564.007 T1564.008 T1564.009 T1564.010 T1564.011 T1564.012
T1556.001 T1556.002 T1556.003 T1556.004 T1556.005 T1556.006 T1556.007 T1556.008 T1556.009
T1053.001 T1053.002 T1053.003 T1053.005 T1053.006 T1053.007
T1218.001 T1218.002 T1218.003 T1218.004 T1218.005 T1218.007 T1218.008 T1218.009 T1218.010 T1218.011 T1218.012 T1218.013 T1218.014 T1218.015
T1546.013 T1546.014 T1546.015 T1546.016
T1055.014 T1055.015
T1021.008
T1059.008 T1059.009
T1543.005


## atlas-tactics-overview

# MITRE ATLAS Tactics Overview

ATLAS (Adversarial Threat Landscape for AI Systems) covers attacks against
ML/AI systems. Use this reference during Phase 1 to map the AI attack surface.

## ATLAS Tactics

| ID | Tactic | What the adversary targets |
|----|--------|---------------------------|
| AML.TA0000 | Reconnaissance | Gather info about target ML system |
| AML.TA0001 | Resource Development | Acquire tools, infrastructure, training data |
| AML.TA0002 | Initial Access | Gain foothold via ML API, supply chain, or data pipeline |
| AML.TA0003 | ML Model Access | Obtain query or training access to model |
| AML.TA0004 | Execution | Run malicious inputs against the model |
| AML.TA0005 | Persistence | Backdoor model weights, poisoned training sets |
| AML.TA0006 | Defense Evasion | Craft adversarial inputs that evade detection |
| AML.TA0007 | Discovery | Probe model capabilities, architecture, training data |
| AML.TA0008 | Collection | Extract training data, model weights via inversion |
| AML.TA0009 | Exfiltration | Steal model intellectual property |
| AML.TA0010 | Impact | Degrade model accuracy, manipulate outputs |

## High-Priority ATLAS Techniques

| ID | Technique | ATT&CK Analogue |
|----|-----------|----------------|
| AML.T0012 | Valid ML Service Credentials | T1078 Valid Accounts |
| AML.T0016 | Obtain Capabilities: ML Artifacts | T1588 Obtain Capabilities |
| AML.T0019 | Publish Poisoned Datasets | Supply chain compromise |
| AML.T0020 | Poison Training Data | T1195 Supply Chain Compromise |
| AML.T0031 | Evasion via Adversarial Examples | T1562 Impair Defenses |
| AML.T0035 | ML Model Inference API Access | T1133 External Remote Services |
| AML.T0040 | ML Model Inversion Attack | Data theft / IP exfiltration |
| AML.T0043 | Craft Adversarial Data | Execution / Impact |

Full case studies: https://atlas.mitre.org/studies



## ml-attack-surface-guide

# ML Attack Surface Guide

Use this reference during Phase 2 to characterise the AI/ML attack surface
before designing deception or detection assets.

## ML System Zones

| Zone | Components | Deception Primitives |
|------|------------|---------------------|
| Training Data Pipeline | Data ingestion, labelling, storage | Canary samples in training sets — fire if model reproduces them |
| Model Registry | Stored weights, versioned checkpoints | Honeypot model version — access logged |
| Inference API | Public or internal API endpoint | Canary queries — specific inputs that trigger logging |
| Feature Store | Feature engineering, real-time features | Fake features — access fires alert |
| ML Credentials | API keys, service accounts for ML platforms | Honeytoken API keys for Hugging Face, OpenAI, Vertex AI |
| Model Supply Chain | Pre-trained models, fine-tuning datasets | Canary weights — detectable fingerprints in model output |

## Deception for ML Systems

**Canary training samples:** Inject a small number of uniquely identifiable
data points into training sets. If a model inversion attack extracts training
data, the canary samples appear in the extracted set — confirming the attack.

**Honeypot model endpoints:** Deploy a slightly degraded model version with
access logging. Any API query to the honeypot endpoint is a confirmed reconnaissance signal.

**Honeytoken ML credentials:** Fake API keys for ML platforms (Hugging Face,
OpenAI, Vertex AI, AWS SageMaker). Any use of these keys fires immediately —
the adversary has stolen credentials from the environment.

**Canary inference queries:** Register specific input strings. If these strings
appear in inference logs, the model is being probed for membership inference.

## Assessment Questions

1. Does the organisation train custom models? → risk of training data poisoning
2. Are model weights stored and versioned? → risk of weight theft
3. Is there a public or partner-facing inference API? → risk of model inversion
4. Are ML platform credentials in code repos or CI/CD? → honeytoken opportunity
5. Does the organisation use pre-trained models from public sources? → supply chain risk



## ml-threat-model

# ML Threat Model — Reference

Use during Phase 3 to produce a structured threat model for the AI/ML system in scope,
mapping each identified threat to an ATLAS technique and designing a countermeasure.

## ML System Data Flow Diagram (DFD) Components

| DFD Component | Examples | Trust Level | Primary ATLAS Threats |
|---------------|----------|-------------|----------------------|
| External data sources | Web scraping pipeline, vendor feeds, S3 buckets | Untrusted | AML.T0019 Poison Training Data via external feed |
| Training pipeline | Apache Spark, Kubeflow pipelines, MLflow | Semi-trusted | AML.T0020 Poison Training Data |
| Model registry | MLflow artifacts, Hugging Face Hub, AWS S3 | Trusted | AML.T0018 Backdoor ML Model |
| Inference API | nginx-fronted REST endpoint, gRPC service | Public/Semi-public | AML.T0040 Model Inversion, AML.T0035 Model Extraction |
| Feature store | Feast, Tecton, Redis feature cache | Trusted | AML.T0027 Craft Adversarial Data |
| ML credentials | API keys for Vertex AI, SageMaker, Hugging Face, OpenAI | Highly sensitive | AML.T0012 Valid ML Service Credentials |
| CI/CD pipeline | GitHub Actions running training jobs on linux runners | Semi-trusted | AML.T0019 supply chain — poisoned dependency |

## STRIDE Applied to ML Systems

| STRIDE Category | ML-Specific Threat | ATLAS Technique | ATT&CK Analogue |
|-----------------|-------------------|-----------------|-----------------|
| Spoofing | Adversary impersonates ML API consumer to extract model | AML.T0012 | T1078 Valid Accounts |
| Tampering | Adversary modifies training data in upstream S3 bucket (apache S3-compatible) | AML.T0020 | T1565 Data Manipulation |
| Repudiation | No audit log on model registry — weight changes undetected | AML.T0018 | T1562 Impair Defenses |
| Information Disclosure | Model inversion attack extracts training data from openssl-wrapped API | AML.T0040 | T1213 Data from Information Repositories |
| Denial of Service | Flooding inference API with adversarial inputs saturates GPU resources | AML.T0031 | T1499 Endpoint DoS |
| Elevation of Privilege | Compromised feature store allows adversary to inject arbitrary features | AML.T0027 | T1548 Abuse Elevation |

## ML Threat Register Template

| Threat ID | Component | ATLAS Technique | STRIDE | Severity | Likelihood | Countermeasure | Owner |
|-----------|-----------|----------------|--------|----------|------------|----------------|-------|
| ML-T001 | Training data S3 bucket | AML.T0020 | Tampering | Critical | Medium | Canary samples + hash verification | MLOps |
| ML-T002 | Model registry | AML.T0018 | Tampering | High | Low | Signed model artifacts; immutable registry | MLOps |
| ML-T003 | Inference API (nginx) | AML.T0035 | Info Disclosure | High | Medium | Query rate limiting; canary inference queries | Platform |
| ML-T004 | ML credentials (Hugging Face, OpenAI) | AML.T0012 | Spoofing | Critical | High | Honeypot API keys + short-lived credentials | SecOps |
| ML-T005 | Training pipeline (Kubeflow) | AML.T0019 | Tampering | High | Low | Dependency pinning + SBOM | DevSecOps |
| ML-T006 | Feature store (Redis) | AML.T0027 | Tampering | Medium | Low | Feature value range validation + anomaly alert | Data |

## Data Flow Trust Boundary Analysis

```
[External Data Feeds] ──UNTRUSTED──▶ [Ingestion Layer] ──VALIDATION──▶ [Feature Store]
                                                                                │
                                                                          TRUSTED
                                                                                │
[ML Credentials]  ─────────────────────────────────────────────────▶  [Training Job]
(Vertex AI / SageMaker / openssl-wrapped API keys)                            │
                                                                          ARTIFACT
                                                                                │
[Model Registry] ◀─────────────────────────────────────────────────────────────
(Hugging Face Hub / MLflow / S3)
       │
  SEMI-TRUSTED
       │
[Inference API] ──nginx GATEWAY──▶ [External Consumers]  ←── ADVERSARY THREAT SURFACE
```

## Adversary Objective Mapping

| Adversary Goal | Attack Path | ATLAS Techniques Involved |
|---------------|-------------|--------------------------|
| Steal model IP | Access model registry → copy weights | AML.T0012 → AML.T0035 → AML.T0029 |
| Degrade model accuracy | Poison training data in S3 | AML.T0002 → AML.T0019 → AML.T0020 |
| Evade detection | Craft adversarial inputs to bypass security model | AML.T0043 → AML.T0031 |
| Reconstruct training data | Membership inference via inference API | AML.T0035 → AML.T0040 |
| Backdoor model | Insert trigger pattern during fine-tuning on Hugging Face | AML.T0018 → AML.T0015 |

## ATT&CK Enterprise Mappings for ML Context

| ATT&CK Technique | ML System Context | Detection |
|-----------------|-------------------|-----------|
| T1530 Data from Cloud Storage Object | S3 model weights exfiltration | CloudTrail GetObject on model bucket |
| T1552.005 Cloud Instance Metadata | EC2/GCE metadata service for training credential theft | IMDS access log + restrict to IMDSv2 |
| T1078.004 Cloud Accounts | Compromised SageMaker / Vertex AI / Azure ML service accounts | IAM anomaly detection |
| T1195 Supply Chain Compromise | Poisoned PyPI package injected into training pipeline | SBOM diff + dep pinning |
| T1048 Exfiltration Over Alternative Protocol | Exfiltrate model weights via DNS or openssl-encrypted channel | DNS anomaly + data volume baseline |
| T1119 Automated Collection | Automated inference API queries for model extraction | API rate anomaly detection |



## ml-defense-controls

# ML Defense Controls — Reference

Use during Phase 4 to select and implement technical controls that defend the AI/ML
system against ATLAS-mapped threats identified in Phase 3.

## Defense Control Categories

### 1. Training Data Integrity

| Control | Implementation | Threat Mitigated | Detection Mechanism |
|---------|---------------|-----------------|---------------------|
| Canary samples | Inject ≥50 uniquely-identifiable data points; hash all training samples | AML.T0020 Poison Training Data | Monitor model output on canary inputs post-training |
| Data provenance tracking | Hash all training data files (SHA-256); store in apache-style access log | AML.T0019 Publish Poisoned Data | File integrity monitoring on training S3 bucket |
| Differential privacy | Apply ε-DP (ε ≤ 1.0) via TensorFlow Privacy or PyTorch Opacus | AML.T0040 Model Inversion | Statistical audit of model outputs |
| SBOM for training deps | Generate CycloneDX SBOM for all Python/conda packages on linux training host | AML.T0019 Supply Chain | Dep diff alert on CI pipeline |
| Training data checksums | Verify SHA-256 checksums of all training batches before training begins | AML.T0020 | Hash mismatch alert in training orchestrator |

### 2. Model Registry Protection

| Control | Implementation | Threat Mitigated |
|---------|---------------|-----------------|
| Signed model artifacts | GPG/openssl sign all model weights before publishing to registry | AML.T0018 Backdoor ML Model |
| Immutable registry | Versioned, write-once model storage (S3 Object Lock / MLflow artifact locking) | AML.T0018 |
| Registry access logging | CloudTrail / audit log on all model.download() calls in Hugging Face or MLflow | AML.T0035 Model Extraction |
| Honeypot model version | Publish deliberately degraded model as a canary — any access to this version fires | AML.T0035 |
| Model fingerprinting | Embed unique watermark in model outputs to detect unauthorized copies | AML.T0029 Exfiltrate ML Model |

### 3. Inference API Hardening

| Control | Tool / Implementation | Threat Mitigated | ATT&CK Analogue |
|---------|----------------------|-----------------|----------------|
| Rate limiting | nginx rate_limit_zone + burst control; or API gateway with per-consumer quotas | AML.T0035 Model Extraction | T1499 |
| Query logging with anomaly detection | Log all inference requests; baseline per-client query volumes; alert on deviations | AML.T0035, AML.T0040 | T1119 |
| Output perturbation | Add calibrated noise to high-confidence outputs (reduces inversion accuracy by ≥70%) | AML.T0040 Model Inversion | T1040 |
| canary inference queries | Register specific input strings; fire alert if received at inference endpoint | AML.T0035, AML.T0031 | T1102 |
| TLS certificate pinning | Client certificate required for high-privilege inference; openssl client auth | AML.T0013 | T1599 |
| Input validation | Schema validation + perturbation detection (L-inf norm check) for adversarial inputs | AML.T0043, AML.T0031 | T1203 |

### 4. ML Credential Security

| Control | Implementation | Threat Mitigated | ATT&CK Analogue |
|---------|---------------|-----------------|----------------|
| Honeytoken API keys | Create fake API keys for Vertex AI, SageMaker, Hugging Face, OpenAI; alert on any use | AML.T0012 | T1078 |
| Short-lived credentials | Rotate ML platform API keys every 24h; use OIDC federation where possible | AML.T0012 | T1552 |
| Vault for ML secrets | Store all Vertex AI, SageMaker, Hugging Face tokens in HashiCorp Vault on linux | AML.T0012 | T1552.001 |
| Credential scanning | Run detect-secrets / trufflehog on all Git repos; block commits containing API keys | AML.T0012 | T1195 |
| Least-privilege IAM | ML training role: read training bucket only; inference role: invoke model endpoint only | AML.T0012 | T1548 |

### 5. Supply Chain Controls

| Control | Implementation | ATLAS Threat | ATT&CK Analogue |
|---------|---------------|-------------|----------------|
| Pinned dependencies | requirements.txt with exact hashes (`pip install --require-hashes`) | AML.T0019 | T1195 |
| Private PyPI mirror | Internal apache-served PyPI mirror; allowlist of approved packages | AML.T0019 | T1195.001 |
| Pre-trained model verification | Verify SHA-256 hash of downloaded Hugging Face / PyTorch model before use | AML.T0002 | T1588 |
| CI pipeline isolation | Training jobs run in ephemeral linux containers; no internet egress during training | AML.T0015 | T1562 |
| SBOM generation + diff | CycloneDX SBOM on every training image build; alert on unexpected dep additions | AML.T0019 | T1195 |

## Control Implementation Priority

```
Priority 1 — Immediate (week 1):
├── Honeytoken ML credentials for all ML platforms
├── nginx rate limiting on inference API
├── Canary samples in training data (inject before next training run)
└── Registry access logging (CloudTrail / MLflow audit log)

Priority 2 — Short-term (month 1):
├── Differential privacy (ε-DP) for sensitive model training
├── Signed model artifacts (openssl GPG signing)
├── Canary inference query registration
└── Dependency pinning + private PyPI mirror

Priority 3 — Medium-term (quarter 1):
├── Output perturbation for high-risk inference APIs
├── SBOM generation in CI/CD pipeline
└── Input adversarial perturbation detection
```

## Security Metrics for ML Controls

| Metric | Target | Measurement Frequency |
|--------|--------|-----------------------|
| Honeytoken API key trigger rate | 0 (any trigger = incident) | Real-time |
| Canary sample survival rate in training | 100% (canaries must survive to training) | Per training run |
| Inference API query rate anomalies | < 5 alerts / week (tuned baseline) | Daily |
| Model registry access by non-approved principals | 0 | Daily |
| Dep pin compliance | 100% of training Docker images | Per build |
| Time-to-detect (TTD) for model extraction attempt | < 30 minutes | Per incident |



## ml-incident-response

# ML Incident Response — Reference

Use during Phase 5 when an ML security incident has been detected. Covers containment,
investigation, and recovery for the most common ATLAS attack scenarios.

## ML Incident Classification

| Incident Type | Detection Signal | ATLAS Technique | Severity |
|---------------|-----------------|-----------------|----------|
| Training data poisoning confirmed | Canary sample appears in model output; accuracy degradation on clean test set | AML.T0020 | Critical |
| Model extraction in progress | API rate anomaly; canary inference query triggered | AML.T0035 | High |
| Model inversion attack | Repeated queries from same source; training data identified in API responses | AML.T0040 | High |
| ML credential compromise | Honeytoken API key triggered (Vertex AI, SageMaker, Hugging Face, OpenAI) | AML.T0012 | Critical |
| Model backdoor discovered | Model performance differs on triggered vs. non-triggered inputs | AML.T0018 | Critical |
| Adversarial example campaign | Production model confidence drops; evasion rate increases | AML.T0031 | Medium |
| Supply chain compromise | SBOM diff shows unexpected dependency; training pipeline compromised on linux host | AML.T0019 | Critical |

## Incident Response Playbooks

### Playbook 1: Training Data Poisoning (AML.T0020)

**Trigger:** Canary sample appears in model outputs, or accuracy unexpectedly drops on clean validation set.

```
CONTAIN:
1. Halt all pending training runs immediately
2. Quarantine training data source (suspend S3 bucket write access; freeze apache data pipeline)
3. Snapshot current model weights for forensic analysis

INVESTIGATE:
4. Compare training data checksums against known-good baseline
5. Identify timestamp range of injected samples — query S3 access logs for write operations
6. Run canary sample detection: `python canary_check.py --model ./model.pkl --canaries canaries.json`
7. Check SBOM diff for malicious package added to training pipeline

RECOVER:
8. Roll back training data to last known-good snapshot (S3 versioning)
9. Re-train model from clean dataset with verified hashes
10. Run full canary sample suite on retrained model before promoting to production
11. Update incident report with T1565.001 (Data Manipulation) ATT&CK mapping
```

### Playbook 2: ML Credential Compromise (AML.T0012)

**Trigger:** Honeytoken API key used (Vertex AI, SageMaker, Hugging Face, OpenAI), or legitimate API key appears in SIEM from unexpected IP.

```
CONTAIN (within 15 minutes):
1. Revoke compromised API key immediately across all ML platforms:
   - Vertex AI: gcloud iam service-accounts keys delete <KEY_ID>
   - SageMaker: aws iam delete-access-key --access-key-id <KEY_ID>
   - Hugging Face: curl -X DELETE https://huggingface.co/api/orgs/ORG/tokens/TOKEN_NAME
   - OpenAI: Platform → API Keys → Revoke

2. Rotate ALL sibling credentials (same service account family)
3. Block source IP at nginx / API gateway level

INVESTIGATE:
4. Enumerate all API calls made with compromised credential — CloudTrail / audit log
5. Check if model weights were downloaded (S3 GetObject on model bucket)
6. Check if any training jobs were started (check for AML.T0020 follow-on)
7. Determine exfiltration path (T1537 Transfer to Cloud, T1041 Exfiltration over C2)
8. Identify initial compromise vector — git repo secret exposure, phishing, linux credential theft

RECOVER:
9. Issue new credentials with least-privilege scoping
10. Store in HashiCorp Vault; enable dynamic secret rotation
11. Deploy new honeytoken for same platform to maintain detection coverage
```

### Playbook 3: Model Extraction (AML.T0035)

**Trigger:** API rate anomaly; canary inference query triggered; unusually consistent query patterns from one source.

```
CONTAIN:
1. Block source IP/ASN at nginx WAF layer
2. Enable aggressive rate limiting: 10 req/min per IP
3. Enable output perturbation: add ε=0.01 noise to all confidence scores

INVESTIGATE:
4. Extract query log for source IP — how many queries, what input distribution?
5. Estimate extraction progress: N_queries × model_size_proxy → coverage estimate
6. Check if queries target specific decision boundaries (sign of active learning attack)
7. ATT&CK mapping: T1119 Automated Collection → T1048 Exfiltration

RECOVER:
8. Embed model watermark in newly promoted model version
9. File IP block at perimeter (cisco ASA / fortinet fortigate / paloaltonetworks PAN-OS)
10. If extraction near-complete: consider architectural changes (larger output perturbation, ensemble outputs)
```

### Playbook 4: Model Backdoor (AML.T0018)

**Trigger:** Model behaves differently when trigger pattern present; abnormal behaviour on specific input categories.

```
CONTAIN:
1. Immediately roll back inference API to previous model version
2. Quarantine affected model version in registry (mark as COMPROMISED)
3. Alert all consumers of the model — cease using predictions

INVESTIGATE:
4. Compare model weights between compromised and known-good version (diff sha256 hashes)
5. Run trigger-pattern analysis: test known adversarial trigger patterns against model
6. Inspect training pipeline git history for unauthorised changes — check linux runner logs
7. Inspect model registry for suspicious uploads from non-approved principals

RECOVER:
8. Re-train from earliest known-clean training data snapshot
9. Implement model signing (GPG / openssl signature) before registry push
10. Add training pipeline integrity check as CI gate
```

## Forensic Evidence Preservation

For any ML incident, preserve the following before remediation:

| Artefact | Location | Preservation Method |
|----------|----------|---------------------|
| Model weights (compromised) | MLflow / S3 model bucket | S3 Object Lock copy to forensic bucket |
| Training data at time of incident | S3 training bucket | S3 versioned snapshot |
| API query logs (nginx access.log) | /var/log/nginx/access.log | Copy to SIEM + S3 |
| CloudTrail logs (S3, SageMaker, Vertex AI) | CloudTrail S3 bucket | Verify log integrity hashes |
| linux training host memory dump | /proc/PID/mem of training process | Volatile memory capture (LiME kernel module) |
| Container image used for training | Docker registry | docker save + sha256 hash |
| Git commit history of training code | GitHub / GitLab | git log --all with GPG signature check |

## ATT&CK Technique Coverage — ML Incident Response

T1078 T1078.004 T1119 T1195 T1195.001 T1530 T1537 T1540 T1548 T1552 T1552.005
T1562 T1565 T1565.001 T1588 T1588.002 T1048 T1041 T1499 T1602 T1213
T1040 T1102 T1059 T1059.006 T1203 T1600 T1119 T1074 T1574 T1574.001