Your agency just received the DeepSeek removal order. Now you're staring at a spreadsheet of vendor tools, wondering which ones might run on Chinese-developed models you've never heard of. You need a systematic way to assess every AI tool in your environment before the next ban drops.
This template provides a repeatable framework for evaluating third-party AI products against supply chain security criteria. It's built around the risk factors cited in the DeepSeek order: data transfer pathways, model provenance, infrastructure location, and vendor transparency.
Purpose of This Template
Use this assessment when:
- Evaluating new AI vendor proposals
- Auditing existing AI tools after a supply chain security incident
- Responding to exclusion orders under the Federal Acquisition Supply Chain Security Act of 2018
- Building your agency's AI procurement standards
The template produces a risk score and a go/no-go recommendation. It doesn't replace legal review, but it gives your procurement and security teams a common language for discussing AI supply chain risk.
Prerequisites
Before scoring vendors, ensure you have:
- Asset inventory: A complete list of AI tools, including those embedded in larger platforms (your CRM's chatbot counts)
- Vendor contact list: Names and email addresses for technical contacts who can answer architecture questions
- Data classification schema: Know which systems touch sensitive data so you can weight risk appropriately
- Authority to act: Confirm who can approve exceptions, deny procurements, or mandate removals in your agency
If you're missing the asset inventory, start there. You can't assess what you can't see, and most agencies discover shadow AI deployments during these reviews.
The Template
Copy this into your spreadsheet or GRC platform. Score each criterion 0-3, where 0 is lowest risk and 3 is highest. Total scores above 15 warrant immediate review; scores above 20 should trigger removal or rejection.
Section 1: Model Provenance (Weight: 2x)
- Training data origin: Does the vendor disclose where training data came from? (0=full disclosure, 3=no disclosure)
- Model development location: Where was the model developed? (0=domestic, 1=allied nation, 2=neutral nation, 3=adversarial nation)
- Open-weight derivatives: Is this built on open-weight models from high-risk sources? (0=proprietary only, 1=reputable open source, 2=unclear lineage, 3=known high-risk base model)
Section 2: Data Handling (Weight: 2x)
- Data residency: Where does user input and output data reside? (0=on-premises or FedRAMP environment, 1=domestic cloud, 2=allied nation cloud, 3=adversarial nation or undisclosed)
- Data retention: How long does the vendor keep prompts and responses? (0=ephemeral/configurable, 1=30 days, 2=90+ days, 3=permanent or undisclosed)
- Third-party sharing: Does the vendor share data with model developers or other parties? (0=contractually prohibited, 1=anonymized only, 2=aggregated, 3=yes or unclear)
Section 3: Infrastructure and Access (Weight: 1x)
- Server location: Where do inference requests run? (0=on-premises, 1=domestic cloud, 2=allied nation, 3=adversarial nation or multi-region)
- Access controls: Can you enforce network-level restrictions? (0=air-gapped or VPC, 1=IP allowlisting, 2=authentication only, 3=public internet required)
- Vendor access: Does the vendor have standing access to your data? (0=no access, 1=audit logs only, 2=support access with approval, 3=unrestricted)
Section 4: Vendor Transparency (Weight: 1x)
- Security documentation: Does the vendor provide architecture diagrams and data flow documentation? (0=comprehensive, 1=basic, 2=on request only, 3=none)
- Compliance certifications: What third-party attestations exist? (0=SOC 2 Type II + FedRAMP, 1=SOC 2 Type II, 2=self-attestation, 3=none)
- Incident disclosure: Does the vendor have a public security incident history? (0=transparent disclosure program, 1=incidents disclosed reactively, 2=no known incidents but no program, 3=known incidents with poor disclosure)
Section 5: Legal and Jurisdictional Risk (Weight: 2x)
- Foreign ownership: What percentage of the company is foreign-owned? (0=domestic, 1=<25% foreign, 2=25-50%, 3=>50% or adversarial nation)
- Legal jurisdiction: Which courts govern disputes? (0=U.S. federal, 1=allied nation, 2=neutral nation, 3=adversarial nation)
- Government access laws: Is the vendor subject to foreign intelligence laws requiring data sharing? (0=no, 1=allied nation with due process, 2=unclear, 3=yes, adversarial nation)
Customizing the Template
Adjust weights for your risk profile: If you handle classified information, multiply the Data Handling section by 3x instead of 2x. For public-facing services with no sensitive data, you might drop Infrastructure scoring to 0.5x.
Add agency-specific criteria: Insert rows for requirements like "supports CAC authentication" or "provides audit logs in SIEM-compatible format." These operational criteria don't affect supply chain risk scoring but help with procurement decisions.
Set score thresholds based on data classification: A tool processing public information might be acceptable at score 18, while anything touching personally identifiable information should stay below 12.
Create vendor-specific questionnaires: Use the template categories to build a Request for Information. Send it before you start scoring. Most vendors can't answer these questions off the shelf, so give them two weeks.
Validation Steps
After scoring your vendors:
Cross-reference with CISA advisories: Check if any vendors appear in Cybersecurity and Infrastructure Security Agency alerts or supply chain risk notices. If a vendor is already flagged, your exception request faces longer odds.
Review data flow diagrams: Ask vendors to map exactly where data goes when a user submits a prompt. If they say "our secure cloud environment," that's a red flag. You need region names, service provider names, and encryption details.
Test with non-sensitive data: Before allowing production use, run the tool with synthetic data and monitor egress traffic. Look for unexpected connections to foreign IP ranges or data volumes that don't match the stated architecture.
Document your risk acceptance: For any tool scoring above your threshold that you keep anyway, write a risk acceptance memo explaining why. Include compensating controls like network segmentation or data sanitization.
Set review intervals: AI vendors change ownership, move infrastructure, and add features constantly. Re-score quarterly for high-risk tools, annually for everything else.
The Federal Acquisition Supply Chain Security Act gives DHS, DoD, and DNI broad authority to exclude risky technology. The law doesn't require proof of actual compromise, just "multiple threat vectors" that could produce intelligence on U.S. persons or agencies. Your assessment template won't prevent every ban, but it'll keep you ahead of the obvious risks and provide documentation when auditors ask why you chose a particular vendor.
