The adoption of AI in healthcare is gathering speed. In recent months, we explored key issues in artificial intelligence, including privacy and safety risks, especially around patient consent and transcription tools, and noted serious threats posed by autonomous AI agents that could expose vulnerabilities in legacy healthcare systems.

For compliance managers and IT leaders, this leads to a challenge: How do we manage the risk of AI we didn’t create but use every day?

Healthcare professionals purchase AI-powered software, including EHRs with analytics, billing platforms, denial-prediction tools, and transcription tools. These introduce supply chain risk: if an underlying AI model fails or leaks data, your organization is liable.

To address these risks, the Health Sector Coordinating Council (HSCC) released a new guide for healthcare organizations on third-party AI risk and supply chain transparency. The Third-Party AI Risk and Supply Chain Transparency Guide is a comprehensive blueprint for healthcare organizations of all sizes.

From the Guide:

“The healthcare sector’s accelerating adoption of artificial intelligence has dramatically expanded its dependence on third-party tools and services, introducing complex cybersecurity challenges that traditional risk management models cannot adequately address.”

This blog summarizes key takeaways from the HSCC guidance and offers actionable steps you can take today to reduce the risk of a supply chain breach originating in the “black box.”

The Guide suggests that its best practices are scalable, and implementation will look different for each organization, depending on its size and resources:

Small/rural organizations: Baseline requirements and minimum question-sets to establish foundational protections

Mid-sized organizations: Enhanced controls and more comprehensive vendor assessments

Large/complex organizations: Advanced risk stratification, extensive validation protocols, and sophisticated governance structures

All organizations: Share strategies amongst other healthcare organizations and with the HSCC AI Task Group.

Why Traditional Vendor Risk Management Is Failing

Vendors that create, receive, maintain, or transmit Protected Health Information (PHI) are HIPAA business associates. Covered entities must conduct due diligence to ensure their vendors are following HIPAA. Historically, business associate due diligence was straightforward. You asked whether the vendor complied with HIPAA and whether they’d conducted a risk analysis, and you ensured that a Business Associate Agreement (BAA) was in place.

AI changes this model. Traditional software is “static,” meaning it only does exactly what its code says. AI, on the other hand, is “probabilistic”: it learns from new data, changes over time (this is called “drift”), and can make decisions that are difficult to understand – a process often called the “Black Box.” Sometimes, even the vendor who sells the software can’t fully explain how the AI makes its choices.

The HSCC Guide highlights a transparency gap in the AI supply chain. “Shadow AI” tools—unknown AI functions embedded in vendors’ products—often appear within healthcare organizations. If vendors use third-party AI, your data may pass through multiple layers of processing. A breach or bias upstream can disrupt your compliance system.

Key Pillars of the HSCC Guidance

The HSCC guidance aligns with the NIST AI Risk Management Framework (RMF) and adds a clinical “healthcare-first” lens. Here are the core pillars every compliance officer should know:

The AI Bill of Materials (AIBOM)

Like a drug’s ingredients list, HSCC proposes an inventory called the AI Bill of Materials (AIBOM) that lists all key AI components used in a product. Who built the model? What data was used? Which application programming interfaces (APIs, which let software programs talk to each other) are involved? Without this list, you can’t conduct a proper HIPAA Risk Analysis because the data flow remains unclear.

Model Cards and “Fact Sheets.”

Organizations should ask vendors for Model Cards, standardized documents that explain an AI’s intended function, capabilities, and limitations, along with key information about its development and use.

The intended use of the AI.
The model’s limitations (where it fails).
The demographic balance of the training data (ensuring the data used to train the AI is representative of diverse groups to prevent bias).
The frequency of updates.

Data Provenance and Lineage

HIPAA requires knowing where PHI resides. The HSCC Guide stresses the importance of tracking data flow through the AI model. This includes knowing whether your patient data is used to train future AI models, a practice that can pose a privacy risk unless you have obtained explicit patient consent. Data provenance (where information comes from) and lineage (its journey through systems) help ensure compliance.

The HIPAA Connection: “Reasonable and Appropriate” Safeguards

Although the HSCC guidance does not change HIPAA law, it offers suggestions that can help reduce your risks and strengthen your compliance. Under the HIPAA Security Rule, covered entities and business associates must implement “reasonable and appropriate” safeguards. In 2026, the Office for Civil Rights (OCR) and the courts increasingly look to sector-specific guidance, such as the HSCC’s, to determine what “reasonable” means.

If you experience a breach caused by a third-party AI tool and haven’t conducted even the minimal due diligence outlined by the HSCC, the OCR could argue that your Risk Analysis was “insufficient,” a finding that has recently resulted in millions of dollars in fines under OCR’s Risk Analysis Initiative.

Remember, the Guide describes its suggestions as scalable, so healthcare organizations are not expected to adopt all of them overnight. Look for ways to improve your compliance, not to make it perfect.

Three Practical Suggestions for Healthcare Professionals

Reading 109 pages of guidance is a tall order for a busy practice owner or IT manager. Here are three immediate, concrete steps to reduce third-party AI risks.

I. Conduct an AI Asset Inventory to Combat Shadow AI

You cannot protect what you don’t know exists. The HSCC Guide notes that most organizations have an incomplete vendor inventory.

The Action: Send a Supplemental AI Questionnaire to all software vendors. Ask: “Do you use AI or machine learning (computer systems that learn from data) in any part of your service? If so, what is the underlying model (the core AI technology used)?”
The Goal: Make a list of every AI-enabled tool in your organization. This is the first step toward moving from “Shadow AI” to “governed AI.”

II. Update Your Vendor Risk Assessment

Do not use outdated vendor risk assessments designed before modern AI tools were available. The Guide recommends asking about model drift (when AI performance changes over time) and explainability (how understandable AI decisions are to users).

When evaluating an AI vendor, request a Model Card. Its absence is a red flag. Also ask how performance is monitored over time.
The Goal: Ensure the vendor has a process for “Post-Deployment Monitoring” so the tool doesn’t start making dangerous clinical or administrative errors 6 months after installation.

III. Establish a Team Approach

AI risk isn’t just an IT problem; it’s a legal, clinical, and compliance problem.

The Action: Form a small group with a compliance manager, IT lead, clinical representative, and legal counsel. Use the HSCC’s AI Governance Maturity Model from the guide to assess your AI risk management progress quarterly.
The Goal: Shift AI from being a tech project to being a core part of your risk management strategy.

The “Black Box” Problem: Why Transparency Matters

The HSCC guide emphasizes “Explainability,” which means understanding why artificial intelligence makes decisions. In healthcare, we cannot accept a “trust me” approach from vendors. For example, if an AI tool recommends a treatment or flags a patient as high-risk, the clinician must understand the reasoning behind the recommendation or the flag.

The guide warns that “Black Box” models – AI systems whose workings even developers do not fully understand – pose a major safety risk. Transparency, or a clear understanding of how AI works, is not only about security; it also affects clinical outcomes. If a model is trained on data that does not fairly represent your patient population, its advice could be biased (unfair or inaccurate) or simply incorrect.

Conclusion: Proactive Compliance is the Only Defense

As we discussed in our previous post on the Stryker cyberattack, the supply chain is the new frontline of healthcare cybersecurity. The HSCC’s new guidance gives us the tools to fight back, but only if they are implemented.

For compliance managers and lawyers, the message is clear: The “reasonable and appropriate” standard for HIPAA has evolved. You are now expected to peer into the AI supply chain, insist on transparency from your vendors, and treat AI as a dynamic risk that requires persistent monitoring.

The HIPAA E-Tool^®is designed to handle these complexities. From managing your AI vendor inventory to ensuring your BAAs are up to date with the latest 2026 standards, we help you turn “catastrophic risk” into manageable and compliant operations.

Don’t wait for a “cascading failure” to realize your AI supply chain is broken. Start your AI inventory today.

AI in the Supply Chain: HSCC Guide Helps Manage the Black Box