Regulated AI on GPUs: What to Check Before You Host in 2026

Why GPU capacity is not a compliance plan

Graphics processing units (GPUs) provide the computing power to train and run artificial intelligence (AI) models. Renting that power does not mean the hosted application meets its legal, contractual or security requirements. Sensitive information also ends up in application logs, in backups, and in external model services that your application calls.

For each storage location and external service, AI teams need to know three things: who can access the sensitive information, how long it is retained, and whether the customer or the provider is responsible for protecting it. The contracts should say so in writing, and the team should hold records showing that the required controls are in place.

This guide explains what engineering and procurement teams should check before hosting regulated AI workloads on GPUs.

Follow the data

Consider a medical-imaging application that uses a GPU to analyze images and produce findings. In this example the application uses synthetic images and invented patient details, with no real patient or customer data. Its processing steps still show where sensitive information would travel if it handled real patient records.

Before introducing sensitive data, document each storage location and external destination, why information is needed there, and when it is deleted. This record shows which services need security controls and which providers need contractual agreements.

Three documents, three jobs

Providers may hand AI teams a HIPAA Business Associate Agreement (BAA), System and Organization Controls (SOC) reports, or evidence of Payment Card Industry Data Security Standard (PCI DSS) validation. These are not interchangeable.

HIPAA and your application's information

The Health Insurance Portability and Accountability Act (HIPAA) requires covered entities and business associates to protect electronic Protected Health Information (ePHI). For AI teams building GPU applications for these organizations, that protection extends to any ePHI the application processes or keeps in storage, logs and backups.

If ePHI is sent to a hosting provider or an external model service, the provider's role has to be defined. A provider that creates, receives, maintains or transmits ePHI on behalf of a covered entity or business associate is itself a business associate. That includes storing encrypted ePHI without holding the decryption key. HHS cloud computing guidance therefore calls for a BAA.

Signing a BAA does not replace risk analysis or required security controls. It is a contractual agreement, not a product certification, and the U.S. Department of Health and Human Services (HHS) does not certify GPU services as HIPAA-compliant. Before deployment, confirm which controls the customer and the provider must operate under the current HIPAA Security Rule. Base the assessment on existing requirements, and track proposed rule changes separately for future planning.

SOC reports and hosting services

A SOC report is relevant only if its scope includes the proposed service. A report covering another service does not give assurance about the GPU environment.

The report type matters too. SOC 2 Type I assesses control design at a specified date. Type II also assesses operating effectiveness over a defined period. SOC 3 gives less detail and is meant for general use.

Review the findings and the customer responsibilities before approving a report. That shows you where control issues exist and which safeguards your team has to manage itself. For background, see the AICPA overview of SOC services.

PCI DSS and payment data

An AI application may process payment-account data or connect to systems inside the cardholder-data environment. In either case, establish whether the GPU workload falls within PCI DSS scope, using the current standard in the PCI SSC document library.

Even when a hosting service has been validated, that validation does not automatically cover the customer's application. Your validation, engineering and procurement teams should confirm which requirements the provider meets and which you must meet through application configuration and operating procedures.

Shared, dedicated or bare metal?

Confirm first what you are actually renting.

For the service you are considering, the provider should specify the GPU model, GPU memory, processor resources, storage and deployment region. Confirm whether the physical host is shared, and who has administrative access to the GPU instance, including permission to install software, change system settings and reach stored data.

Permission to manage an instance does not mean the provider maintains its software. The service agreement should state who supplies software licenses and who maintains the operating system, GPU drivers, containers and application. Support terms should say which hardware and software problems the provider handles. Those answers show which tasks need your own staff or an additional management service.

The controls that matter

Sensitive information can reach storage, logs, backups and external services during GPU processing, so safeguards must cover the whole application environment.

• Access and network restrictions

  Limit user and service accounts to the permissions they need. Restrict administrative access and application connections, including outbound connections to model APIs.

• Encryption and key control

  Encrypt sensitive data in transit using TLS and protect it at rest. Confirm who controls the encryption keys. Storage encryption does not protect plaintext data while it is processed in GPU memory.

• Logs and software maintenance

  Keep unnecessary sensitive prompts and outputs out of logs. Take containers and packages from trusted sources and record their versions. Assign responsibility for patches and update testing to your team or the provider.

• Model and dataset permissions

  Confirm that dataset and model licenses permit your intended use. Restrict access to model weights and checkpoints when they contain confidential information.

• Retention, deletion and backups

  Set retention periods for inputs, outputs, temporary files and backup copies, and make deletion procedures cover them all. Run restore tests to confirm the application, models and required data can be recovered.

Some controls follow legal or contractual requirements, and others are technical precautions based on workload risk. Document which is which, and assign an owner to each control.

What to ask for in writing

General compliance claims may refer to a different service or configuration. Ask for documents that describe the environment you are buying.

• Coverage

  For ePHI workloads, the BAA should identify the covered services. Assurance reports should state the assessed system, the date or period, the findings and customer responsibilities. External model providers that receive sensitive information also need contractual and security checks.

• Control ownership

  A responsibility matrix should name who manages access, patches, logs, backups and recovery. Service terms should identify deployment regions and subcontractors, and explain how provider support access is authorized and recorded.

• Incidents and recovery

  Incident procedures should say whom the provider notifies, how fast, and who preserves evidence. Recovery commitments should state the Recovery Point Objective (RPO), the acceptable window of data loss, and the Recovery Time Objective (RTO), the target time to restore service. Restore-test records should show whether those targets can be met.

• Exit terms

  Explain how data is exported and when stored copies are deleted, including information held in logs and backups.

Before approval, separate legal and contractual obligations from workload-specific technical precautions. Resolve any missing evidence or unassigned responsibility before sensitive information enters the environment.

The procurement checklist

Before approving GPU hosting, the AI team should:

• Map where sensitive data is stored or sent, including external inference APIs, and confirm applicable contracts and jurisdictions.
• Confirm who can access data and systems, and how customer and provider access is authorized and recorded.
• Check that the responsibility matrix and service schedule assign an owner to every security control.
• Verify backup coverage and restore-test results against the agreed RPO and RTO for the application, models and required data.
• Confirm data export, retention and deletion terms for storage, logs and backups when the service ends.

Conclusion

Adequate GPU capacity is necessary for an AI workload, but it does not make a hosting arrangement suitable for regulated data. That also depends on how the application handles sensitive information, and on whether the customer and the providers fulfill their contractual and security responsibilities.

Before deployment, confirm that the proposed services meet these requirements and that every control has an owner. If data handling, service scope or control ownership is unclear, approval should wait until it is resolved. That gives the team a sound basis for introducing sensitive information into the environment.