SPIFFE for AI Workloads
Workload identity is the missing security layer in most AI deployments. Here's how to add it.
Most AI deployments have a security model that looks like this: the inference endpoint is behind a VPC. Anyone inside the VPC can call it.
This is not wrong — it's just incomplete. It doesn't cover the case where something inside the VPC shouldn't be calling the inference endpoint. It doesn't cover what happens when a compromised container starts making requests. It doesn't give you an audit trail of which service called what, when, with what attestation evidence.
The tooling for workload identity in ephemeral compute didn't exist at the level AI workloads require. SPIFFE/SPIRE changes that.
Why AI workloads are different
Traditional services run for months on the same instance. You can manage their certificates manually or with a long-lived rotation schedule. AI inference servers — vLLM on GPU nodes, for example — might run for 8 hours and then get replaced as the autoscaler responds to demand.
During those 8 hours, the inference server needs to authenticate to a vector database, a document store, and a downstream model gateway. If you're doing mTLS, that means certificates. If those certificates are managed manually or with a rotation policy designed for long-lived services, you're not rotating them fast enough.
SPIRE's workload API handles this. The SPIRE Agent runs on every node. It attests the node's platform via the platform's metadata service — AWS IID, GCP MT, Azure MSI. Then it issues SVIDs to every workload on the node via a Unix socket. The workload never has to know where the certificates come from.
Integrating with vLLM and LiteLLM
vLLM and LiteLLM proxy both support SPIFFE as a workload identity source via the standard Workload API — no code changes to the AI runtime.
For LiteLLM, configure the proxy to use the SPIRE agent's workload API for outbound mTLS connections. The proxy presents its SVID to upstream model providers that support mTLS. You get mutual TLS without managing a certificate infrastructure.
Certificate rotation without downtime
The SVID rotation policy is 24 hours. The SPIRE agent polls every 15 minutes. When a new SVID is available, it writes to disk and signals the workload via SIGUSR2.
LiteLLM handles SIGUSR2 by reloading the certificate from disk. The next outbound request uses the new certificate. No connection drop. No restart. No downtime.
What you get
Every SVID issuance is logged: workload SPIFFE ID, node ID, attestation data, timestamp. This is your audit trail for authentication events — which satisfies the equivalent SOC 2 control, and gives you forensic capability if something behaves unexpectedly.
Combined with Langfuse trace data (anonymized before ingestion), you have end-to-end traceability from workload identity to model inference.