The substitution is three lines of client code. The part that costs an afternoon is that the most powerful-looking Azure OpenAI role is explicitly unable to make an inference call.
An Azure OpenAI resource key is a bearer secret with no identity, no expiry and no scope narrower than the whole resource. Every deployment on the resource is reachable with it, every caller looks identical in the audit trail, and rotating it means coordinating every consumer at once.
A managed identity replaces it with a short-lived Microsoft Entra ID token issued to a specific workload identity. The credential is never stored, the token expires on its own, and the grant is a role assignment you can scope to a resource group, a resource, or nothing at all. Combined with a private endpoint, it removes the two things an attacker needs — the network path and the static secret.
Microsoft documents four roles for Azure OpenAI, and the summary table on its RBAC article makes one distinction that is worth reading twice:
That third entry is the trap. Granting an application the Contributor role because it sounds broader produces an application that can rotate the keys it is no longer using and cannot call the model at all. The role you want for a workload is Cognitive Services OpenAI User
, and nothing else.
Microsoft also notes that subscription-level Owner and Contributor are inherited and take priority over these roles assigned at resource group level — which is why this often appears to work on a developer’s own machine and fail in the deployed application.
Cognitive Services OpenAI User
on the Azure OpenAI resource — at resource scope, not subscription scope.
PRINCIPAL_ID=$(az webapp identity assign \
--name app-inference --resource-group rg-app \
--query principalId -o tsv)
AOAI_ID=$(az cognitiveservices account show \
--name mg-openai-weu --resource-group rg-inference --query id -o tsv)
az role assignment create \
--assignee-object-id $PRINCIPAL_ID \
--assignee-principal-type ServicePrincipal \
--role "Cognitive Services OpenAI User" \
--scope $AOAI_ID
Once this is in place, disable key access on the resource. A key that still works is a key that something is still using, and you will not find out which until you turn it off.
The Azure OpenAI clients take a token provider rather than a key. The scope is the Azure AI services audience, not a per-resource one:
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AzureOpenAI
token_provider = get_bearer_token_provider(
DefaultAzureCredential(),
"https://cognitiveservices.azure.com/.default",
)
client = AzureOpenAI(
azure_endpoint="https://mg-openai-weu.openai.azure.com/",
azure_ad_token_provider=token_provider,
api_version="2024-10-21",
)
response = client.chat.completions.create(
model="chat-default",
messages=[{"role": "user", "content": "ping"}],
max_tokens=16,
)
get_bearer_token_provider
returns a callable the SDK invokes per request, so token refresh is handled for you — do not cache the token yourself. DefaultAzureCredential
resolves to the managed identity when running on Azure compute and to the developer’s az login
session locally, which is what makes the same code work in both places. On a machine with several signed-in identities, be explicit rather than letting the chain pick.
The azure_endpoint
must be the custom subdomain hostname. Entra ID authentication requires a custom subdomain, so a resource created without one cannot do this at all, and the subdomain cannot be added by renaming — see the private endpoint page, which depends on the same property for a different reason.
The example above uses a system-assigned identity because it is one command. For anything that is deployed repeatedly by a pipeline, a user-assigned identity is usually the better shape, and the reason is an ordering problem rather than a preference.
A system-assigned identity is created with its host and destroyed with it. Its principal ID therefore does not exist until the app service, container app or virtual machine exists — so an infrastructure template has to create the compute, read the principal ID out of the result, and only then create the role assignment. Every redeploy that recreates the compute produces a new principal, orphaning the old role assignment and requiring a new one. In a template that is a dependency chain and a propagation wait in the middle of every deployment.
A user-assigned identity is an independent ARM resource. You create it once, assign Cognitive Services OpenAI User
to it once, and then attach it to whatever compute needs it — including compute that does not exist yet. Roles are granted before the workload is deployed rather than after, several services can share one grant, and rebuilding the application changes nothing about its permissions.
The cost is a piece of configuration. DefaultAzureCredential
cannot guess which identity you meant on a host carrying more than one, so a user-assigned identity requires telling it — by setting AZURE_CLIENT_ID
to the identity’s client ID in the environment, or by passing that ID to the credential explicitly. A host with a system-assigned identity and a user-assigned one, with neither specified, will authenticate as whichever the chain finds first, and the resulting 401 says nothing about which identity was used.
One more thing worth doing in production: DefaultAzureCredential
walks a chain of credential sources — environment variables, managed identity, developer tooling — and each failed step costs a probe. Where you know the answer, name it. Using the managed identity credential directly removes several seconds from a cold start and turns a misconfiguration into an immediate error rather than a silent fallback to a developer credential that happens to be present on a build agent.
Four causes, in the order they are worth checking:
Cognitive Services OpenAI User
or Cognitive Services OpenAI Contributor
and not Cognitive Services Contributor
.https://cognitiveservices.azure.com/.default
. The ARM scope https://management.azure.com/.default
is for control-plane calls such as reading quota, and a token for one will not work on the other.Managing which role each service holds across a fleet of Azure OpenAI resources is a real piece of work in itself; that is the subject of the RBAC page.