Skip to content

Key Vault 403 (Forbidden) — missing secrets recover permission

Error message (sample)

The following is a redacted example of the error returned by Key Vault (customer-specific identifiers removed):

The user, group or application '' does not have the secrets recover permission on Key Vault ''. For help resolving this issue, see https://go.microsoft.com/fwlink/?linkid=2125287. Status: 403 (Forbidden)

Symptoms

  • Services fail when attempting to access or recover secrets from Key Vault and return HTTP 403 Forbidden.
  • Related errors referencing does not have secrets recover permission or ErrorCode: Forbidden.
  • The Master Tables, such as NPOSettings, does not have any seed data populated during deployment time Alt text

  • config app Startup function shows errors. However, the details of the errors are only available via App Insight tracking.

    Alt text

Root cause

The Key Vault is using RBAC authorization and the requesting managed identities/service principals do not have the required Key Vault RBAC roles. Specifically:

  • The following applications need Key Vault Secrets User role (or equivalent with secret read/list):
  • config app
  • master maintenance app
  • execution engine app
  • polling engine app
  • The config app must additionally have Key Vault Secrets Officer role where it needs to recover or undelete secrets.
  • Key Vault must be configured to use Azure RBAC (not access policies) for these role assignments to take effect.

Resolution

  1. Enable Key Vault RBAC (if not already enabled)

  2. Portal: Key Vault -> Access control (IAM) -> Identity & access management. Confirm Key Vault is using RBAC authorization model.

    Alt text

  3. Assign roles to each application identity

  4. Portal: Key Vault -> Access control (IAM) -> + Add -> Add role assignment

    • Role: Key Vault Secrets User for the four apps listed above
    • Role: Key Vault Secrets Officer for the config app (only if it needs to recover secrets)

  5. Azure CLI (replace placeholders):

# variables
SUBSCRIPTION_ID="<subscription-id>"
RESOURCE_GROUP="<resource-group>"
KEYVAULT_NAME="cog-pf-dev-vault-02"
APP_PRINCIPAL_ID="<object-id-or-service-principal-id>" # or managed identity principal id

# assign Key Vault Secrets User
az role assignment create --assignee $APP_PRINCIPAL_ID --role "Key Vault Secrets User" \
  --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.KeyVault/vaults/$KEYVAULT_NAME

# assign Key Vault Secrets Officer (if required for config app)
az role assignment create --assignee $APP_PRINCIPAL_ID --role "Key Vault Secrets Officer" \
  --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.KeyVault/vaults/$KEYVAULT_NAME
  • PowerShell (replace placeholders):
$scope = "/subscriptions/$SubscriptionId/resourceGroups/$ResourceGroup/providers/Microsoft.KeyVault/vaults/$KeyVaultName"
New-AzRoleAssignment -ObjectId <object-id> -RoleDefinitionName "Key Vault Secrets User" -Scope $scope

  1. Confirm assignments and test

  2. Portal: Key Vault -> Access control (IAM) -> Role assignments — verify the identities appear with the correct role.

  3. Test from the application or use the managed identity/service principal to request a secret; the 403 should no longer occur.

Verification checklist

  • [ ] Key Vault is set to use Azure RBAC
  • [ ] config, master maintenance, execution engine, polling engine identities have Key Vault Secrets User
  • [ ] config identity also has Key Vault Secrets Officer if secret recover operations are required
  • [ ] Application requests succeed (no 403)

Notes and references

  • Microsoft troubleshooting link from error message: https://go.microsoft.com/fwlink/?linkid=2125287
  • Key Vault RBAC guide: https://learn.microsoft.com/azure/key-vault/general/rbac-guide