AKS: you update-credentials and can’t pull from your ACR?

Ran into this one the other day. Suspect it may be an issue for some people as their Service Principle secrets are going to expire soon (default is 1 year).

I used the following method below to build an AKS cluster:

## Create a new SP
az ad sp create-for-rbac --name AKS01 --skip-assignment

## Give the new SP rights to my ACR
az role assignment create --assignee AppID --role acrpull --scope /subscriptions/{SubScriptionID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.ContainerRegistry/registries/{ACRName}

## Create:
az aks create --resource-group {ResourceGroupName} --name AKS01 --ssh-key-value /path/tomy/.ssh/id_rsa.pub --service-principal SPAppID --client-secret PASSWORD

## Add an app from my ACR:

kubectl run mysimplenodeapp --image=myacr.azurecr.io/mysimplenodeapp :latest --port=5000
kubectl expose deployment mysimplenodeapp --type=LoadBalancer --name=mysimplenodeapp --port 5000

## Update the cluster:

az aks upgrade --resource-group {ResourceGroupName} --name AKS01 --kubernetes-version 1.12.6

Standard Stuff.

The trick is, when you need to update you SP credentials, how are you going to do it? Seems that there are 2 ways you can update the credentials, in the portal and via command line.

Long story short: Use the command line method! Ran into a problem when the secret was created in the portal. The portal generated a very complex password and after updating the AKS cluster:

az aks update-credentials \
--name "AKS01" \
--resource-group {ResourceGroupName} \
--client-secret 'sJk_^/(_{##)-!&@(:|&&;:}*/G[{i$m+(}JC@?;]).X+3(Vb:>>%z?_J:' \
--service-principal {SPAppID } \
--reset-service-principal

Images couldn’t be pulled from the ACR:

Failed to pull image "myacr.azurecr.io/mysimplenodeapp:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://myacr.azurecr.io/v2/mysimplenodeapp/manifests/latest: unauthorized: authentication required

When I updated the SP credentails using the CLI way:

SP_ID=$(az aks show -g {ResourceGroupName} -n AKS01 --query servicePrincipalProfile.clientId -o tsv)
SP_SECRET=$(az ad sp credential reset --name $SP_ID --query password -o tsv)

az aks update-credentials \
    --resource-group {ResourceGroupName} \
    --name AKS01 \
    --reset-service-principal \
    --service-principal $SP_ID \
    --client-secret $SP_SECRET

My image pulled from the ACR right away!

Seems that when you reset the credential via the CLI, it generates a “GIUD” as the secret, which doesn’t have any of the non alphanumeric characters that the portal produces. And all seems fine.

So if it is time for you to update-credentials, use the CLI method:

SP_ID=$(az aks show -g {ResourceGroupName} -n AKS01 --query servicePrincipalProfile.clientId -o tsv)
SP_SECRET=$(az ad sp credential reset --name $SP_ID --query password -o tsv)

az aks update-credentials \
    --resource-group {ResourceGroupName} \
    --name AKS01 \
    --reset-service-principal \
    --service-principal $SP_ID \
    --client-secret $SP_SECRET

Hope that helps someone!