minikube - Kubernetes cannot pull image from private docker image repository

Question

I have problem with kubernetes (minikube) and pull images from local image repository on docker. Docker repository was created:

docker run --entrypoint htpasswd registry:2 -Bbn zordon examplePassword > /mnt/LINUX/auth/htpasswd

docker run -d 
  -p 5000:5000 
  --restart=always 
  --name registry 
  -v /mnt/LINUX/dockerreg:/var/lib/registry 
  -v /mnt/LINUX/auth:/auth 
  -e "REGISTRY_AUTH=htpasswd" 
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" 
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd 
  registry:2

Then I want to create simple pod with image which was succesfully uploaded to local repository:

curl localhost:5000/v2/_catalog
{"repositories":["car/configuration"]}

I have also create secret on minikube cluster with:

kubectl create secret docker-registry docregkey --docker-server=localhost:5000 --docker-username=zordon --docker-password=examplePassword [email protected]

and define simple Pod:

    apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: car/configuration:latest
    imagePullPolicy: Always
  restartPolicy: Always
  imagePullSecrets:
  - name: docregkey

unfortunatelly I getting still:

Failed to pull image "car/configuration:latest": rpc error: code = Unknown desc = Error response from daemon: pull access denied for car/configuration, repository does not exist or may require 'docker login'

How i can fix this problem ?

Answer 1

For minikube to pull from your own local docker registry, the tag affects the pull policy. Per Images docs, pull policy is IfNotPresent by default EXCEPT if

1. you use :latest as the tag for the image to use
2. OR you omit the tag for the image to use.

In those cases the pull policy will effectively default to Always, which will attempt to pull from docker hub. This will cause minikube to be unable to fetch local images that have no tag or "latest" tag.

Moral of the story is, don't rely on the default because it is too confusing :)

So always explicitly state the pull policy:

1. when deploying into minikube the pull policy should be IfNotPresent or Never for the local images
2. when deploying into a cloud host (like AWS), pull policy should be as for public images (see below)
3. the pull policy should be Always for those public images that use a tag like "latest" or "stable" (because the image the tag points will change over time), and IfNotPresent for tags that always point to the same image (to avoid fetching more than necessary)

This means that if you avoid using tags like latest and stable etc, there is only one rule to follow:

1. explicitly set the imagePullPolicy in your spec (or on the command line in the case of run) to IfNotPresent, as this is will always look for it locally first, and go to public registry if it is not found locally, and this will work whether or not you are deploying into minikube or cloud.