Kubernetes: Failed to pull image. Server gave HTTP response to HTTPS client - json

Question

I'm trying to use Kubernetes with Docker. My image runs with Docker. I have one master-node and two worker-nodes. I also created a local registry like this $ docker run -d -p 5000:5000 --restart=always --name registry registry:2 and pushed my image into it. Everything worked fine so far.
I added { "insecure-registries":["xxx.xxx.xxx.xxx:5000"] } to the daemon.json file at /etc/docker. And I also changed the content of the docker-file at /etc/default/to DOCKER_OPTS="--config-file=/etc/docker/daemon.json". I made the changes on all nodes and I restarted the docker daemon afterwards.
I am able to pull my image from every node with the following command:
sudo docker pull xxx.xxx.xxx.xxx:5000/helloworldimage
I try to create my container from the master node with the command bellow:
sudo kubectl run test --image xxx.xxx.xxx.xxx:5000/helloworldimage
Than I get the following error:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned default/test-775f99f57-m9r4b to rpi-2
Normal BackOff 18s (x2 over 44s) kubelet, rpi-2 Back-off pulling image "xxx.xxx.xxx.xxx:5000/helloworldimage"
Warning Failed 18s (x2 over 44s) kubelet, rpi-2 Error: ImagePullBackOff
Normal Pulling 3s (x3 over 45s) kubelet, rpi-2 Pulling image "xxx.xxx.xxx.xxx:5000/helloworldimage"
Warning Failed 3s (x3 over 45s) kubelet, rpi-2 Failed to pull image "xxx.xxx.xxx.xxx:5000/helloworldimage": rpc error: code = Unknown desc = failed to pull and unpack image "xxx.xxx.xxx.xxx:5000/helloworldimage:latest": failed to resolve reference "xxx.xxx.xxx.xxx:5000/helloworldimage:latest": failed to do request: Head https://xxx.xxx.xxx.xxx:5000/v2/helloworldimage/manifests/latest: http: server gave HTTP response to HTTPS client
Warning Failed 3s (x3 over 45s) kubelet, rpi-2 Error: ErrImagePull
This is the docker version I use:
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:37:22 2019
OS/Arch: linux/arm
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:31:17 2019
OS/Arch: linux/arm
Experimental: false
containerd:
Version: 1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
This is the Kubernetes version I use:
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0+k3s.1", GitCommit:"0f644650f5d8e9f091629f860b342f221c46f6d7", GitTreeState:"clean", BuildDate:"2020-01-06T23:20:30Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/arm"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0+k3s.1", GitCommit:"0f644650f5d8e9f091629f860b342f221c46f6d7", GitTreeState:"clean", BuildDate:"2020-01-06T23:20:30Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/arm"}```

Answer 1

Kubernetes: Failed to pull image. Server gave HTTP response to HTTPS client.
{ "insecure-registries":["xxx.xxx.xxx.xxx:5000"] }
to the daemon.json file at /etc/docker.
I solved this problem by configuring it on all kubernetes nodes.

Answer 2

It appears that in some situations solution described here solved the problem:
sudo systemctl edit docker
Add below lines:
[Service]
ExecStart=/usr/bin/dockerd -H fd:// --insecure-registry registry:5000
sudo systemctl daemon-reload
systemctl restart docker
systemctl status docker