How to get kubeadm init to not fail?

[bis-sb]

I am going through the setup procedure but when I run kubeadm init I am met by errors:

[init] Using Kubernetes version: v1.9.5
[init] Using Authorization modes: [Node RBAC]
[preflight] Running pre-flight checks.
[preflight] Some fatal errors occurred:
        [ERROR IsPrivilegedUser]: user is not running as root
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

I check what groups my user belongs to and it lists root as one of the groups:

sbrown@Dev-kube-01:~$ groups
sbrown root adm cdrom sudo dip plugdev lxd lpadmin sambashare docker

I am a novice with Ubuntu so I'm having trouble making sense of this error: why does kubernetes not think my user is privileged? Does the error mean that I'll have to run sudo kubeadm init... or does it intend for me to run everything as root? Is it possible for me to modify my current user so that I can run kubeadm init without IsPrivilegedUser throwing an error?

[bis-sb]

I just went ahead and used the root account; now I encounter other errors:

[ERROR Swap]: running with swap on is not supported. Please disable swap
[ERROR CRI]: unable to check if the container runtime at "/var/run/dockershim.sock" is running: exit status 1

I'm fine with disabling swap, but I was confused about the CRI issue and based on some quick searching it seemed as though the crictl tool required some configuring but the installation instructions has no remarks about crictl, i.e. how Kubernetes is using crictl, what problem crictl solves, what the point of dockershim.sock is, who is expected to create dockershim.sock for crictl, etc.

OK so disabling the swap and cri pre-flight checks makes things get a little further, except now I notice that despite pre-pulling the docker images (as mentioned here), Kubernetes still tries to pull the images anyway and fails. The kubeadm init reference page seems to imply that pre-pulling the images is sufficient, but it isn't, although docker pull <image> also seems to try to look on the internet instead of using the local image so it may just be a docker issue. If there were a setting that corresponded to "pull the images from my private registry within my network" then this would be a non-issue (I already have all the images on the private registry and pulled them from there onto the Kubernetes master machine).

Not trying to complain, just trying to understand how to set this tool up and have a place to document issues so that at a future time when I need to set it up I can refer here. Help in understanding these issues is appreciated.

[sri-oc]

Container runtime is required for the kubelet to launch the pods on a node. Runtime exposes interfaces for the kubelet to schedule containers and manage the state of the pods. One of the container runtime is docker. Kubernetes installation instruction has the information to setup the docker (under the heading Installing Docker).

More information about CRI

[bis-sb]

I already went through the instructions for installing kubeadm and that didn't give me any issues, although I was wondering if Docker CE 17.03 is an absolute, or if I could get away with using 17.12.1 CE (which is the one that was already installed on the server).

That blog post on the CRI really helped me understand its role, thank you. The only thing that's still unclear is whether or not I'm expected to create /var/run/dockershim.sock (and how I would do that)

I still am having trouble getting kubeadm init to use the images that are already on the machine instead of trying to get them from the internet (which is inaccessible).

[bis-sb]

I found some unofficial documentation which lists some environment variables that looked promising but the environment variables have no effect: I look at the manifest files that kubeadm generates and the images that are referenced are all still gcr.io/google_container instead of <my private registry>/gcr.io/google_container

EDIT: Going back to the kubeadm documentation, I decided to try passing in a config file to kubeadm init and it has solved my issues, e.g.:

apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
etcd:
   image: "<my repo>:5000/gcr.io/google_containers/etcd-amd64:3.1.11"
imageRepository: "<my repo>:5000/gcr.io/google_containers"
kubernetesVersion: "1.9.5"

EDIT 2: or at least I thought it solved my issue. Kubeadm looks like it's going to my registry to pull the images but it still is reporting:

Unfortunately, an error has occurred:
        timed out waiting for the condition

This error is likely caused by:
        - The kubelet is not running
        - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
        - There is no internet connection, so the kubelet cannot pull the following control plane images:
                - <my repo>:5000/gcr.io/google_containers/kube-apiserver-amd64:v1.9.5
                - <my repo>:5000/gcr.io/google_containers/kube-controller-manager-amd64:v1.9.5
                - <my repo>:5000/gcr.io/google_containers/kube-scheduler-amd64:v1.9.5

Could this be because it doesn't trust the certificate that my private registry is using? Is there a way to upload the CA cert into kubernetes so that it trusts the cert my registry presents? Those images that it refers to in the error message are valid, i.e. I run docker pull <image> and it is able to get the image from my registry.

[bis-sb]

OK I wrongly assumed from that error message that the error was in pulling the images: it actually looks like it's because kubelet wasn't running:

I0320 15:13:43.853388    8656 server.go:182] Version: v1.9.5
I0320 15:13:43.853429    8656 feature_gate.go:226] feature gates: &{{} map[]}
I0320 15:13:43.853513    8656 plugins.go:101] No cloud provider specified.
W0320 15:13:43.853551    8656 server.go:328] standalone mode, no API client
W0320 15:13:43.878162    8656 server.go:236] No api server defined - no events will be sent to API server.
I0320 15:13:43.878191    8656 server.go:428] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
error: failed to run Kubelet: Running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. 

So in a new terminal I ran kubelet --fail-swap-on=false and now kubelet is running. Now I see dockershim.sock under /var/run/

Oddly enough kubeadm still fails the cri preflight check: [ERROR CRI]: unable to check if the container runtime at "/var/run/dockershim.sock" is running: exit status 1 despite dockershim.sock existing. Ignoring these preflight checks still results in failure, this time due to port 10250 not being available: [ERROR Port-10250]: Port 10250 is in use

So does kubeadm init expect to create the kubelet itself? In that case why would it look for dockershim.sock before even running kubelet?

[sri-oc]

Kubeadm init will start pods for etcd, dns and cni on the master node. For this it requires dockershim.sock.

I suggest you to install docker ce on the master before initializing kubeadm

[bis-sb]

Docker is running fine, what's confusing me is there appears to be a circular dependency when running kubeadm init:

• if kubelet is not already running, then the error is [ERROR CRI]: unable to check if the container runtime at "/var/run/dockershim.sock" is running: exit status 1
• if kubelet is running, then the error is [ERROR Port-10250]: Port 10250 is in use

EDIT: I did some more debugging by running kubelet and then using crictl stats resulted in the following error: getting status of runtime failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService
That error lead me to this issue which indicates that I have the wrong crictl version installed!!

EDIT 2: Actually according to this matrix I do have the correct version (1.0.0-alpha.0) so I guess I still am confused how to resolve this apparent circular dependency.

[bart0sh]

That circular dependency is also mentioned here: kubernetes-sigs/cri-tools#153 (comment)

It looks like a bug in kubeadm. It shouldn't run cri pre-flight check before running kubelet as dockershim.sock seems to be created by kubelet.

[bis-sb]

ok so that is a bug, thanks for clarifying that

[jacobtomlinson]

I am also having the same issue where the preflight check is looking for /var/run/dockershim.sock before it should exist.

The following runs successfully:

kubeadm init --ignore-preflight-errors=cri

[bart0sh]

Preflight check issue should be fixed by PR 58802

Temporary workaround for this issue is to switch off CRI check:

kubeadm init --ignore-preflight-errors=cri

[bis-sb]

Thanks for the pointers, I'll try them out if I revisit Kubernetes (or maybe by then it'll be easier to set up)

[faizurrahmann]

Hi friends,
I am getting below mentioned kubeadm init error due to the number of available CPUs 1 is less than the required 2. Anybody know how to ignore this error by using "--ignore-preflight-errors=" ?

Error:
Use "kubeadm init [command] --help" for more information about a command.
root@ip-172-31-15-167:~ kubeadm init --ignore-preflight-errors=cpu --apiserver-advertise-address 172.31.15.167 --pod-network-cidr=172.16.0.0/16
[init] Using Kubernetes version: v1.15.3
[preflight] Running pre-flight checks
[WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR NumCPU]: the number of available CPUs 1 is less than the required 2
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
root@ip-172-31-15-167:~# docker --version
Docker version 19.03.1, build 74b1e89

[neolit123]

@faizurrahmann

[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...

^ it tells you here.

try kubeadm init ... --ignore-preflight-errors=NumCPU

[faizurrahmann]

Yes --ignore-preflight-errors=NumCPU worked for me. I installed kubernetes initialization. Thank You.
I got it. The possible string value is what associated with error
Note below:
[ERROR NumCPU]: the number of available CPUs 1 is less than the required 2

[Mounika666]

kubeadm init --ignore-preflight-errors=all

It worked for me

[Anushamobis]

sudo kubeadm init --ignore-preflight-errors=all
worked for me

[lutuantai95]

On offical docs of kubeadm, there is no mention of --ignore-preflight-errors=all. I just see them using kube init.
It's absurd that kubectl can run without root but kubeadm gives errors.
Is there any way to fix it without using --ignore-preflight-errors=all and sudo ?