Skip to content
This repository was archived by the owner on Feb 6, 2025. It is now read-only.

Add kucero addon to auto rotates control plane certificates #1152

Merged
merged 7 commits into from
Jun 22, 2020
Merged

Add kucero addon to auto rotates control plane certificates #1152

merged 7 commits into from
Jun 22, 2020

Conversation

jenting
Copy link

@jenting jenting commented Jun 10, 2020

Why is this PR needed?

Add kucero as an addon to auto rotates control plane certificates.

kucero includes two components:

  • A Daemonset to pulling and rotates the kubeadm-managed certificates.
  • A Controller to sign the kubelet server CSR.

refs: https://github.com/SUSE/avant-garde/issues/1632

What does this PR do?

Add kucero as an addon.

Anything else a reviewer needs to know?

We add kucero on both 1.17.4 and 1.18.2 because we want kucero deployed before node upgrade. Add kucero into the previous version make the user have to install kucero addon first.
Otherwise, the kubelet server CSR will in pending state since no signer signs the kubelet server certificate.

Info for QA

Previously, the kubelet server certificate is self-signed by skuba with kubelet CA cert/key pair.
With this PR, the kubelet sends server CSR in K8s cluster, and kucero signed the server certificate to kubelet with kubelet CA cert/key pair, then kubelet save the server certificate in /var/lib/kubelet/pki/kubelet-server-current.pem.

Note that, the metrics-server trusted kubelet server certificate by kubelet CA cert.
If the kubelet server certificate signed by other CA, the metrics-server will in CrashLoopBack.

Related info

Packages to be released:

Status BEFORE applying the patch

  1. The kubeadm-managed server/client certificates will expire after 1 year.
  2. The kubelet server certificate is self-signed by skuba with kubelet CA cert/key pair.

Status AFTER applying the patch

  1. The kucero will periodically checks the kubeadm-managed server/client certificates residual time, and rotates the certificates if necessary automatically, and restart kubelet after rotates certificates finished.
  2. The kubelet will set serverTLSBootstrap: true, and kubelet will send server CSR. Then kucero will sign the kubelet server certificate by kubelet CA cert/key pair and kubelet saves the server certificate in /var/lib/kubelet/pki/kubelet-server-current.pem. The time kubelet send server CSR is handled by kubelet daemon.

Docs

SUSE/doc-caasp#879

Merge restrictions

(Please do not edit this)

We are in v4-maintenance phase, so we will restrict what can be merged to prevent unexpected surprises:

What can be merged (merge criteria):
    2 approvals:
        1 developer: code is fine
        1 QA: QA is fine
    there is a PR for updating documentation (or a statement that this is not needed)

@jenting jenting added ci-label:go13 Requires go13 ci-worker:experimental-v5 initial sp2 worker tests for caasp v5 v5 labels Jun 10, 2020
@jenting jenting changed the title Kucero Add kucero addon to auto rotates control plane certificates Jun 10, 2020
@jenting jenting self-assigned this Jun 10, 2020
@jenting jenting requested a review from maximenoel8 June 10, 2020 13:23
JenTing Hsiao added 6 commits June 11, 2020 16:57
kucero is a KUbernetes CErtificate ROtation focus on
kubernetes control plane certificate rotation automatically

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Add kucero addon to 1.17.4 to force user install kucero addon
first before node upgrade.
Otherwise, if upgrade control plane node before kucero addon installed,
the node's kubelet CSR generated but no kucero signer signs it.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
@jenting jenting removed ci-label:go13 Requires go13 ci-worker:experimental-v5 initial sp2 worker tests for caasp v5 v5 labels Jun 11, 2020
@jenting jenting marked this pull request as ready for review June 11, 2020 08:59
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

return nil
}

func kubeletConfigure(t *Target, data interface{}) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing kubelet server cert creation and upload is because we enable server rotation so it's expected everything will be setup automatically?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I intended to add kucero at previous version 1.17.4 is because I want to install kucero first and enable kubelet serverTLSBootstrap: true when cluster upgrade.

Copy link
Contributor

@innobead innobead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@innobead
Copy link
Contributor

cc @c3y1huang @cclhsu please help review. thanks.

@innobead innobead merged commit 74f429a into SUSE:master Jun 22, 2020
@jenting jenting deleted the kucero branch June 22, 2020 03:52
mmnelemane pushed a commit to mmnelemane/skuba that referenced this pull request Jul 3, 2020
* Add a new addon kucero

kucero is a KUbernetes CErtificate ROtation focus on
kubernetes control plane certificate rotation automatically

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Remove self-signed kubelet server cert

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Enable kubelet server TLS bootstrap for cluster upgrade

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Enable kubelet server TLS bootstrap for cluster init

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add kucero to 1.17.4

Add kucero addon to 1.17.4 to force user install kucero addon
first before node upgrade.
Otherwise, if upgrade control plane node before kucero addon installed,
the node's kubelet CSR generated but no kucero signer signs it.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add copyright year 2020

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Update test case

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants