Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm: add docs for UpgradeAddonsBeforeControlPlane feature gate #41606

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,30 @@ or `kubeadm upgrade apply`), kubeadm respects the value of `UnversionedKubeletCo
(during `kubeadm join`, `kubeadm reset`, `kubeadm upgrade ...`), kubeadm attempts to use unversioned ConfigMap name first;
if that does not succeed, kubeadm falls back to using the legacy (versioned) name for that ConfigMap.

List of deprecated feature gates:

{{< table caption="kubeadm deprecated feature gates" >}}
Feature | Default
:-------|:--------
`UpgradeAddonsBeforeControlPlane` | `false`
{{< /table >}}

Feature gate descriptions:

`UpgradeAddonsBeforeControlPlane`
: This is as a **disabled** feature gate that was introduced for Kubernetes v1.28, in order to allow reactivating a legacy
and deprecated behavior during cluster upgrade. For kubeadm versions prior to v1.28, kubeadm upgrades cluster addons (including
CoreDNS and kube-proxy) immediately during `kubeadm upgrade apply`, regardless of whether there are other control plane
instances that have not been upgraded. This may cause compatibility problems. Since v1.28, kubeadm defaults to a mode that
always checks whether all the control plane instances have been upgraded before starting to upgrade the addons. This behavior
is applied to both `kubeadm upgrade apply` and `kubeadm upgrade node`. kubeadm determines whether a control plane instance
has been upgraded by checking whether the image of the kube-apiserver Pod has been upgraded. You must perform control plane
instances upgrade sequentially or at least ensure that the last control plane instance upgrade is not started until all the
other control plane instances have been upgraded completely, and the addons upgrade will be performed after the last control plane
instance is upgraded. The deprecated `UpgradeAddonsBeforeControlPlane` feature gate gives you a chance to keep the old upgrade
behavior. You should not need this old behavior; if you do, you should consider changing your cluster or upgrade processes, as this
feature gate will be removed in a future release.

### Adding kube-proxy parameters {#kube-proxy}

For information about kube-proxy parameters in the kubeadm configuration see:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,20 @@ Pick a control plane node that you wish to upgrade first. It must have the `/etc
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
```

{{< note >}}
For versions earlier than v1.28, kubeadm defaulted to a mode that upgrades the addons (including CoreDNS and kube-proxy)
immediately during `kubeadm upgrade apply`, regardless of whether there are other control plane instances that have not
been upgraded. This may cause compatibility problems. Since v1.28, kubeadm defaults to a mode that checks whether all
the control plane instances have been upgraded before starting to upgrade the addons. You must perform control plane
instances upgrade sequentially or at least ensure that the last control plane instance upgrade is not started until all
the other control plane instances have been upgraded completely, and the addons upgrade will be performed after the last
control plane instance is upgraded. If you want to keep the old upgrade behavior, please enable the `UpgradeAddonsBeforeControlPlane`
feature gate by `kubeadm upgrade apply --feature-gates=UpgradeAddonsBeforeControlPlane=true`. The Kubernetes project does
not in general recommend enabling this feature gate, you should instead change your upgrade process or cluster addons so
that you do not need to enable the legacy behavior. The `UpgradeAddonsBeforeControlPlane` feature gate will be removed in
a future release.
{{</ note >}}

1. Manually upgrade your CNI provider plugin.

Your Container Network Interface (CNI) provider may have its own upgrade instructions to follow.
Expand Down