Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rework snapshotter selection, detect fuseblk and use overlayfs-fuse #2874

Merged
merged 3 commits into from
Aug 18, 2022
Merged

rework snapshotter selection, detect fuseblk and use overlayfs-fuse #2874

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
20da9c7
rework snapshotter selection, detect fuseblk and use overlayfs-fuse
BenTheElder Aug 10, 2022
cf1bcbd
bump base image
BenTheElder Aug 11, 2022
391395e
bump node image
BenTheElder Aug 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 22 additions & 8 deletions images/base/files/usr/local/bin/entrypoint
View file
Original file line number Diff line number Diff line change
@@ -101,22 +101,36 @@ overlayfs_preferrable() {

configure_containerd() {
local snapshotter=${KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER:-}
if [[ -n "$userns" ]]; then
# userns (rootless) configs

# Adjust oomScoreAdj
# handle userns (rootless)
if [[ -n "$userns" ]]; then
# enable restrict_oom_score_adj
sed -i 's/restrict_oom_score_adj = false/restrict_oom_score_adj = true/' /etc/containerd/config.toml

# Use fuse-overlayfs if overlayfs is not preferrable: https://github.com/kubernetes-sigs/kind/issues/2275
if [[ -z "$snapshotter" ]] && ! overlayfs_preferrable; then
snapshotter="fuse-overlayfs"
fi
else
# we need to switch to the 'native' snapshotter on zfs
if [[ -z "$snapshotter" ]] && [[ "$(stat -f -c %T /kind)" == 'zfs' ]]; then
snapshotter="native"
fi

# if we have not already overridden the snapshotter, attempt to auto select
if [[ -z "$snapshotter" ]]; then
# we need to switch to 'native' or 'fuse-overlayfs' on zfs
container_filesystem="$(stat -f -c %T /kind)"
if [[ "$container_filesystem" == 'zfs' ]]; then
# if fuse is present, use fuse-overlayfs, else fallback to native
# we do not use the ZFS snapshotter because of skew issues vs the host
if [[ -e /dev/fuse ]]; then
snapshotter="fuse-overlayfs"
else
snapshotter="native"
fi
# fuse likely implies fuse-overlayfs, we should switch to fuse-overlayfs (or native)
elif [[ "$container_filesystem" == 'fuseblk' ]]; then
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the functional difference, the rest is rearranging a little.

we will select fuse-overlayfs if we detect we're in fuseblk and we haven't otherwise already selected a snapshotter due to userns or KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's any supported snapshotter environment where we'd be in fuse within the container filesystem but not using fuse-overlayfs.

Technically the user could configure a mount over /kind but that's already an unsafe choice, we should perhaps but a README in there to let people know that this is an internal directory.

snapshotter="fuse-overlayfs"
fi
fi

# if we've overridden or auto-selected the snapshotter vs the default, update containerd
if [[ -n "$snapshotter" ]]; then
echo "INFO: changing snapshotter from \"overlayfs\" to \"$snapshotter\""
sed -i "s/snapshotter = \"overlayfs\"/snapshotter = \"$snapshotter\"/" /etc/containerd/config.toml
2 changes: 1 addition & 1 deletion pkg/apis/config/defaults/image.go
View file
Original file line number Diff line number Diff line change
@@ -18,4 +18,4 @@ limitations under the License.
package defaults

// Image is the default for the Config.Image field, aka the default node image.
const Image = "kindest/node:v1.24.3@sha256:e1a5c449fb2556b5fbba29fb43f28055cacb12ce53d1f281e8be442d0dd5cd1d"
const Image = "kindest/node:v1.24.3@sha256:09961d2443a557dfa59126ce8b5388e9c06610b0276dc0a986a74d3d0f01e53e"
2 changes: 1 addition & 1 deletion pkg/build/nodeimage/defaults.go
View file
Original file line number Diff line number Diff line change
@@ -20,4 +20,4 @@ package nodeimage
const DefaultImage = "kindest/node:latest"

// DefaultBaseImage is the default base image used
const DefaultBaseImage = "docker.io/kindest/base:v20220808-a2dcbb47"
const DefaultBaseImage = "docker.io/kindest/base:v20220811-20da9c7c"