[400] admission webhook "soft-validate-cr.kubedirector.hpe.com" does not support dry run

[jear]

Hi,

Installed k8s cluster.
Deployed Kubeflow - I can create / delete KF notebooks ( Jupyter/Rstudio/Vscode) without any problem
Then Deployed Kubedirector -> but when I try to create KF notebooks, I get :
[400] admission webhook "soft-validate-cr.kubedirector.hpe.com" does not support dry run https://kubeflow.gpu02.lysdemolab.fr/jupyter/api/namespaces/kubeflow-user-example-com/notebooks

Then removed Kubedirector ( make teardown )
But still the same error in KF :-(

So seems that the KD mutating webhook is still there after removal.

[jear]

k get mutatingwebhookconfigurations.admissionregistration.k8s.io 
I0322 14:55:11.635908   20498 request.go:668] Waited for 1.178537203s due to client-side throttling, not priority and fairness, request: GET:https://10.69.0.65/k8s/clusters/c-m-qws9mqhq/apis/sources.knative.dev/v1beta1?timeout=32s
NAME                                               WEBHOOKS   AGE
admission-webhook-mutating-webhook-configuration   1          14d
cache-webhook-kubeflow                             1          14d
cert-manager-webhook                               1          14d
inferenceservice.serving.kserve.io                 2          14d
inferenceservice.serving.kubeflow.org              3          14d
istio-sidecar-injector                             4          14d
katib.kubeflow.org                                 2          14d
kube-prometheus-stack-1644-admission               1          40d
kubedirector-webhook                               2          3d21h
sinkbindings.webhook.sources.knative.dev           1          14d
webhook.eventing.knative.dev                       1          14d
webhook.istio.networking.internal.knative.dev      1          14d
webhook.serving.knative.dev                        1          14d

[jear]

deleting it solved the issue.

k delete mutatingwebhookconfigurations.admissionregistration.k8s.io kubedirector-webhook
I0322 14:58:39.975317   20534 request.go:668] Waited for 1.175550214s due to client-side throttling, not priority and fairness, request: GET:https://10.69.0.65/k8s/clusters/c-m-qws9mqhq/apis/argoproj.io/v1alpha1?timeout=32s
mutatingwebhookconfiguration.admissionregistration.k8s.io "kubedirector-webhook" deleted

[jear]

LGTM, but worth updating the make teardown script.

[joel-bluedata]

PR #563 happens to include an update to make teardown, which will fix things for those K8s versions where that owner-ref-to-cluster-scope has stopped working. However we probably won't put that PR into the very next KD release so maybe I'll pull in just that particular makefile modification for the next release.

[jear]

Or support dry run...

[joel-bluedata]

That should be addressed by PR #563 as well, in that we explicitly set sideEffects to None for the webhook as part of moving it from the v1beta1 MutatingWebhook API to v1.

I'm curious about the situation where you're seeing this though. What version of K8s and Kubeflow for example?

[joel-bluedata]

Going to reopen this to consider setting sideEffects to None for the 0.9.0 release (before PR #563 is merged).