Korifi installer for kind clusters

danail-branekov · georgethebeatle · danail-branekov · commit 5a2928196185 · 2023-10-30T18:07:33.000+02:00
fixes cloudfoundry#2895 Co-authored-by: Georgi Sabev <georgethebeatle@gmail.com> Co-authored-by: Danail Branekov <danailster@gmail.com>
diff --git a/INSTALL.kind.md b/INSTALL.kind.md
@@ -3,29 +3,25 @@
 
 # Install Korifi on kind
 
-This document integrates our [install instructions](./INSTALL.md) with specific tips to install Korifi locally using [kind](https://kind.sigs.k8s.io/).
+In order to install korifi on kind effortlessly we have prepared an installation job definition that you simply apply to your kind cluster. It will install korifi with reasonable defautls using a local docker registry (also running on your kind cluster).
 
-## Initial setup
-
-Export the following environment variables:
-
-```sh
-ROOT_NAMESPACE="cf"
-KORIFI_NAMESPACE="korifi-system"
-ADMIN_USERNAME="kubernetes-admin"
-BASE_DOMAIN="apps-127-0-0-1.nip.io"
-```
-
-`apps-127-0-0-1.nip.io` will conveniently resolve to `127.0.0.1` using [nip.io](https://nip.io/), which is exactly what we need.
-
-### Cluster creation
+## Cluster creation
 
 In order to access the Korifi API, we'll need to [expose the cluster ingress locally](https://kind.sigs.k8s.io/docs/user/ingress/). To do it, create your kind cluster using a command like this:
 
 ```sh
 cat <<EOF | kind create cluster --name korifi --config=-
 kind: Cluster
 apiVersion: kind.x-k8s.io/v1alpha4
+containerdConfigPatches:
+- |-
+  [plugins."io.containerd.grpc.v1.cri".registry]
+    [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
+      [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localregistry-docker-registry.default.svc.cluster.local:30050"]
+        endpoint = ["http://127.0.0.1:30050"]
+    [plugins."io.containerd.grpc.v1.cri".registry.configs]
+      [plugins."io.containerd.grpc.v1.cri".registry.configs."127.0.0.1:30050".tls]
+        insecure_skip_verify = true
 nodes:
 - role: control-plane
   extraPortMappings:
@@ -35,46 +31,71 @@ nodes:
   - containerPort: 443
     hostPort: 443
     protocol: TCP
+  - containerPort: 30050
+    hostPort: 30050
+    protocol: TCP
 EOF
 ```
 
-### Container registry
+## Install Korifi
 
-We recommend you use [DockerHub](https://hub.docker.com/) as your container registry.
+- Run the installer job:
 
-## Dependencies
+```sh
+kubectl apply -f https://github.com/cloudfoundry/korifi/releases/latest/download/install-korifi-kind.yaml
+```
 
-Follow the [common instructions](./INSTALL.md#dependencies), with the exception of Metrics Server.
+- If you want track the job progress, run:
 
-### Metrics Server
+```sh
+kubectl -n korifi-installer logs --follow job/install-korifi
+```
 
-Make sure you pass the following flags to the Metrics Server container (see [_Configuration_](https://github.com/kubernetes-sigs/metrics-server#configuration)):
+- **Optional** After the job is complete you can delete the `korifi-installer` namespace
 
--   `--kubelet-insecure-tls`
--   `--kubelet-preferred-address-types=InternalIP`
+```sh
+kubectl delete namespace korifi-installer
+```
 
-## Pre-install configuration
+## Test Korifi
 
-No changes here, follow the [common instructions](./INSTALL.md#pre-install-configuration).
-For the container registry credentials `Secret`, we recommend you [create an access token](https://hub.docker.com/settings/security?generateToken=true) on DockerHub.
+- Target the api:
 
-## Install Korifi
+```sh
+cf api https://localhost --skip-ssl-validation
+```
+
+- Authenticate as the cf admin user:
+
+```sh
+cf auth kind-korifi
+```
 
-No changes here, follow the [common instructions](./INSTALL.md#install-korifi).
-If using DockerHub as recommended above, set the following values:
+- Create and target an org and a space
+
+```sh
+cf create-org org && cf create-space -o org space && cf target -o org
+```
 
--   `kpackImageBuilder.builderRepository`: `index.docker.io/<username>/kpack-builder`;
--   `containerRepositoryPrefix`: `index.docker.io/<username>/`;
+- Push a buildpack app and access it:
 
-Remember to set `generateIngressCertificates` to `true` if you want to use self-signed TLS certificates.
+```sh
+make build-dorifi
+cf push dorifi -p tests/assets/dorifi
+curl -k https://dorifi.apps-127-0-0-1.nip.io
+```
 
-If `$KORIFI_NAMESPACE` doesn't exist yet, you can add the `--create-namespace` flag to the `helm` invocation.
+- Push a docker app and access it:
 
-## Post-install Configuration
+```sh
+cf push nginx --docker-image nginxinc/nginx-unprivileged:1.23.2
+curl -k https://nginx.apps-127-0-0-1.nip.io
+```
 
-Yon can skip this section.
+## Cleanup
 
-## Test Korifi
+When you no longer need korifi you can delete the whole kind cluster via:
 
-No changes here, follow the [common instructions](./INSTALL.md#test-korifi).
-When running `cf login`, make sure you select the entry associated to your kind cluster (`kind-korifi` in our case).
+```sh
+kind delete cluster --name korifi
+```
diff --git a/INSTALL.md b/INSTALL.md
@@ -19,7 +19,7 @@ This document was tested on:
 
 -   [EKS](https://aws.amazon.com/eks/), using AWS' [Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/) (see [_Install Korifi on EKS_](./INSTALL.EKS.md));
 -   [GKE](https://cloud.google.com/kubernetes-engine), using GCP's [Artifact Registry](https://cloud.google.com/artifact-registry);
--   [kind](https://kind.sigs.k8s.io/), using [DockerHub](https://hub.docker.com/) (see [_Install Korifi on kind_](./INSTALL.kind.md)).
+-   [kind](https://kind.sigs.k8s.io/): see [_Install Korifi on kind_](./INSTALL.kind.md).
 
 ## Initial setup
 
@@ -222,4 +222,4 @@ cf create-space -o org1 space1
 cf target -o org1
 cd <directory of a test cf app>
 cf push test-app
-```
+```
diff --git a/scripts/installer/Dockerfile b/scripts/installer/Dockerfile
@@ -0,0 +1,34 @@
+FROM ubuntu
+
+ARG HELM_CHART_SOURCE
+
+COPY scripts/install-dependencies.sh scripts/install-dependencies.sh
+COPY scripts/create-new-user.sh scripts/create-new-user.sh
+COPY tests/dependencies tests/dependencies
+COPY tests/vendor tests/vendor
+COPY ${HELM_CHART_SOURCE} helm/korifi
+
+RUN apt-get update \
+  && apt-get install --yes \
+       --no-install-recommends \
+       apt-transport-https \
+       ca-certificates \
+       conntrack \
+       gnupg2 \
+       curl \
+       git \
+       sudo \
+  && rm -rf /var/lib/apt/lists/*
+
+# helm
+RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" >/etc/apt/sources.list.d/helm-stable-debian.list \
+  && curl -fsSL https://baltocdn.com/helm/signing.asc | gpg --dearmor >/usr/share/keyrings/helm.gpg
+
+RUN apt-get update \
+  && apt-get install --yes \
+       helm \
+  && rm -rf /var/lib/apt/lists/*
+
+# kubectl
+RUN curl -fsSLo /usr/bin/kubectl "https://dl.k8s.io/release/$(curl -fsSL https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" \
+  && chmod +x /usr/bin/kubectl
diff --git a/scripts/installer/install-korifi-kind.yaml b/scripts/installer/install-korifi-kind.yaml
@@ -0,0 +1,113 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: korifi-installer
+
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: cf
+  labels:
+    pod-security.kubernetes.io/audit: restricted
+    pod-security.kubernetes.io/enforce: restricted
+
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: korifi
+  labels:
+    pod-security.kubernetes.io/audit: restricted
+    pod-security.kubernetes.io/enforce: restricted
+
+---
+apiVersion: v1
+data:
+  .dockerconfigjson: eyJhdXRocyI6eyJsb2NhbHJlZ2lzdHJ5LWRvY2tlci1yZWdpc3RyeS5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsOjMwMDUwIjp7InVzZXJuYW1lIjoidXNlciIsInBhc3N3b3JkIjoicGFzc3dvcmQiLCJhdXRoIjoiZFhObGNqcHdZWE56ZDI5eVpBPT0ifX19
+kind: Secret
+metadata:
+  name: image-registry-credentials
+  namespace: cf
+type: kubernetes.io/dockerconfigjson
+
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: korifi-installer
+  namespace: korifi-installer
+
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: korifi-installer
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: cluster-admin
+subjects:
+- kind: ServiceAccount
+  name: korifi-installer
+  namespace: korifi-installer
+
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: install-korifi
+  namespace: korifi-installer
+spec:
+  template:
+    metadata:
+      name: install-korifi
+    spec:
+      serviceAccountName: korifi-installer
+      restartPolicy: Never
+      containers:
+      - name: install-korifi
+        image: cloudfoundry/korifi-installer
+        command:
+        - bash
+        - -c
+        - |
+          scripts/install-dependencies.sh --insecure-tls-metrics-server
+
+          helm repo add twuni https://helm.twun.io
+          # the htpasswd value below is username: user, password: password encoded using `htpasswd` binary
+          # e.g. `docker run --entrypoint htpasswd httpd:2 -Bbn user password`
+          #
+          helm upgrade --install localregistry twuni/docker-registry \
+            --namespace default \
+            --set service.type=NodePort,service.nodePort=30050,service.port=30050 \
+            --set persistence.enabled=true \
+            --set persistence.deleteEnabled=true \
+            --set secrets.htpasswd='user:$2y$05$Ue5dboOfmqk6Say31Sin9uVbHWTl8J1Sgq9QyAEmFQRnq1TPfP1n2'
+
+          registry_status_code=""
+          while [[ "$registry_status_code" != "200" ]]; do
+            echo Waiting for the local docker registry to start...
+            registry_status_code=$(curl -o /dev/null -w "%{http_code}" --user user:password http://localregistry-docker-registry.default.svc.cluster.local:30050/v2/_catalog 2>/dev/null)
+            sleep 1
+          done
+
+          helm upgrade --install korifi helm/korifi \
+            --namespace korifi \
+            --set=adminUserName="kubernetes-admin" \
+            --set=defaultAppDomainName="apps-127-0-0-1.nip.io" \
+            --set=generateIngressCertificates="true" \
+            --set=logLevel="debug" \
+            --set=debug="false" \
+            --set=stagingRequirements.buildCacheMB="1024" \
+            --set=api.apiServer.url="localhost" \
+            --set=controllers.taskTTL="5s" \
+            --set=jobTaskRunner.jobTTL="5s" \
+            --set=containerRepositoryPrefix="localregistry-docker-registry.default.svc.cluster.local:30050/" \
+            --set=kpackImageBuilder.clusterStackBuildImage="paketobuildpacks/build-jammy-base" \
+            --set=kpackImageBuilder.clusterStackRunImage="paketobuildpacks/run-jammy-base" \
+            --set=kpackImageBuilder.builderRepository="localregistry-docker-registry.default.svc.cluster.local:30050/kpack-builder" \
+            --wait
+
+          kubectl wait --for=condition=ready clusterbuilder --all=true --timeout=15m
