|
| 1 | +--- |
| 2 | +title: 'Stop Making Kubernetes Auth Hard' |
| 3 | +date: 2024-09-19 |
| 4 | +byline: Thomas Rampelberg |
| 5 | +--- |
| 6 | + |
| 7 | +I've spent most of my time working with Kubernetes being afraid of auth. I |
| 8 | +understand how RBAC works and I know that `.kube/config` is what's required to |
| 9 | +talk to an API server, but that's pretty much where my understanding stopped. |
| 10 | +Configuring the API server to use an auth plugin, getting tokens or certificates |
| 11 | +and setting up plugins on my (and all my ueser's) laptops made me think that it |
| 12 | +was all a monumental task. Just getting my environment setup correctly _was_ a |
| 13 | +monumental task. Well, as part of implementing kty's oauth support, I've been |
| 14 | +forced to figure out how it all actually works. And, it turns out, it doesn't |
| 15 | +need to be nearly as complex as I thought it was. |
| 16 | + |
| 17 | +## TLDR |
| 18 | + |
| 19 | +Use OpenID and grant groups or users the correct permissions in your cluster. It |
| 20 | +is likely that your organization already has an OpenID provider in place. |
| 21 | +Google, Github, Okta can all be used (and many more). That's it, that's all you |
| 22 | +need. Don't bother with IAM, service accounts or any of that other stuff. Those |
| 23 | +are all reasonable for machines - not for users. |
| 24 | + |
| 25 | +If you'd like to see how easy it is to get running, check out the |
| 26 | +[getting started guide](/getting-started). kty uses OpenID to verify your |
| 27 | +identity over SSH and then takes care of the rest for you. |
| 28 | + |
| 29 | +Make sure to get it going with `kubectl` as well. Check out |
| 30 | +[kubelogin](https://github.com/int128/kubelogin?tab=readme-ov-file), a `kubectl` |
| 31 | +plugin that will do the OIDC dance for you. Note that if you can't make the |
| 32 | +modifications required for the API server, you'll want to use an |
| 33 | +[oidc-proxy](https://github.com/TremoloSecurity/kube-oidc-proxy). Luckily, most |
| 34 | +Kubernetes solutions support OIDC out of the box like [EKS][eks-oidc] or |
| 35 | +[GKE][gke-oidc]. |
| 36 | + |
| 37 | +[eks-oidc]: |
| 38 | + https://docs.aws.amazon.com/eks/latest/userguide/authenticate-oidc-identity-provider.html |
| 39 | +[gke-oidc]: https://cloud.google.com/kubernetes-engine/docs/how-to/oidc |
| 40 | + |
| 41 | +Now, if you're interested in how it all works and would like to start being |
| 42 | +comfortable with how it all works, read on. |
| 43 | + |
| 44 | +## Authentication |
| 45 | + |
| 46 | +Let's start out by splitting "auth" into two parts: authentication and |
| 47 | +authorization. Authentication is how you prove who you are. The result of the |
| 48 | +authentication process is an identity that can be used to see what you are, or |
| 49 | +aren't authorized to do. If we didn't actually care about verifying your |
| 50 | +identity, authentication could be nothing more than sending the username in |
| 51 | +cleartext to the API server. Obviously, we'd like a solution that is a little |
| 52 | +bit more secure than that. |
| 53 | + |
| 54 | +Kubernetes has a [whole bunch][auth-plugins] of ways to authenticate. Because it |
| 55 | +is the easiest to understand, let's start wit the static token file. This is |
| 56 | +equivalent to having a username as password. You put the token aka "password" |
| 57 | +into the file and then associate it with a username. If this sounds like |
| 58 | +`/etc/passwd`, that's because it is! Each request sent to the API server |
| 59 | +contains your token as a header. The API server looks up the token in its file |
| 60 | +and maps that to a user or set of groups. Very similar to sending the username |
| 61 | +to the API server, but now we've got a piece of shared data, the token, that |
| 62 | +verifies the identity. |
| 63 | + |
| 64 | +Open ID Connect (OIDC) gets rid of the pre-shared secret and instead uses some |
| 65 | +cryptography magic to do the same thing. This allows for identity to be created |
| 66 | +in a central location and subsequently verified by anyone. When you authenticate |
| 67 | +with an OIDC provider, the end result of the process is an [ID |
| 68 | +token][oidc-id-token]. |
| 69 | + |
| 70 | +The ID token is a JSON web token (JWT) that contains a bunch of information |
| 71 | +about your identity. This is signed by the provider and can be verified by |
| 72 | +anyone with the public key. Most importantly, OIDC providers publish their |
| 73 | +configuration so that anyone can verify the token. If you're interested in |
| 74 | +what's in that configuration, check out [kty's][odic-config]. |
| 75 | + |
| 76 | +With an ID token and the way to verify it in hand, the API server can extract an |
| 77 | +identity from the token and use that as part of RBAC to understand what you're |
| 78 | +allowed to do. The association between the token and either groups or users |
| 79 | +happens as part of claims. If you've got a JWT, you can see the claims in your |
| 80 | +token by going to [jwt.io](https://jwt.io) and pasting it in. Here's a token |
| 81 | +that I've gotten for kty. |
| 82 | + |
| 83 | +```JSON |
| 84 | +{ |
| 85 | + "iss": "https://kty.us.auth0.com/", |
| 86 | + "aud": "P3g7SKU42Wi4Z86FnNDqfiRtQRYgWsqx", |
| 87 | + "iat": 1726784050, |
| 88 | + "exp": 1726820050, |
| 89 | + "sub": "github|123456", |
| 90 | + "email": "me@my-domain.com" |
| 91 | +} |
| 92 | +``` |
| 93 | + |
| 94 | +For this token, we could configure the API server to map the `email` claim to a |
| 95 | +user. This is just like the token file from above! Instead of using the |
| 96 | +pre-shared secret as the mapping, we've used the public key from the OIDC |
| 97 | +provider. |
| 98 | + |
| 99 | +[auth-plugins]: |
| 100 | + https://kubernetes.io/docs/reference/access-authn-authz/authentication/ |
| 101 | +[oidc-id-token]: https://auth0.com/docs/secure/tokens/id-tokens |
| 102 | +[oidc-config]: https://kty.us.auth0.com/.well-known/openid-configuration |
| 103 | + |
| 104 | +## Authorization |
| 105 | + |
| 106 | +Here's where it gets interesting. RBAC doesn't care about how you authenticated. |
| 107 | +If the API says you're a user - then you are that user. All it cares about is |
| 108 | +your identity and what roles that identity is bound to. Let's look at a simple |
| 109 | +role: |
| 110 | + |
| 111 | +```yaml |
| 112 | +apiVersion: rbac.authorization.k8s.io/v1 |
| 113 | +kind: ClusterRole |
| 114 | +metadata: |
| 115 | + name: view |
| 116 | +rules: |
| 117 | + - apiGroups: |
| 118 | + - '' |
| 119 | + resources: |
| 120 | + - pods |
| 121 | + verbs: |
| 122 | + - get |
| 123 | + - list |
| 124 | + - watch |
| 125 | +``` |
| 126 | +
|
| 127 | +Any identity that is bound to this role can get, list or watch pods in any |
| 128 | +namespace. How does an identity get associated with this role? That's where the |
| 129 | +`ClusterRoleBinding` comes into play. |
| 130 | + |
| 131 | +```yaml |
| 132 | +apiVersion: rbac.authorization.k8s.io/v1 |
| 133 | +kind: ClusterRoleBinding |
| 134 | +metadata: |
| 135 | + name: view |
| 136 | +roleRef: |
| 137 | + apiGroup: rbac.authorization.k8s.io |
| 138 | + kind: ClusterRole |
| 139 | + name: view |
| 140 | +subjects: |
| 141 | + - apiGroup: rbac.authorization.k8s.io |
| 142 | + kind: User |
| 143 | + name: me@my-domain.com |
| 144 | +``` |
| 145 | + |
| 146 | +Assuming that we're still talking about the token from above, this role binding |
| 147 | +associates all the permissions in the `view` role with the user |
| 148 | +`me@my-domain.com`. That's it! We've authenticated the identity and then |
| 149 | +verified that it can do some actions on the cluster. As RBAC is opt-in, you need |
| 150 | +to be granted permissions to do anything. There are some policies that come by |
| 151 | +default, in fact the `view` cluster role is one that comes out of the box (but |
| 152 | +simplified in this example). To see what can be granted, make sure to check out |
| 153 | +the [documentation][rbac]. |
| 154 | + |
| 155 | +For extra credit, you can also bind roles to groups. We can configure a claim |
| 156 | +from the JWT to be a group in addition to the email address. Imagine granting |
| 157 | +permissions on a cluster based on which teams a user is a part of. In fact, you |
| 158 | +can map almost anything from someone's Github profile directly over to a group. |
| 159 | +This way, you can setup permissions once and manage membership entirely through |
| 160 | +your OIDC provider. When using groups, the role binding ends up looking a little |
| 161 | +different: |
| 162 | + |
| 163 | +```yaml |
| 164 | +apiVersion: rbac.authorization.k8s.io/v1 |
| 165 | +kind: ClusterRoleBinding |
| 166 | +metadata: |
| 167 | + name: view |
| 168 | +roleRef: |
| 169 | + apiGroup: rbac.authorization.k8s.io |
| 170 | + kind: ClusterRole |
| 171 | + name: view |
| 172 | +subjects: |
| 173 | + - apiGroup: rbac.authorization.k8s.io |
| 174 | + kind: Group |
| 175 | + name: my-team |
| 176 | +``` |
| 177 | + |
| 178 | +[rbac]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/ |
| 179 | + |
| 180 | +## Bringing it Together |
| 181 | + |
| 182 | +So, what does this all mean? Well, it means that we've now got a central |
| 183 | +location to manage access to our cluster. If you're using groups, membership |
| 184 | +when the token is granted is mapped to a role binding that grants exactly what |
| 185 | +someone needs to work with your cluster. The IDs can be user friendly, so you |
| 186 | +can read through the `RoleBinding` YAML to understand what's allowed or not. If |
| 187 | +you're using `kty`, you don't even need any plugins or configuration! Your users |
| 188 | +can use `ssh` and immediately get access to the cluster. |
| 189 | + |
| 190 | +Please don't be afraid of auth! Don't continue to use incredibly complex systems |
| 191 | +that are hard to setup and/or easy to break. Say no to services that require |
| 192 | +blanket permissions like the Kubernetes dashboard. Use OIDC and make sure that |
| 193 | +users have exactly the permissions they need. |
0 commit comments