# Enabling AWS IAM Group Access to an EKS Cluster Using RBAC

## 👋 Introduction

Growing pains are hard. Your organization is growing, apps are succeeding, and your engineering team has tripled—now, manage your Amazon Kubernetes Service (EKS) and permissions for an expansive team of non-stop complexity. Growing pains are hard.

At Grip, we encountered similar frustrations with managing our EKS cluster permissions due to our growing number of engineers and the increasing complexity of our systems. The time invested in managing our EKS permissions model grew significantly, encouraging us to seek out a comprehensive and sustainable solution to this problem.

## 📄 TL;DR

* Kubernetes has 2 main methods for **authorization:**
  * **[RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)** - Role-based access control
  * [ABAC](https://kubernetes.io/docs/reference/access-authn-authz/abac/) - Attribute-based access control
* **Authentication** can be done in multiple methods.
  * EKS is configured to use [aws-iam-authenticator](https://github.com/kubernetes-sigs/aws-iam-authenticator#how-does-it-work) which allows Kubernetes to integrate with AWS IAM.
* There are two configurations required in order to define authentication and authorization:
  * Authorization - A `RoleBinding` or a `ClusterRoleBinding`
  * Authentication - A `ConfigMap` named `aws-auth` which is under the `kube-system` namespace
* **There is *no standardized method* for providing IAM group access to an EKS cluster or namespace.**
* Our solution utilizes an IAM role to authenticate the user group automatically and transparently when `kubectl` is being used.

## 🛂 RBAC Overview

If you are familiar with Kubernetes RBAC, you can go ahead and jump to [EKS RBAC Integration with IAM](#heading-eks-rbac-integration-with-iam)

Kubernetes's access control naming convention is similar to AWS IAM, so make sure you don't confuse the two.

* Kubernetes `Role` - A set of permissions for a namespace

%[https://gist.github.com/guygrip/f42044ec56c552ccb0ceb39410c99a31]

* Kubernetes `ClusterRole` - A set of permissions for an entire *cluster*

%[https://gist.github.com/guygrip/970df7ee606ebd01addb574203cf4b36]

* Kubernetes RBAC `RoleBinding` - A binding between **one** `Role` and **one or more** Users or Groups

%[https://gist.github.com/guygrip/db9461fef30d01e03b3992c472b4faae]

* Kubernetes RBAC `ClusterRoleBinding` - A binding between **one** `ClusterRole` and **one or more** Users or Groups

%[https://gist.github.com/guygrip/5c52659f7e3a5cf16873a4a1f8efbfca]

Since this isn’t really a “Getting started with RBAC” kind of post. If you’re still not sure about the terminology, Check out [Kubernetes RBAC docs](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for further reading.

## 🔑 EKS RBAC Integration with IAM

Now that we’ve finished our overview of Kubernetes RBAC, let’s dive right in and see how it integrates with AWS IAM. The general idea is:

1. `aws-iam-authenticator` authenticates an IAM identity.
   * This is achieved by [token authentication webhook](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication).

2. `aws-iam-authenticator` translates the authenticated identity to a Kubernetes RBAC identity.
3. Each IAM identity is mapped to a Kubernetes username or added to a group.
4. Kubernetes RBAC uses these translated identities to enforce permissions to different resources.
   * Kubernetes RBAC is not aware of IAM identities.

AWS EKS uses a specific `ConfigMap` named `aws-auth` to manage the **AWS** Roles and **AWS** Users who are allowed to connect and manage the cluster (or namespace).

This is an example of `aws-auth.yaml` file:

%[https://gist.github.com/guygrip/a20001c8a3eade08f49a34605a303a37]

### Authentication and Authorization Flow

%%[eks-rbac-image-auth-diagram]

## 👨‍👩‍👦 Adding an IAM group to EKS RBAC

According to AWS official documentation, `aws-auth.yaml` doesn’t support IAM groups translation to RBAC.

We conducted comprehensive research to verify this fact, and we found that there is no official documentation on the full format specification of `aws-auth.yaml`. So we read the implementation of `aws-iam-authenticator` to find alternative solutions.

We have built a full format specification based on the implementation - [`aws-auth-specification.yaml`](https://gist.github.com/guygrip/dc64629887d39d4fa7696936be07f71f)

As you can see, `aws-auth` supports three IAM entities `mapAccounts`, `mapUsers` and, `mapRoles` but none of them allow us to map an IAM group.

**Therefore, it is not possible to define an IAM group for RBAC using standard methods.**

Furthermore, we found multiple open issues on GitHub asking for help with the same issue:

* [GitHub[aws-iam-authenticator] - Can I not add an IAM group to my ConfigMap?](https://github.com/kubernetes-sigs/aws-iam-authenticator/issues/176)
* [Github[containers-roadmap] - [EKS] [request]: MapGroups in ConfigMap](https://github.com/aws/containers-roadmap/issues/150)
* [GitHub[aws-iam-authenticator] - How to use IAM Groups?](https://github.com/kubernetes-sigs/aws-iam-authenticator/issues/262)

## ✅ Our Chosen Solution

After comparing the [different options](#header-other-possible-solutions), we decided to use the most robust solution we found, which is the “assume role” method.

### General Idea

1. Create an IAM role - `dev.eks-access.role`
2. Create a policy that allows it to manage EKS and attach it to the role - `dev.eks-access.policy`
3. Add the role ARN under the `mapRoles` section of `aws-auth.yaml`.
4. Create a policy that allows assuming the role and attach it to your R&D group - `dev.assume-eks-access-role.policy`


%%[eks-rbac-image-aws-iam-relation]

#### Solution Downside

The main downside of this solution is that R&D team members must assume the `dev.eks-access.role` every time they want to interact with the cluster. A common workaround is to use AWS profiles, but this also requires changing the profile before accessing the cluster.

We wanted to get rid of this constraint. After some brainstorming, we assumed that the way `aws-iam-authenticator` works on our dev machines allows it to assume a role on the fly, and only for Kubernetes-related operations.

#### Solving the Downside - `.kube/config` Authentication Configuration

The way that `kubectl` (and other Kubernetes operations) authenticate to the cluster is using the configuration described in `.kube/config`.

This way, the `kubectl` command knows how to authenticate to EKS using the `aws-iam-authenticator`

```yaml
users:
- name: arn:aws:eks:us-east-2:123456789123:cluster/dev.my-eks-cluster
  user:
    exec:
      apiVersion: client.authentication.Kubernetes.io/v1alpha1
      args:
      - --region
      - us-east-2
      - eks
      - get-token
      - --cluster-name
      - dev.my-eks-cluster
      command: aws
      env: null
```

This authentication configuration is equivalent to the following command

```bash
aws --region us-east-2 eks get-token --cluster-name dev.my-eks-cluster
```

So, if we could assume the role only in this context, all the Kubernetes operations will be done with the assumed role, but other AWS operations will be done with the configured R&D user.

And as we expected, both `get-token` and `update-kubeconfig` commands support `--role-arn` flag

```bash
OPTIONS
     . . .
    --role-arn (string)  
        Assume this role for credentials when signing the token.
    . . .
```

Finally! we can just use `--role-arn` and assume the role on the fly **only for Kubernetes operations**! This eliminated our previous friction points 🥳

%%[eks-rbac-image-auth-diagram-role-arn]


### Step-by-Step Guide

This guide gives an example of how this should be done; however, it's suggested that a more strict permission model be implemented for production environments.

#### Create the Necessary IAM Entities

First, we need to create all the needed IAM entities.

 **Create `dev.eks-access.role` Role**

<table>
<tr>
<td>
ℹ️
</td>
<td>
We give the `root` account trust to assume the role. This means that the role will allow everyone in the account to assume the role **from the role perspective**.<br />
This doesn’t mean that everyone in the account can assume the role, because only users with a policy that allows them to assume the role will be able to do so.
</td>
</tr>
</table>


Create the role and give it the following trust relationship:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789123:root"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
        }
    ]
}
```

 **Create `dev.assume-eks-access-role.policy` Policy**

This policy allows an IAM entity to assume the `dev.eks-access.role`.

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::123456789123:role/dev.eks-access.role"
        }
    ]
}
```

 **Create `dev.eks-access.policy` Policy**

This is an example of a policy that effectively allows full access to access EKS

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "eks:ListClusters",
                "eks:DescribeAddonVersions",
                "eks:CreateCluster"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "eks:*",
            "Resource": "arn:aws:eks:*:123456789123:cluster/*"
        }
    ]
}
```

**Connect the Dots**

Inside AWS IAM management console do the following:

* Attach `dev.eks-access.policy` to `dev.eks-access.role`
* Attach `dev.assume-eks-access-role.policy` to your desired group (e.g. `dev.rnd.group`)

#### Configure EKS to Work With the Role

 **Add the role to the cluster `aws-auth`**

```bash
eksctl create iamidentitymapping --cluster {CLUSTER_NAME} --arn arn:aws:iam::123456789123:role/dev.eks-access.role --group system:masters --username aws_eks_access_role
```
<table>
<tr>
<td>
⚠️
</td>
<td>
We give the role `system:masters` permissions, meaning the role can modify **everything** in the cluster. Be cautious!
</td>
</tr>
</table>
 **Add the Role to the `ClusterRoleBinding`**

Create a `cluster-role-binding.yaml`

```yaml
apiVersion: rbac.authorization.Kubernetes.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: null
  name: entire-cluster-admin-access
roleRef:
  apiGroup: rbac.authorization.Kubernetes.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - apiGroup: rbac.authorization.Kubernetes.io
    kind: User
    name: aws_eks_access_role
```
<table>
<tr>
<td>
⚠️
</td>
<td>
We give the role `cluster-admin` permissions, meaning the role can modify **everything** in the cluster. Be cautious!
</td>
</tr>
</table>
Run this command to apply the new `ClusterRoleBinding` on the Kubernetes cluster:

```bash
kubectl apply -f cluster-role-binding.yaml
```

#### Update the Kubernetes config file (`.kube/config`)

After all the entities exist and all  configurations are set, all you need to do is to configure `.kube/config` to be able to access the EKS cluster using this command:

```bash
aws eks --region us-east-2 update-kubeconfig --name {CLUSTER_NAME} --role-arn arn:aws:iam::123456789123:role/dev.eks-access.role
```

## 🗺️ Other Possible Solutions

These are some other solutions that solve this issue from a different perspective:

* **Use Terraform to “manually” give access to all  users:**
  * This solution isn’t comprehensive, as Terraform isn’t used by all organizations. Source - [Can I not add an IAM group to my ConfigMap? · Issue #176 · kubernetes-sigs/aws-iam-authenticator · GitHub](https://github.com/kubernetes-sigs/aws-iam-authenticator/issues/176#issuecomment-508562203)
* **Deploy [iam-eks-user-mapper](https://github.com/ygrene/iam-eks-user-mapper)**
  * This project runs as a container in the cluster and modifies the configurations on the fly.
* **Use AWS Lambda to periodically update all cluster permissions.**

## 🌯 Wrap Up

This blog post was a comprehensive overview intended to solve a seemingly very simple problem.

By acquiring the relevant knowledge and understanding of some internal implementations, we managed to find a good workaround that provides a great solution to this problem. We hope that other organizations find value in this information, and use it to simplify and streamline their workloads using Kubernetes.

If you are interested in becoming a member of our team, we are hiring!

## 📖 Related documentation 

* [Enabling IAM user and role access to your cluster - Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html)
* [Using RBAC Authorization | Kubernetes](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
* [Authorization Overview | Kubernetes](https://kubernetes.io/docs/reference/access-authn-authz/authorization/)
* [EKS (AWS) AND RBAC, step by step. Introduction | by David De Juan Calvo | Globant | Medium](https://medium.com/globant/rbac-and-eks-aws-step-by-step-e2f9c38f1aeb)
* [GitHub - kubernetes-sigs/aws-iam-authenticator](https://github.com/kubernetes-sigs/aws-iam-authenticator#full-configuration-format)
* [Mapping IAM Groups to EKS User Access | by Zach Arnold | Ygrene Tech](https://ygrene.tech/mapping-iam-groups-to-eks-user-access-66fd745a6b77)
* [Troubleshooting IAM - Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/troubleshooting_iam.html#security-iam-troubleshoot-ConfigMap)
* [Kubernetes RBAC and IAM Integration in Amazon EKS using a Java-based Kubernetes Operator | Containers](https://aws.amazon.com/blogs/containers/kubernetes-rbac-and-iam-integration-in-amazon-eks-using-a-java-based-kubernetes-operator/)
* [https://aws.amazon.com/premiumsupport/knowledge-center/eks-kubernetes-object-access-error](https://aws.amazon.com/premiumsupport/knowledge-center/eks-kubernetes-object-access-error/)
* [Identity and Access Management - EKS Best Practices Guides](https://aws.github.io/aws-eks-best-practices/security/docs/iam/)
* [GitHub - keikoproj/aws-auth: Manage the aws-auth config map for EKS Kubernetes clusters](https://github.com/keikoproj/aws-auth)