In this post, I’ll explain how you can get so comfortable that you will use it all the time, not only when you are asked to. So if you want to have less friction when data is involved (or should be involved), this post is for you!

Data-Driven Decision-Making - Overview

Data-Driven Decision Making (DDDM) transitioned from a trend to a “must-have” for every organization long ago. It’s an essential skill for teams to prioritize tasks, locate pain points, or optimize processes. This is important not only for decision-makers, but also for team members trying to drive decisions and gain influence.

Looking into the formality of DDDM, I’ve found many different definitions, all amounting to pretty much the same. One example is this definition from Tableau:

“[The act of ] using facts, metrics, and data to guide strategic business decisions that align with your goals, objectives, and initiatives.”

But we’re not here to talk about all the benefits or ways to implement DDDM - we’re here to talk about you, and how you can leverage the fact that this is how decisions are made in your company/team to your advantage.

First, we need to understand the basic structure of a “data project” and get some hands-on experience - and this is exactly what we’re going to do. In the next section, we’ll walk through the steps of a data project with a public database. This should give you the tools to perform “fun side projects” like this or implement the methods using your data and your team’s questions.

Data-Based Argument Cycle - Example

In this simulation, we’re working for a company that’s building a sports streaming product and, as part of the process, has built databases about basketball players and teams. We will go through all the steps of the process, explaining what exactly we’re doing in each step, and emphasizing where the process is probably different from what you might experience in your organization.

Defining the question

To define the question, we first need to define the problem, so let’s imagine a scenario:

You noticed that when ordering the users on their “Minutes Played” field, you get that the player who played the most minutes is Vin Baker, even though you know that’s not the case. You are convinced that something is off, but you want some proof to back this up. If you cannot access the code or if finding the issue in the code is difficult, you can try to back up your “this is a bug” argument with data!

Accessing the data

The first thing you should do is learn where your organization stores its data and how to access it (for example, the data is stored in AWS RDS and is accessible using an SQL interface).

In our example (and because we’re not actually working for a streaming company), we’ll get the data from a public source - https://www.basketball-reference.com. We’ll first run this Python code to get the data:

import pandas as pd

url = 'https://www.basketball-reference.com/leagues/NBA_1995_totals.html'
df = pd.read_html(url)[0] 

df.to_csv('all_players_1995.csv')

These lines will create a .csv file containing our database, which in this case will be a data frame (but for now is a table) where each row represents a player with their respective stats.

While understanding all aspects of the data is important, it is usually unnecessary to go into every detail of the first query. Usually, you’d want to get a general grip (badum-tss) and go into details only when a specific question is given. So let’s get to the question.

Querying

In the case of this example, we have a .csv file and not a fully-functioning DB with an SQL interface, so we’ll improvise such an interface. We’ll run the following Python lines creating a global DataFrame (called “df”) that will be accessible with SQL:

from pandasql import sqldf
import pandas as pd

df = pd.read_csv('all_players_1995.csv') # read the CSV 
df = df[df['Player'] != 'Player'] # drop unrelated rows (specific for this)
df['MP'] = df['MP'].astype(int) # convert column type (MP=Minutes Played)

Now that we’re able to run SQL queries, let's first test the initial problem and locate the top five players, which will show the following result:

It seems like our suspicions were right. Vin Baker does have the most minutes played! But feeling familiar with the data, we can notice that there is also a column called “Tm," representing the team of the player! This means that a player might have played for more than one team and thus have more than one row.

This is also easy to test using SQL, combining all rows of the same player and summing up their minutes:

q = ''' SELECT Player, SUM(MP)
FROM df
GROUP BY Player
ORDER BY 2 DESC
LIMIT 5'''
sqldf(q, globals())

Which presents the following result:

Bingo!

Presenting results

Showing the tables above along with an explanation would have been enough to make your case, but in other cases, we may want to drive the point further with illustrations of the data. This is also relatively easy to do, like in this example showing the correlation between minutes played and points scored (column name is PTS):

import plotly.express as px

q = ''' SELECT Player, SUM(MP) as MP, SUM(PTS) as PTS
FROM df
GROUP BY Player'''

agg_df = sqldf(q, globals())
fig = px.scatter(agg_df, x="MP", y="PTS")
fig.show()

Which will show the following graph:

Just Another Tool

Imagine you’re a developer, and you’ve found a bug in the data presented to a client. Instead of starting a cycle of meetings with your manager, a PM, and an analyst - you can come prepared for the first meeting with answers to many of the questions that they might ask:

1. How badly is the client affected?

2. How many other clients were affected?

3. How will fixing the issue affect the database?

The goal is not to master data but to feel comfortable enough with each part of the process so it feels easy. Even if you have a way around the issue, it will slow you down in the long run - it’s like debugging with “prints” because you don’t feel comfortable with the debugger.

As I said, I adopted data as a tool relatively late, not for lack of knowledge. But intuitively enough, the more I went through the cycle, the easier it became, and the quicker I found myself using it again. I now use the data in my organization as easily as I use Google Search. It’s just another tool.

So the next time you feel like you’d want to check something in your environment, I recommend you go through these steps and answer them on your own. Asking for help is okay (and even recommended), but ensure it’s only help and not outsourcing.

And if you can’t wait until you find the next opportunity in your team (or you’re afraid you’ll embarrass yourself, which you won’t), you can always use the code here and research some basketball. As they say - “practice makes perfect” :)

🕸️ Chrome Extension Logging: Architecture

Our journey began with the challenge of understanding how to enable logging across all parts of a Chrome extension effectively. Since our goal is to stream all of the logs to a centralized location, we've started exploring different techniques to capture and consolidate logs from content scripts, pop-ups, and background scripts. By understanding the logging capabilities and limitations of each component, our goal was to create a unified and comprehensive logging solution.

One challenge we encountered during our journey was that content scripts in Chrome extensions may face restrictions imposed by Content Security Policies (CSPs), which can block HTTP requests required for logging purposes. To overcome this limitation, we found it necessary to centralize the logging mechanism within the service worker. By leveraging the capabilities of the service worker, we were able to capture and consolidate the logs from content scripts and other components, ensuring uninterrupted logging even in the presence of CSP restrictions.

🎥 Streaming Extension Logs to Datadog

Now that we've got all of the logs in one centralized location (our service worker), we want to start streaming them to Datadog. We assumed that it would be very simple and straightforward to use the Datadog Browser Log Collection. This assumption was a major mistake.

🫥 The Challenge - Missing document and window

We've finished implementing the entire architecture as described. There was only one thing left and it was using @datadog/browser-logs to stream the logs to Datadog.

We've implemented the basic functionality and were disappointed to discover the following error:

It seems like the Datadog logging library for the web requires document and window configured in the global context. Since we are running inside a service worker, both of these variables don't exist in our context.

A short Google search raised the following open Github issues in the Datadog logging library

Despite this setback, we saw the issue as surmountable. We began implementing minor polyfills for document and window to get the library working. Although the library stopped raising exceptions after implementing the necessary polyfills, it didn't send the logs to Datadog.

At this point, we've come to the understanding that we need to rethink our process and find new ways to achieve the same goal.

After some brainstorming, we've come up with the following options:

• ❌ Stream the logs from one of the content scripts instead of the service worker

  • CSP constraints on content scripts are very limiting

  • Many content scripts can be loaded at the same time to multiple tabs - which requires a complex architecture

• ❌ Find another logging library and platform that supports Chrome extensions

  • There's absolutely no logging library for Chrome extensions - All existing logging libraries are either for regular websites or for Node.JS

• ❓ Upload the logs to our backend which in turn will stream the logs to Datadog

  • This will dramatically increase the load on our servers

• ❓ Implement the same API requests as Datadog's logging SDK performs

  • This is an "undocumented" browser-intake API. It may break or change at any moment without us knowing.

• ❓ Check out if there's an official Datadog API we can call directly instead of using a logging library using @datadog/datadog-api-client

  • There is an official API for logging - Datadog Logs API reference

  • According to the documentation, the API requires an API key and not a client token.

    • Datadog API keys are private, we can't use them in our extension since they will be accessible to everyone

      For security reasons, API keys cannot be used to send data from a browser, as they would be exposed client-side in the JavaScript code. Instead, web browsers and other clients use client tokens to send data to Datadog.

✅ The Solution - Using Datadog API

During the research process, we discovered an interesting insight. Datadog client tokens and API keys are very similar to each other.

We can see that the client token structure is identical to the API key (32-byte hex encoded string). The only difference is that the client token starts with pub which probably stands for "public".

Based on these findings, we decided to try using the official Datadog logging API with the client token instead of the API key.

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration({
  authMethods: {
    apiKeyAuth: "pub**********", // Our client token
  },
});

const apiInstance = new v2.LogsApi(configuration);

const params: v2.LogsApiSubmitLogRequest = {
  body: [
    {
      ddsource: "grip",
      ddtags: "env:staging,version:1.33.7",
      message: "Hello World",
      service: "guyg",
    },
  ],
};

await apiInstance.submitLog(params);

And it worked! 🎉

🔄 Future Considerations

Given our success with this approach, we'll likely keep using it in the future. However, it's important to remember that our workaround relies on a certain similarity between client tokens and API keys. If Datadog changes how client tokens or API keys are structured, our approach may need to be revised.

Moreover, since our workaround employs an 'undocumented' method - using the client token in place of the API key - there could be hidden limitations or restrictions we haven't yet encountered. As a result, we must continuously monitor our logging system and remain prepared for potential troubleshooting and adjustments.

🧩 Final Thoughts

Logging in Chrome extensions and integrating with Datadog presented unique challenges because of specific architecture and platform issues. But, with some creative thinking and technical know-how, we managed to build a solution that works for us.

Our experience shows that problem-solving, understanding your tools, and determination can help overcome tough challenges. Our unique solution for Chrome extension logging is proof that it's possible to find new ways to tackle problems when we're willing to think differently.

We hope our story can help others facing similar challenges in their work. With the right mindset and a willingness to try new approaches, you can solve any problem that comes your way!

📚 References

• Chrome Extensions Documentation

• Datadog Browser Log Collection

• Datadog Logs API Reference

• GitHub: DataDog/browser-sdk Issue #432

• Datadog API Keys and Client Tokens Documentation

We were already using PostgreSQL in a different database, so we figured we could use it in the centralized one as well. I was tasked with spearheading the migration project – defining the new database, writing the necessary code, and converting our system to use the new DB.

This blog post explains our process of migrating our centralized database architecture from MongoDB to PostgreSQL, without too much focus on data transfer. Let’s dive in!

First step: Defining the DB requirements using UML diagrams

Our Mongo database had several unused collections and excess data that no one was querying. We wanted to improve our data model and introduce new relations between our objects. So, the first step was to design the new desired database scheme, based on the existing data structure.

The clearest way of creating the DB structure and explaining the required relations was using a UML diagram. UML (Unified Modeling Language) was created to standardize software systems design. We used structural UML diagrams to help visualize the models (i.e., tables) with their properties (i.e., columns) and the relationships between them using specific connectors and arrows.

Explanation: The tables highlighted in blue are collections that existed in MongoDB with a similar name. The white ones are new tables to support our models. Note that we have different relationships between models: one-to-many (interactions to saas_applications), and many-to-many (insight_to_interaction, as demonstrated by the linking table).

Using a UML diagram, we could ensure we were creating the correct data structure and could update it without writing (or deleting) a single line of code.

Second step: Defining the initial ORM using SQLAlchemy and Alembic

After we were content with the database structure we designed in the chart, it was time to define these tables in our code. Our product’s backend is mostly written in Python, so we used the SQLAlchemy package to define our ORM¹. SQLAlchemy is a Python package that allows you to create Python classes with columns as their properties to define the database tables. It also provides a clear API to query the tables using these classes.

class CorpusInteraction(Base, TimestampMixin):
    __tablename__ = "interactions"
    id = Column(
        UUID(as_uuid=True),
        primary_key=True,
        default=uuid.uuid4,
        server_default=func.gen_random_uuid(),
    )
    event_type = Column(EVENT_TYPE, nullable=False)
    interaction_type = Column(INTERACTION_TYPE, nullable=False)
    auto_generated = Column(
        Boolean, default=False, server_default=false(), nullable=False
    )
    saas_application_id = Column(
        UUID(as_uuid=True),
        ForeignKey(CorpusSaaSApplication.id),
        default=null()
    )
    saas_application: CorpusSaaSApplication = relationship(
        "CorpusSaaSApplication", back_populates="tagged_interactions"
    )
    insights = relationship(
        "CorpusInsight",
        secondary=insights_to_interactions,
        back_populates="interactions",
        lazy="joined",
    )

Explanation: This is a class that represents the interactions table (as evident in the __tablename__ variable), and each member of the class represents a column of the model. For convenience, we can also define relationships as members of the class, that depend on foreign key columns. For example, saas_application is a relationship that is based on the saas_application_id column in the table.

We also needed a way to automatically create the DB tables and track changes made to their structure. That’s where Alembic came in handy. Alembic is a Python package that was made to be used with SQLAlchemy to detect changes made to the ORM and create migrations to be applied to the database. You can think of it as “git” for database changes.

Alembic compares the current defined state of the ORM classes to the one existing in the database and generates the revision automatically (!) listing the changes to be made².

Once we defined an Alembic workspace for the new database, creating the database (in an initial revision) was as simple as a single command line:

> alembic revision --autogenerate -m "v1"
> alembic upgrade head

Further changes to the structure will be handled in the same manner – creating a new revision and upgrading the database. Easy as pie!

Reference notes:

1. We chose to define our ORM model in Python, but any other ORM technology could be used instead - if you’re using mostly JavaScript/TypeScript code, Prisma is a cutting-edge alternative that defines the ORM model, generates code automatically for the created tables and handles DB changes by creating migrations automatically.

2. It is considered good practice to go over the code of Alembic’s auto-generated migrations and make changes if necessary.

Third step: Migrating the data and code to the new database

After we had settled on our desired database model, defined all the necessary relationships, and created an initial migration, it was time to move the data itself - and then came the fun part!

We didn’t need a super-generic solution here – a simple Python script that could read the MongoDB data and build the ORM objects accordingly would suffice! We could use any external third-party software for this task, but we were content with writing a simple once-off script that did the job. We first ran the script locally on a new database to test for failures and bugs, and only then, ran it on our intended production database.

The final task was the most demanding in terms of time and effort. Our centralized DB is used in the production code of our main product, so we needed to make sure the transition was as smooth as possible. To also get up and running quickly with the new PostgreSQL database, we created a module in our code that acted as a “compatibility layer.” Its job was to replace the previous MongoDB functions and calls with SQLAlchemy ones, but convert their results and output to the previous format. We created it so the existing code could still operate without failing because we switched the database infrastructure underneath.

As soon as the compatibility layer was complete and we observed that the production code was still working, we began the important final transition. We slowly switched our code to use the main SQLAlchemy functions and got rid of the compatibility layer - most of them had the same names and parameters!

In a matter of weeks, after we created the populated database, we could use it properly and had gotten rid of all the old Mongo code!

Closing thoughts

Our migration process revolved specifically around MongoDB and PostgreSQL, and it worked quite well in our case. If we were to switch again to a different database infrastructure, we would probably need to build a strong foundation for the new technology first. Because we were already working with PostgreSQL, creating a new DB was easy.

I’m sure there are other things we could improve upon in our process, and I would love to receive your feedback in a comment or even by reaching out via email.

In the meantime, enjoy your fully migrated PostgreSQL database!

Growing pains are hard. Your organization is growing, apps are succeeding, and your engineering team has tripled—now, manage your Amazon Kubernetes Service (EKS) and permissions for an expansive team of non-stop complexity. Growing pains are hard.

At Grip, we encountered similar frustrations with managing our EKS cluster permissions due to our growing number of engineers and the increasing complexity of our systems. The time invested in managing our EKS permissions model grew significantly, encouraging us to seek out a comprehensive and sustainable solution to this problem.

📄 TL;DR

• Kubernetes has 2 main methods for authorization:
  • RBAC - Role-based access control
  • ABAC - Attribute-based access control
• Authentication can be done in multiple methods.
  • EKS is configured to use aws-iam-authenticator which allows Kubernetes to integrate with AWS IAM.
• There are two configurations required in order to define authentication and authorization:
  • Authorization - A RoleBinding or a ClusterRoleBinding
  • Authentication - A ConfigMap named aws-auth which is under the kube-system namespace
• There is no standardized method for providing IAM group access to an EKS cluster or namespace.
• Our solution utilizes an IAM role to authenticate the user group automatically and transparently when kubectl is being used.

🛂 RBAC Overview

If you are familiar with Kubernetes RBAC, you can go ahead and jump to EKS RBAC Integration with IAM

Kubernetes's access control naming convention is similar to AWS IAM, so make sure you don't confuse the two.

• Kubernetes Role - A set of permissions for a namespace

• Kubernetes ClusterRole - A set of permissions for an entire cluster

• Kubernetes RBAC RoleBinding - A binding between one Role and one or more Users or Groups

• Kubernetes RBAC ClusterRoleBinding - A binding between one ClusterRole and one or more Users or Groups

Since this isn’t really a “Getting started with RBAC” kind of post. If you’re still not sure about the terminology, Check out Kubernetes RBAC docs for further reading.

🔑 EKS RBAC Integration with IAM

Now that we’ve finished our overview of Kubernetes RBAC, let’s dive right in and see how it integrates with AWS IAM. The general idea is:

1. aws-iam-authenticator authenticates an IAM identity.

   • This is achieved by token authentication webhook.

2. aws-iam-authenticator translates the authenticated identity to a Kubernetes RBAC identity.

3. Each IAM identity is mapped to a Kubernetes username or added to a group.
4. Kubernetes RBAC uses these translated identities to enforce permissions to different resources.
   • Kubernetes RBAC is not aware of IAM identities.

AWS EKS uses a specific ConfigMap named aws-auth to manage the AWS Roles and AWS Users who are allowed to connect and manage the cluster (or namespace).

This is an example of aws-auth.yaml file:

Authentication and Authorization Flow

👨‍👩‍👦 Adding an IAM group to EKS RBAC

According to AWS official documentation, aws-auth.yaml doesn’t support IAM groups translation to RBAC.

We conducted comprehensive research to verify this fact, and we found that there is no official documentation on the full format specification of aws-auth.yaml. So we read the implementation of aws-iam-authenticator to find alternative solutions.

We have built a full format specification based on the implementation - aws-auth-specification.yaml

As you can see, aws-auth supports three IAM entities mapAccounts, mapUsers and, mapRoles but none of them allow us to map an IAM group.

Therefore, it is not possible to define an IAM group for RBAC using standard methods.

Furthermore, we found multiple open issues on GitHub asking for help with the same issue:

• GitHub[aws-iam-authenticator] - Can I not add an IAM group to my ConfigMap?
• Github[containers-roadmap] - [EKS] [request]: MapGroups in ConfigMap
• GitHub[aws-iam-authenticator] - How to use IAM Groups?

✅ Our Chosen Solution

After comparing the different options, we decided to use the most robust solution we found, which is the “assume role” method.

General Idea

1. Create an IAM role - dev.eks-access.role
2. Create a policy that allows it to manage EKS and attach it to the role - dev.eks-access.policy
3. Add the role ARN under the mapRoles section of aws-auth.yaml.
4. Create a policy that allows assuming the role and attach it to your R&D group - dev.assume-eks-access-role.policy

Solution Downside

The main downside of this solution is that R&D team members must assume the dev.eks-access.role every time they want to interact with the cluster. A common workaround is to use AWS profiles, but this also requires changing the profile before accessing the cluster.

We wanted to get rid of this constraint. After some brainstorming, we assumed that the way aws-iam-authenticator works on our dev machines allows it to assume a role on the fly, and only for Kubernetes-related operations.

Solving the Downside - .kube/config Authentication Configuration

The way that kubectl (and other Kubernetes operations) authenticate to the cluster is using the configuration described in .kube/config.

This way, the kubectl command knows how to authenticate to EKS using the aws-iam-authenticator

users:
- name: arn:aws:eks:us-east-2:123456789123:cluster/dev.my-eks-cluster
  user:
    exec:
      apiVersion: client.authentication.Kubernetes.io/v1alpha1
      args:
      - --region
      - us-east-2
      - eks
      - get-token
      - --cluster-name
      - dev.my-eks-cluster
      command: aws
      env: null

This authentication configuration is equivalent to the following command

aws --region us-east-2 eks get-token --cluster-name dev.my-eks-cluster

So, if we could assume the role only in this context, all the Kubernetes operations will be done with the assumed role, but other AWS operations will be done with the configured R&D user.

And as we expected, both get-token and update-kubeconfig commands support --role-arn flag

OPTIONS
     . . .
    --role-arn (string)  
        Assume this role for credentials when signing the token.
    . . .

Finally! we can just use --role-arn and assume the role on the fly only for Kubernetes operations! This eliminated our previous friction points 🥳

Step-by-Step Guide

This guide gives an example of how this should be done; however, it's suggested that a more strict permission model be implemented for production environments.

Create the Necessary IAM Entities

First, we need to create all the needed IAM entities.

Create dev.eks-access.role Role

Create the role and give it the following trust relationship:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789123:root"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
        }
    ]
}

Create dev.assume-eks-access-role.policy Policy

This policy allows an IAM entity to assume the dev.eks-access.role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::123456789123:role/dev.eks-access.role"
        }
    ]
}

Create dev.eks-access.policy Policy

This is an example of a policy that effectively allows full access to access EKS

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "eks:ListClusters",
                "eks:DescribeAddonVersions",
                "eks:CreateCluster"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "eks:*",
            "Resource": "arn:aws:eks:*:123456789123:cluster/*"
        }
    ]
}

Connect the Dots

Inside AWS IAM management console do the following:

• Attach dev.eks-access.policy to dev.eks-access.role
• Attach dev.assume-eks-access-role.policy to your desired group (e.g. dev.rnd.group)

Configure EKS to Work With the Role

Add the role to the cluster aws-auth

eksctl create iamidentitymapping --cluster {CLUSTER_NAME} --arn arn:aws:iam::123456789123:role/dev.eks-access.role --group system:masters --username aws_eks_access_role

Create a cluster-role-binding.yaml

apiVersion: rbac.authorization.Kubernetes.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: null
  name: entire-cluster-admin-access
roleRef:
  apiGroup: rbac.authorization.Kubernetes.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - apiGroup: rbac.authorization.Kubernetes.io
    kind: User
    name: aws_eks_access_role

kubectl apply -f cluster-role-binding.yaml

Update the Kubernetes config file (.kube/config)

After all the entities exist and all configurations are set, all you need to do is to configure .kube/config to be able to access the EKS cluster using this command:

aws eks --region us-east-2 update-kubeconfig --name {CLUSTER_NAME} --role-arn arn:aws:iam::123456789123:role/dev.eks-access.role

🗺️ Other Possible Solutions

These are some other solutions that solve this issue from a different perspective:

• Use Terraform to “manually” give access to all users:
  • This solution isn’t comprehensive, as Terraform isn’t used by all organizations. Source - Can I not add an IAM group to my ConfigMap? · Issue #176 · kubernetes-sigs/aws-iam-authenticator · GitHub
• Deploy iam-eks-user-mapper
  • This project runs as a container in the cluster and modifies the configurations on the fly.
• Use AWS Lambda to periodically update all cluster permissions.

🌯 Wrap Up

This blog post was a comprehensive overview intended to solve a seemingly very simple problem.

By acquiring the relevant knowledge and understanding of some internal implementations, we managed to find a good workaround that provides a great solution to this problem. We hope that other organizations find value in this information, and use it to simplify and streamline their workloads using Kubernetes.

If you are interested in becoming a member of our team, we are hiring!

📖 Related documentation

• Enabling IAM user and role access to your cluster - Amazon EKS
• Using RBAC Authorization | Kubernetes
• Authorization Overview | Kubernetes
• EKS (AWS) AND RBAC, step by step. Introduction | by David De Juan Calvo | Globant | Medium
• GitHub - kubernetes-sigs/aws-iam-authenticator
• Mapping IAM Groups to EKS User Access | by Zach Arnold | Ygrene Tech
• Troubleshooting IAM - Amazon EKS
• Kubernetes RBAC and IAM Integration in Amazon EKS using a Java-based Kubernetes Operator | Containers
• https://aws.amazon.com/premiumsupport/knowledge-center/eks-kubernetes-object-access-error
• Identity and Access Management - EKS Best Practices Guides
• GitHub - keikoproj/aws-auth: Manage the aws-auth config map for EKS Kubernetes clusters