Containerized workloads on Amazon EKS often need access to sensitive credentials. A pod running a backend service might need a database connection string. A payment processor might require API keys. A message consumer might need authentication tokens for an external queue. In each case, the workload needs a way to retrieve secrets securely at runtime without hardcoding them into container images or Kubernetes manifests.
The default mechanism for storing secrets in Kubernetes is the Secret resource, which persists data in etcd. In this use case, the EKS cluster encrypts Kubernetes secrets at rest using a dedicated KMS key (configured in Article 1: Provision a secure Amazon EKS Cluster), but the secrets would still reside inside the cluster. These secrets are base64-encoded (not encrypted) in YAML manifests, require manual rotation, lack native audit trails, and can be read by anyone with RBAC access to the namespace. Across multiple teams and environments, secrets spread across clusters without centralized visibility or a rotation strategy.
AWS Secrets Manager addresses these limitations by providing a centralized, external store for credentials. It encrypts secrets at rest and in transit, controls access through IAM policies with full CloudTrail audit logging, and supports automatic rotation. The remaining challenge is delivering those externally stored secrets to a running pod without copying them into Kubernetes. The Secrets Store CSI Driver solves that by treating external secrets as a volume mount.
Secrets Store CSI Driver is a Kubernetes Container Storage Interface (CSI) driver that intercepts volume requests from workload pods and delegates the actual secret retrieval to a cloud-specific provider. For AWS, the AWS Secrets Store CSI Driver Provider handles the communication with Secrets Manager (and Parameter Store). When a pod starts and requests a CSI secret volume, the provider uses the pod’s own identity (via EKS Pod Identity) to fetch the secret from AWS and mount it as a file inside the pod’s filesystem. The secret does not pass through etcd.
In this article, I’ll create an IAM role with Pod Identity for a demo application to read from AWS Secrets Manager, install the Secrets Store CSI Driver and the AWS provider via Helm, and verify the integration by mounting a secret from AWS Secrets Manager into a pod using Terraform and GitHub Actions.
Solution Overview
This solution adds the ability to integrate secrets into the existing EKS cluster. The changes span two layers of the multi-configuration Terraform architecture:
Infrastructure configuration (infrastructure/): Create an IAM role with a Pod Identity trust policy, attach an inline policy granting secretsmanager:GetSecretValue and secretsmanager:DescribeSecret scoped to secrets under the ${var.name}/* prefix, and create the Pod Identity association that maps the role to the demo application’s service account in the default namespace.
Platform configuration (platform/): Install the Secrets Store CSI Driver (chart v1.6.0) and the AWS Secrets Store CSI Driver Provider (chart v3.1.1) as Helm releases in the kube-system namespace.
To verify this implementation, I used a dedicated GitHub Actions workflow to create a secret in AWS Secrets Manager, deploy a SecretProviderClass and a demo pod, and then read and display the mounted secret file from inside the pod to confirm end-to-end functionality.
The implementation follows five steps:
1. Create the IAM role with a Pod Identity trust policy allowing pods.eks.amazonaws.com to assume it.
2. Attach permissions granting the role access to secrets stored under the application’s prefix in Secrets Manager.
3. Create the Pod Identity association linking the IAM role to the demo application’s service account.
4. Install the Secrets Store CSI Driver and the AWS Provider via Helm in the platform configuration.
5. Verify by creating a secret, deploying a SecretProviderClass and demo pod, and confirming the secret is mounted.
You can find the complete implementation in my GitHub repository: kunduso-org/aws-eks-terraform (branch: csi-secrets-store). The code includes Terraform configurations, GitHub Actions CI/CD, and security scanning with Checkov.
Prerequisites
This use case builds on the first article in this series, Provision a Secure Amazon EKS Cluster using Terraform and GitHub Actions. Please go through that before starting here. The csi-secrets-store branch also includes components from Article 2, Article 3, and Article 4. Those components are not required for secrets integration, but the branch builds on them.
Understanding EKS Pod Identity (Article 5) is helpful context for this article, as the secrets integration relies on Pod Identity to grant the demo pod AWS credentials.
Implementation
Step 1: Create the IAM role with a Pod Identity trust policy
In the previous articles, IAM roles were created for the EKS cluster, the EC2 instances acting as nodes, and platform components such as the AWS Load Balancer Controller and Karpenter. In this use case, the IAM role serves the standalone workload (the demo application). The assume_role_policy follows the same pattern as the platform components, allowing the service principal pods.eks.amazonaws.com to perform sts:AssumeRole and sts:TagSession.
Step 2: Attach permissions for Secrets Manager access
The inline policy grants two actions, secretsmanager:DescribeSecret and secretsmanager:GetSecretValue, scoped to secrets under the ${var.name}/* prefix. The DescribeSecret permission is required by the AWS provider to resolve the secret’s ARN from its name. The resource ARN uses data sources for region and account ID to avoid hardcoding values.
Step 3: Create the Pod Identity association
The Pod Identity association ties the IAM role to a specific Kubernetes service account and namespace. Any pod running with service account secrets-demo-sa in the default namespace receives temporary credentials for the associated IAM role, which grants DescribeSecret and GetSecretValue on secrets under the ${var.name}/* prefix.
Step 4: Install the Secrets Store CSI Driver and AWS Provider
In this step, I installed both components using the helm_release resource in the platform configuration. The Secrets Store CSI Driver (chart v1.6.0) runs as a DaemonSet and handles the Kubernetes CSI volume interface. It requires tokenRequests configured with the pods.eks.amazonaws.com audience so the kubelet generates an audience-bound service account token that the AWS provider needs for Pod Identity authentication. Without this, the mount fails with serviceAccount.tokens not provided error.
The AWS Secrets Store CSI Driver Provider (chart v3.1.1) also runs as a DaemonSet and handles the actual communication with AWS Secrets Manager. Since the CSI driver is installed separately, secrets-store-csi-driver.install is set to false to prevent the provider chart from installing a duplicate driver.
Step 5: Create the verification workflow
The verification code is in the /secrets-check folder. It consists of three resources that work together to prove the end-to-end integration.
SecretProviderClass tells the CSI driver which secret to fetch and how to authenticate. The objectName matches the secret name in AWS Secrets Manager (app-14/credentials), and usePodIdentity: "true" instructs the AWS provider to use Pod Identity instead of IRSA. Without usePodIdentity, the provider defaults to the IRSA authentication path and fails with An IAM role must be associated with service account.
ServiceAccount creates the Kubernetes service account (secrets-demo-sa) that matches the Pod Identity association from Step 3. The deployment references this service account, so the Pod Identity Agent injects the correct credentials.
Deployment creates a single nginx pod that mounts the CSI secret volume at /mnt/secrets. The secretProviderClass field in the volume spec references the SecretProviderClass by name (app-secrets). When the pod starts, the kubelet triggers the CSI driver, which calls the AWS provider, which authenticates via Pod Identity and retrieves the secret from Secrets Manager.
The volume section references the CSI driver (secrets-store.csi.k8s.io) and the SecretProviderClass by name. This is what connects the deployment to the secret retrieval configuration. When the kubelet sees this volume definition, it delegates the mount operation to the CSI driver, which looks up the named SecretProviderClass to determine which secret to fetch and how to authenticate.
One detail worth noting is that the AWS provider replaces slashes (/) with underscores (_) in mounted filenames. The secret named app-14/credentials in Secrets Manager appears as a file called app-14_credentials inside the pod at /mnt/secrets/app-14_credentials.
Deployment
This use case utilizes three GitHub Actions pipelines. The first pipeline (terraform.yml) deploys the resources in the infrastructure folder, which, for this use case, consists of the IAM role, the inline policy, and the Pod Identity association. The second pipeline (deploy-platform.yml) installs the Secrets Store CSI Driver and the AWS Provider as DaemonSets across all nodes. The third pipeline (secrets-check.yml) runs the end-to-end verification by creating an AWS Secrets Manager secret, applying the SecretProviderClass, deploying the demo application, reading the mounted secret from inside the pod, and finally cleaning up all resources after approval.
Verification
After the secrets-check workflow completed, the verify step confirmed that the secret was mounted successfully by reading the file from inside the pod.
The output shows the JSON credentials ({"username": "admin", "password": "..."}) retrieved directly from AWS Secrets Manager and mounted at /mnt/secrets/app-14_credentials without ever being stored in Kubernetes etcd.
Conclusion
In this article, I integrated AWS Secrets Manager with Amazon EKS using the Secrets Store CSI Driver, AWS Provider, and Pod Identity. The critical configuration details are tokenRequests with the pods.eks.amazonaws.com audience on the CSI driver, usePodIdentity: "true" in the SecretProviderClass, and an exact match between the Pod Identity association’s service account/namespace and the deployment’s serviceAccountName. With these in place, secrets flow from Secrets Manager directly into the pod’s filesystem at mount time, bypassing etcd entirely.
If you have any questions or suggestions, feel free to comment or get in touch.
