AWS EKS: Production Kubernetes Cluster Setup from Scratch

EKS Architecture Overview

EKS (Elastic Kubernetes Service) manages the Kubernetes control plane — etcd, kube-apiserver, kube-controller-manager, kube-scheduler. You pay $0.10/hour per cluster for this. Worker nodes run in your VPC, on EC2 instances (managed node groups, self-managed, or Fargate) that you pay for normally.

The key components:

• Control plane: AWS-managed, multi-AZ, automatically patched
• Node groups: EC2 Auto Scaling Groups attached to the cluster
• VPC: Your network — subnets, security groups, NAT gateways
• IAM: Role-based access for both the cluster and your applications (IRSA)
• Add-ons: CoreDNS, kube-proxy, VPC CNI, EBS CSI driver — managed by EKS

---

Prerequisites

# Install required tools
# AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip && sudo ./aws/install

# eksctl
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin

# kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

# Verify
aws --version
eksctl version
kubectl version --client

Configure AWS credentials:

aws configure
# AWS Access Key ID: <your-key>
# AWS Secret Access Key: <your-secret>
# Default region: us-east-1
# Default output format: json

---

Option A: eksctl (Fastest Path)

eksctl is the official CLI for EKS — it handles VPC, subnets, IAM roles, and node groups in one command.

Quick cluster

eksctl create cluster \
  --name production \
  --region us-east-1 \
  --version 1.29 \
  --nodegroup-name standard-workers \
  --node-type m5.xlarge \
  --nodes 3 \
  --nodes-min 2 \
  --nodes-max 10 \
  --managed

This takes ~15 minutes and creates a fully functional cluster with a VPC and public/private subnets.

Production cluster config file

For repeatable, reviewable cluster configuration, use a YAML manifest:

# cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: production
  region: us-east-1
  version: "1.29"

vpc:
  cidr: "10.0.0.0/16"
  nat:
    gateway: HighlyAvailable  # One NAT GW per AZ

availabilityZones: ["us-east-1a", "us-east-1b", "us-east-1c"]

managedNodeGroups:
  - name: system-nodes
    instanceType: m5.large
    desiredCapacity: 3
    minSize: 2
    maxSize: 5
    volumeSize: 50
    volumeType: gp3
    labels:
      role: system
    taints:
      - key: CriticalAddonsOnly
        value: "true"
        effect: NoSchedule
    iam:
      withAddonPolicies:
        autoScaler: true
        cloudWatch: true
        ebs: true

  - name: app-nodes
    instanceType: m5.2xlarge
    desiredCapacity: 3
    minSize: 2
    maxSize: 20
    volumeSize: 100
    volumeType: gp3
    labels:
      role: application
    iam:
      withAddonPolicies:
        autoScaler: true
        cloudWatch: true
        ebs: true
        albIngress: true

addons:
  - name: vpc-cni
    version: latest
  - name: coredns
    version: latest
  - name: kube-proxy
    version: latest
  - name: aws-ebs-csi-driver
    version: latest
    wellKnownPolicies:
      ebsCSIController: true

cloudWatch:
  clusterLogging:
    enableTypes: ["audit", "authenticator", "controllerManager"]

eksctl create cluster -f cluster.yaml

---

Option B: Terraform

For infrastructure-as-code teams already using Terraform, the terraform-aws-modules/eks/aws module is well-maintained:

# main.tf
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  cluster_name    = "production"
  cluster_version = "1.29"

  vpc_id                         = module.vpc.vpc_id
  subnet_ids                     = module.vpc.private_subnets
  cluster_endpoint_public_access = true

  eks_managed_node_groups = {
    system = {
      instance_types = ["m5.large"]
      min_size       = 2
      max_size       = 5
      desired_size   = 3

      labels = { role = "system" }
      taints = [{
        key    = "CriticalAddonsOnly"
        value  = "true"
        effect = "NO_SCHEDULE"
      }]
    }

    application = {
      instance_types = ["m5.2xlarge"]
      min_size       = 2
      max_size       = 20
      desired_size   = 3

      labels = { role = "application" }

      block_device_mappings = {
        xvda = {
          device_name = "/dev/xvda"
          ebs = {
            volume_size = 100
            volume_type = "gp3"
            encrypted   = true
          }
        }
      }
    }
  }

  cluster_addons = {
    coredns                = { most_recent = true }
    kube-proxy             = { most_recent = true }
    vpc-cni                = { most_recent = true }
    aws-ebs-csi-driver     = { most_recent = true }
  }

  # Enable IRSA
  enable_irsa = true

  tags = {
    Environment = "production"
    ManagedBy   = "terraform"
  }
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "production-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_nat_gateway     = true
  one_nat_gateway_per_az = true
  enable_dns_hostnames   = true

  # Required tags for EKS subnet discovery
  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }
  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }
}

terraform init && terraform apply

---

Configure kubectl Access

# Update kubeconfig
aws eks update-kubeconfig --name production --region us-east-1

# Verify
kubectl get nodes
kubectl get pods -A

---

IAM Roles for Service Accounts (IRSA)

IRSA lets pods assume IAM roles without storing credentials — pods authenticate using their ServiceAccount JWT via OIDC.

# Check OIDC provider is configured
aws eks describe-cluster --name production --query "cluster.identity.oidc.issuer"

# Associate OIDC provider (if not done by eksctl)
eksctl utils associate-iam-oidc-provider --cluster production --approve

# Create an IAM role for a pod that needs S3 access
eksctl create iamserviceaccount \
  --name s3-reader \
  --namespace production \
  --cluster production \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \
  --approve

Use the ServiceAccount in your deployment:

spec:
  serviceAccountName: s3-reader

The pod gets temporary AWS credentials injected at /var/run/secrets/eks.amazonaws.com/serviceaccount/token, automatically rotated by EKS.

---

Essential Add-ons

Cluster Autoscaler

helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm install cluster-autoscaler autoscaler/cluster-autoscaler \
  --namespace kube-system \
  --set autoDiscovery.clusterName=production \
  --set awsRegion=us-east-1 \
  --set rbac.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=<autoscaler-iam-role-arn>

AWS Load Balancer Controller (replaces classic in-tree ALB ingress)

helm repo add eks https://aws.github.io/eks-charts
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  --namespace kube-system \
  --set clusterName=production \
  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=<alb-controller-role-arn>

Use it with Ingress resources:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80

---

Cost Optimization

Right-size nodes: Start with m5.xlarge and adjust based on CloudWatch Container Insights metrics. Avoid over-provisioning.

Use Spot Instances for non-critical workloads:

# In eksctl node group config
instancesDistribution:
  maxPrice: 0.08
  instanceTypes: ["m5.xlarge", "m5a.xlarge", "m4.xlarge"]
  onDemandBaseCapacity: 0
  onDemandPercentageAboveBaseCapacity: 0
  spotInstancePools: 3

Karpenter over Cluster Autoscaler: Karpenter provisions nodes faster and more efficiently — it creates nodes that exactly fit pending pods rather than scaling pre-defined node groups.

Enable compute savings plans: Commit to 1-3 years of compute usage for 30-60% discounts on On-Demand pricing.

Delete idle node groups: Use CloudWatch Container Insights to identify node groups with consistently < 20% utilization.

---

Upgrading EKS

EKS supports in-place upgrades. Always upgrade control plane first, then node groups:

# Upgrade control plane (takes ~10 minutes)
aws eks update-cluster-version \
  --name production \
  --kubernetes-version 1.30

# Watch progress
aws eks describe-update \
  --name production \
  --update-id <update-id>

# After control plane is upgraded, upgrade managed node groups
eksctl upgrade nodegroup \
  --cluster production \
  --name app-nodes \
  --kubernetes-version 1.30

# Update add-ons
aws eks update-addon --cluster-name production --addon-name coredns --resolve-conflicts OVERWRITE
aws eks update-addon --cluster-name production --addon-name kube-proxy --resolve-conflicts OVERWRITE
aws eks update-addon --cluster-name production --addon-name vpc-cni --resolve-conflicts OVERWRITE

Test in staging first. EKS supports n-1 skew (nodes can be one minor version behind the control plane) but upgrade promptly.