Kubecost Setup for Kubernetes Cost Visibility and Showback

Kubernetes Makes Cost Invisible — Kubecost Fixes That

Here's the problem with Kubernetes cost management: your cloud bill shows EC2 instances and EKS charges, but it doesn't show you that the search team's pods are consuming 60% of your cluster while the payments team's pods sit at 8% utilization. Without pod-level cost attribution, Kubernetes becomes a shared cost black hole.

I've deployed Kubecost at four companies now. The pattern is always the same:

Discovery	Typical Finding	Dollar Impact
Idle resources	35-45% of cluster CPU/memory is allocated but unused	$3,000-$15,000/mo wasted
Oversized requests	Pods requesting 4x what they actually use	$2,000-$8,000/mo in over-provisioning
Unattributed cost	20-30% of cluster cost has no team owner	Cannot optimize what you can't see
Abandoned workloads	Dev/test deployments running 24/7	$500-$3,000/mo per forgotten namespace

Let's get Kubecost running and start finding that money.

Installing Kubecost with Helm

Prerequisites

# Ensure Helm is installed and your kubeconfig points to the right cluster
kubectl config current-context
helm version

Basic Installation

# Add the Kubecost Helm repo
helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm repo update

# Install Kubecost (free tier — up to $50K/mo in monitored spend)
helm install kubecost kubecost/cost-analyzer \
  --namespace kubecost \
  --create-namespace \
  --set kubecostToken="YOUR_TOKEN_FROM_kubecost.com" \
  --set prometheus.server.persistentVolume.size=32Gi \
  --set kubecostMetrics.emitPodAnnotations=true \
  --set kubecostMetrics.emitNamespaceAnnotations=true

Production-Grade Installation with Existing Prometheus

If you already run Prometheus (and you should), point Kubecost at it instead of deploying a second instance:

helm install kubecost kubecost/cost-analyzer \
  --namespace kubecost \
  --create-namespace \
  --set kubecostToken="YOUR_TOKEN" \
  --set global.prometheus.enabled=false \
  --set global.prometheus.fqdn="http://prometheus-server.monitoring.svc.cluster.local:9090" \
  --set kubecostModel.etlBucketConfigSecret=kubecost-etl-bucket \
  --set persistentVolume.size=32Gi \
  --set kubecostMetrics.emitPodAnnotations=true

Terraform Deployment

resource "helm_release" "kubecost" {
  name             = "kubecost"
  repository       = "https://kubecost.github.io/cost-analyzer/"
  chart            = "cost-analyzer"
  namespace        = "kubecost"
  create_namespace = true
  version          = "2.2.1"

  set {
    name  = "kubecostToken"
    value = var.kubecost_token
  }

  set {
    name  = "prometheus.server.persistentVolume.size"
    value = "32Gi"
  }

  set {
    name  = "kubecostMetrics.emitPodAnnotations"
    value = "true"
  }

  set {
    name  = "kubecostModel.allocation.nodeDownsampling"
    value = "true"
  }
}

Configuring Cloud Cost Integration

Kubecost needs access to your cloud billing data for accurate pricing. Without it, it estimates based on on-demand rates and misses Reserved Instance or Savings Plan discounts.

AWS Integration

# kubecost-aws-config.yaml
apiVersion: v1
kind: Secret
metadata:
  name: cloud-integration
  namespace: kubecost
type: Opaque
stringData:
  cloud-integration.json: |
    {
      "aws": [
        {
          "athenaBucketName": "s3://your-kubecost-athena-results",
          "athenaRegion": "us-east-1",
          "athenaDatabase": "athenacurcfn_cost_and_usage_report",
          "athenaTable": "cost_and_usage_report",
          "athenaWorkgroup": "primary",
          "projectID": "123456789012",
          "serviceKeyName": "",
          "serviceKeySecret": ""
        }
      ]
    }

kubectl apply -f kubecost-aws-config.yaml

# Verify the integration
kubectl port-forward -n kubecost svc/kubecost-cost-analyzer 9090:9090
# Visit http://localhost:9090/model/cloud/config

IAM Policy for CUR Access

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "athena:StartQueryExecution",
        "athena:GetQueryExecution",
        "athena:GetQueryResults",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:athena:us-east-1:123456789012:workgroup/primary",
        "arn:aws:s3:::your-cur-bucket/*",
        "arn:aws:s3:::your-kubecost-athena-results/*"
      ]
    }
  ]
}

Setting Up Namespace-Level Showback

Showback is the foundation of Kubernetes cost accountability. Each team sees what their namespaces cost.

Label Your Namespaces

# namespace-payments.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: payments
  labels:
    team: payments
    cost-center: CC-2045
    environment: production
  annotations:
    kubecost.com/owner: "payments-team@company.com"
    kubecost.com/budget: "5000"  # Monthly budget in USD

Kubecost Allocation API — Cost by Namespace

# Get last 30 days of cost by namespace
curl -s "http://kubecost-cost-analyzer.kubecost:9090/model/allocation?window=30d&aggregate=namespace" \
  | jq '.data[0] | to_entries[] | {
    namespace: .key,
    totalCost: (.value.totalCost | . * 100 | round / 100),
    cpuCost: (.value.cpuCost | . * 100 | round / 100),
    ramCost: (.value.ramCost | . * 100 | round / 100),
    pvCost: (.value.pvCost | . * 100 | round / 100),
    efficiency: (.value.totalEfficiency | . * 10000 | round / 100)
  }'

Example Showback Report

This is what a monthly showback report looks like:

Namespace	CPU Cost	Memory Cost	PV Cost	Network Cost	Total	Efficiency
payments	$1,240	$890	$120	$45	$2,295	68%
search	$3,450	$2,100	$800	$210	$6,560	42%
platform	$890	$1,200	$340	$30	$2,460	55%
data-pipeline	$2,100	$3,400	$1,500	$180	$7,180	31%
staging	$1,800	$1,100	$200	$15	$3,115	18%
dev	$950	$620	$50	$10	$1,630	12%
Total	$10,430	$9,310	$3,010	$490	$23,240	38%

Two things jump out: search and data-pipeline are the biggest spenders, and staging/dev have terrible efficiency (18% and 12%). Those are your optimization targets.

Identifying Idle and Wasted Spend

Find Over-Provisioned Workloads

# Get workloads where requests are 3x+ actual usage
curl -s "http://kubecost-cost-analyzer.kubecost:9090/model/allocation?window=7d&aggregate=controller" \
  | jq '.data[0] | to_entries[] | select(
    .value.cpuCoreRequestAverage > (.value.cpuCoreUsageAverage * 3)
  ) | {
    workload: .key,
    cpuRequested: (.value.cpuCoreRequestAverage | . * 100 | round / 100),
    cpuUsed: (.value.cpuCoreUsageAverage | . * 100 | round / 100),
    wasteRatio: ((.value.cpuCoreRequestAverage / .value.cpuCoreUsageAverage) | . * 10 | round / 10),
    monthlyCost: (.value.totalCost | . * 100 | round / 100)
  }' | head -40

Idle Cost Breakdown

# Cluster-level idle cost — resources allocated to nodes but not requested by any pod
curl -s "http://kubecost-cost-analyzer.kubecost:9090/model/allocation?window=30d&aggregate=cluster&idle=true" \
  | jq '.data[0].__idle__ | {
    idleCpuCost: (.cpuCost | . * 100 | round / 100),
    idleRamCost: (.ramCost | . * 100 | round / 100),
    totalIdleCost: (.totalCost | . * 100 | round / 100)
  }'

Typical result:

Cost Category	Monthly Amount	% of Total
Active workload cost	$14,200	61%
Idle CPU cost	$4,800	21%
Idle memory cost	$3,400	15%
System overhead	$840	3%
Total cluster cost	$23,240	100%

That's $8,200/month in idle resources — 36% of the total bill. This is normal for clusters without right-sizing.

Setting Up Slack Alerts

Cost alerts that go to Slack actually get attention. Email alerts get ignored.

# kubecost-alerts-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: alert-configs
  namespace: kubecost
data:
  alerts.json: |
    {
      "alerts": [
        {
          "type": "budget",
          "threshold": 5000,
          "window": "30d",
          "aggregation": "namespace",
          "filter": "payments",
          "ownerContact": ["slack:payments-cost-alerts"]
        },
        {
          "type": "efficiency",
          "threshold": 0.3,
          "window": "7d",
          "aggregation": "namespace",
          "ownerContact": ["slack:finops-alerts"]
        },
        {
          "type": "spendChange",
          "relativeThreshold": 0.25,
          "window": "7d",
          "baselineWindow": "30d",
          "aggregation": "namespace",
          "ownerContact": ["slack:finops-alerts"]
        },
        {
          "type": "recurringUpdate",
          "window": "7d",
          "aggregation": "namespace",
          "ownerContact": ["slack:weekly-cost-report"],
          "filter": ""
        }
      ],
      "slackWebhookUrl": "https://hooks.slack.com/services/T00/B00/XXXX",
      "globalAlertEmails": ["finops@company.com"]
    }

kubectl apply -f kubecost-alerts-configmap.yaml
# Restart Kubecost to pick up new config
kubectl rollout restart deployment kubecost-cost-analyzer -n kubecost

Automated Right-Sizing Recommendations

Kubecost generates right-sizing recommendations based on actual usage. Export them and track savings:

# Get right-sizing recommendations for all workloads
curl -s "http://kubecost-cost-analyzer.kubecost:9090/model/savings/requestSizing?window=14d&targetCPUUtilization=0.7&targetRAMUtilization=0.8" \
  | jq '.[] | {
    controller: .controller,
    namespace: .namespace,
    currentMonthlyCost: (.currentMonthlyCost | . * 100 | round / 100),
    recommendedMonthlyCost: (.recommendedMonthlyCost | . * 100 | round / 100),
    monthlySavings: ((.currentMonthlyCost - .recommendedMonthlyCost) | . * 100 | round / 100),
    currentCpuRequest: .currentCpuRequest,
    recommendedCpuRequest: .recommendedCpuRequest,
    currentRamRequest: .currentRamBytes,
    recommendedRamRequest: .recommendedRamBytes
  }' | head -50

Sample Recommendations Output

Workload	Current Cost	Recommended Cost	Monthly Savings	Action
search/elasticsearch	$3,200	$1,920	$1,280	Reduce CPU 4 -> 2.4 cores
data-pipeline/spark-driver	$1,800	$900	$900	Reduce memory 16Gi -> 8Gi
platform/nginx-ingress	$450	$280	$170	Reduce CPU 2 -> 1.2 cores
staging/api-gateway	$620	$180	$440	Scale to 0 nights/weekends
dev/full-stack	$480	$120	$360	Scale to 0 nights/weekends

Total identified savings: $3,150/month or $37,800/year.

Scheduling Non-Production Scale-Down

The fastest win: stop paying for dev/staging clusters at night and on weekends.

# kube-downscaler for non-prod namespaces
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube-downscaler
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-downscaler
  template:
    spec:
      containers:
        - name: downscaler
          image: hjacobs/kube-downscaler:23.2.0
          args:
            - --include-resources=deployments,statefulsets
            - --default-uptime=Mon-Fri 08:00-20:00 America/New_York
            - --namespace=staging,dev
          env:
            - name: DOWNSCALE_PERIOD
              value: "0"

Savings math: Dev and staging namespaces cost $4,745/month. Running only during business hours (60 hours/week vs 168):

Active hours: 60/168 = 36% of the time
Savings: $4,745 * 0.64 = $3,037/month

The Showback Maturity Model

Level	Capability	Timeline	Impact
1 — Visibility	Kubecost deployed, basic cost by namespace	Week 1	Know where money goes
2 — Showback	Monthly reports to team leads, idle cost identified	Week 2-4	Teams see their costs
3 — Optimization	Right-sizing applied, non-prod scheduling	Month 2	20-35% cost reduction
4 — Budgets	Per-namespace budgets with alerts	Month 3	Proactive cost management
5 — Chargeback	Costs billed back to team budgets	Month 4+	Full accountability

Most teams see the biggest impact at Level 3 — optimization. Getting to chargeback takes organizational buy-in that's more political than technical. Start with showback and let the numbers create the motivation.

Deploy Kubecost today. Within a week, you'll know exactly where your Kubernetes money is going. Within a month, you'll have cut 20-30% of waste. It pays for itself before you even finish the setup.