Kubernetes Cost Optimization: A FinOps Decision Framework for Right-Sizing, Autoscaling, and Spot Capacity

Why Kubernetes Clusters Default to Overspend

Kubernetes cost problems are rarely about the wrong instance type. They are almost always a resource-request problem wearing an autoscaling costume.

Every pod spec carries two numbers that matter for cost: requests (what the scheduler reserves) and limits (what the kubelet will let the container use before throttling or OOM-killing it). The scheduler bin-packs nodes against requests, not against actual usage. If a service requests 2 vCPU and 4Gi of memory but typically uses 400m and 900Mi, the cluster autoscaler will still provision capacity for the requested 2 vCPU — on every replica, all the time. Cast AI's 2026 benchmark data puts average CPU overprovisioning at 69% across production clusters, meaning roughly two-thirds of provisioned CPU capacity in the average fleet is requested but never used.

This happens for defensible reasons. Engineers set generous requests to avoid noisy-neighbor throttling and OOM kills during traffic spikes, and nobody revisits those numbers after the incident that prompted them. The result compounds: a service that was over-provisioned by 3x in year one is still over-provisioned by 3x in year three, and now it has ten times as many replicas.

The consequence for architecture is that cost optimization is not a FinOps team problem to solve after the fact with a dashboard — it's a scheduling and autoscaling problem that has to be designed into the platform. The rest of this piece works through the stack in the order that actually pays off: requests, then node provisioning, then workload placement, then allocation and accountability.

Layer 1: Right-Sizing Requests and Limits

Right-sizing is unglamorous and it is where the majority of savings live, because it fixes the number every other layer builds on top of.

VPA in recommendation-only mode

The Vertical Pod Autoscaler (VPA) has three operating modes: Off (recommendation only), Initial (sets requests at pod creation, never touches running pods), and Auto (rewrites requests on running pods via eviction and restart, or in-place resize on clusters running Kubernetes 1.35+, where in-place pod resize reached stable and is enabled by default). Start every workload in Off mode. The VPA recommender still computes .status.recommendation with target, lower-bound, and upper-bound values — a free rightsizing audit with zero production risk, because nothing is actually enforced.

Review those recommendations against 2-4 weeks of real traffic (long enough to capture a weekly cycle) before promoting anything to Auto. Even after promotion, most teams keep VPA and HPA off of the same metric. If VPA is rewriting CPU/memory requests and HPA is scaling pod count on CPU/memory utilization at the same time, the two controllers can create a feedback loop where a resize changes utilization, which changes the HPA's replica target, which changes utilization again. The stable pairing is HPA on CPU/memory with VPA left in Off/recommendation mode, or HPA on a custom application metric (queue depth, requests-per-second) with VPA in Auto mode on CPU/memory.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-service-vpa
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: checkout-service
  updatePolicy:
    updateMode: "Off"   # recommendation-only; promote after 2-4 weeks of data
  resourcePolicy:
    containerPolicies:
      - containerName: '*'
        controlledValues: RequestsOnly   # never let VPA touch limits automatically
        minAllowed:
          cpu: 100m
          memory: 128Mi
        maxAllowed:
          cpu: 4
          memory: 8Gi

Setting controlledValues: RequestsOnly matters: without it, VPA's default (RequestsAndLimits) scales the limit proportionally with the request, which can silently raise or lower your OOM-kill ceiling as traffic shifts — a change you want to make deliberately, not as a side effect of an autoscaler.

The request-to-limit ratio

Keep the ratio between CPU request and CPU limit at 4:1 or tighter. A container requesting 100m with a 4000m limit makes percentage-based HPA thresholds nearly meaningless — the pod can burst to 40x its scheduled share before hitting a wall, which both defeats bin-packing (the scheduler reserved 100m but the node might need to supply 4000m under load) and makes "70% CPU" trigger at wildly different absolute usage levels across services.

KRR and static analysis as a stopgap

For clusters without VPA data yet, tools like krr (Kubernetes Resource Recommender, from Robusta) read Prometheus history and produce request/limit recommendations without deploying anything into the cluster — a reasonable first pass before committing to a VPA rollout.

Layer 2: Node Provisioning — Karpenter vs. Cluster Autoscaler

Once requests reflect real usage, the second lever is how the cluster turns pending pods into nodes.

Cluster Autoscaler works against fixed node groups: it scales a predefined instance-type pool up or down and only rarely reshuffles running pods once they're placed. Karpenter (originally AWS-native, now with provider support for Azure and GCP) provisions directly against the cloud API, choosing from the full range of available instance types and sizes for each batch of pending pods, and continuously asks "can I do better?" through its consolidation controller.

Consolidation alone — with no spot instances involved — has produced real savings. One widely cited case put an 18% cost reduction on a 1,000-node cluster purely from enabling aggressive consolidation, because Karpenter repacks pods onto fewer, better-utilized nodes as workloads scale down, rather than waiting for a node to go fully idle before removing it (which is closer to how Cluster Autoscaler behaves).

Layering spot capacity on top

Spot/preemptible instances are the largest single discount available (60-90% off on-demand pricing depending on instance type and region), but only pay off for workloads that tolerate interruption: stateless services behind a load balancer, batch and CI runners, and background job workers. The Cast AI 2025 Kubernetes benchmark found that a partial-spot mix saved an average of 59% on compute versus all-on-demand, and fully spot-committed fleets saved 77% — but that number only holds if interruption handling is actually correct, not aspirational.

A workable spot policy for a mixed fleet:

• Stateful services (databases, brokers, anything with local disk state that isn't trivially reschedulable) stay on-demand or reserved.
• Stateless, horizontally-scaled services run on spot with a PodDisruptionBudget that keeps a minimum on-demand floor (e.g., 25% of replicas) so a correlated spot reclaim event in one AZ doesn't take the service below serving capacity.
• Karpenter's NodePool spec expresses this directly via weighted capacity-type requirements and a consolidateAfter window, so the scheduler prefers spot but has an on-demand fallback path already defined rather than improvised during an incident.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
      nodeClassRef:
        name: default
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 30s
  limits:
    cpu: 1000

The failure mode worth designing against explicitly is correlated spot reclamation: cloud providers reclaim spot capacity by instance type and AZ in clusters, not independently per node, so a fleet concentrated on one popular instance family in one AZ can lose a meaningful percentage of capacity within the same two-minute interruption-notice window. Diversifying the instance-type requirements in the NodePool (rather than pinning to one "efficient" SKU) is the mitigation, and it's one of the reasons Karpenter's wide instance search is a real advantage over a hand-maintained Cluster Autoscaler node group list.

Layer 3: Storage, Idle Resources, and the Overhead Nobody Graphs

Compute gets the attention because it's the line item everyone graphs, but two categories consistently hide waste:

• Idle namespaces and orphaned PVCs. Persistent volumes outlive the workloads that created them more often than teams expect — a deleted StatefulSet doesn't always take its PVCs with it, and the storage cost keeps accruing silently. A periodic sweep (weekly, automated, reported rather than auto-deleted) catches this before it becomes a line item nobody can explain.
• Non-prod environments running prod-shaped requests. Staging and dev clusters frequently inherit the same resource requests as production because someone copy-pasted the manifest. Scheduled scale-to-zero for non-business-hours (via a CronJob that scales Deployments to 0 replicas, or cluster-level node group scheduling) is one of the highest ROI, lowest-risk changes available, because non-prod traffic genuinely has predictable idle windows that production doesn't.

Layer 4: Cost Allocation — Making the Savings Durable

None of the above sticks without accountability, and accountability requires allocation that engineering teams trust. The FinOps Foundation's Container Costs Working Group recommends allocating shared cluster costs (control plane, system daemonsets, unallocated capacity) proportionally by each team's share of total resource requests, and allocating workload-level cost using a blended formula — roughly 70% weighted on resource requests and 30% on actual usage — which the working group's data shows lands in the 85-90% accuracy range against ground-truth billing.

The sequencing matters as much as the formula. Move through showback before chargeback:

1. Showback (first 60-90 days). Send namespace-owning teams a monthly report: total namespace cost, top five most expensive workloads, idle/waste percentage, and month-over-month trend. No budget consequence yet — the point is building the habit of looking at the number and catching allocation bugs (mislabeled namespaces, missing team tags) before they become a billing dispute.
2. Validate. Reconcile the tool's namespace-level numbers against the cloud bill at the account level for at least one full billing cycle. Discrepancies here are almost always missing labels or shared-cost double-counting, not the allocation methodology being wrong.
3. Chargeback. Once teams trust the numbers, tie them to budget ownership. This is also where cost allocation stops being a side project and becomes something finance can plan against — the same discipline this piece describes for compute applies more broadly across a FinOps program, where cost visibility has to exist before the workload ever lands in production.

OpenCost (CNCF sandbox project) and commercial tools (Kubecost, Cast AI, Vantage) both implement variants of this allocation model against Prometheus metrics and cloud billing APIs; the choice between them is less important than actually running the showback period before enforcing budgets.

A Decision Checklist

Before reaching for a new tool, work through this in order:

1. Do requests reflect real usage? If VPA recommendation data hasn't been reviewed in the last quarter, start there — every other optimization is scaled by this number being roughly correct.
2. Is the request-to-limit ratio tight enough that HPA thresholds mean something? Audit for >4:1 ratios.
3. Is the node provisioner bin-packing continuously, or only reacting to scale-up events? If Cluster Autoscaler hasn't been re-evaluated against Karpenter in the last 12-18 months, the gap has likely widened as Karpenter's provider support and consolidation logic matured.
4. Which workloads can tolerate interruption, and do they have a PodDisruptionBudget and an on-demand floor defined before they're moved to spot capacity — not after the first incident?
5. Is there a namespace-level cost report that a team lead actually reads? If allocation exists but nobody is accountable to it, showback hasn't started yet in any way that matters.
6. Are non-prod environments scaled to zero outside business hours, and are orphaned PVCs swept on a schedule?

Skipping to step 4 (spot) or a reserved-instance purchase without steps 1-3 is the most common expensive mistake: it locks in savings against an inflated baseline, and the baseline stays inflated because nobody had to look at it to get the discount.

Sources

• Cast AI, 2025/2026 Kubernetes Cost Benchmark Report — CPU overprovisioning and spot savings figures
• FinOps Foundation, Container Costs Working Group — cost allocation methodology
• Kubernetes documentation — Vertical Pod Autoscaler, in-place Pod resize (stable in 1.35)
• AWS Compute Blog — Karpenter spot-to-spot consolidation

Syslabs' engineering team works on Kubernetes cost and platform architecture problems like this one for clients running production clusters at scale.