Performance Tuning¶

🎯 Learning Objectives

Identify performance bottlenecks
Optimize cluster and node performance
Learn advanced tuning techniques
Troubleshoot performance issues
Implement performance monitoring

Performance optimization is critical for production clusters. Understanding bottlenecks and tuning techniques ensures optimal cluster performance.

Performance Monitoring

Always measure before optimizing. Use metrics to identify bottlenecks, not assumptions.

Over-Optimization

Premature optimization can add complexity. Optimize based on actual bottlenecks.

Performance Metrics¶

Key Metrics to Monitor¶

Cluster Level: - API server latency - etcd performance - Scheduler latency - Controller reconciliation time

Node Level: - CPU utilization - Memory usage - Disk I/O - Network throughput

Application Level: - Pod startup time - Request latency - Throughput - Error rates

Baseline Metrics

Establish baseline metrics before optimization. Compare before/after measurements.

API Server Optimization¶

Tuning Flags¶

# API server performance flags
--max-requests-inflight=400
--max-mutating-requests-inflight=200
--request-timeout=60s
--min-request-timeout=1800

Request Limits

Adjust request limits based on cluster size and workload. Monitor API server metrics.

etcd Optimization¶

# etcd performance flags
--quota-backend-bytes=8589934592
--max-request-bytes=1572864
--heartbeat-interval=100
--election-timeout=1000

etcd Performance

etcd performance directly impacts API server. Use SSD storage and optimize network.

Node Optimization¶

Kubelet Tuning¶

# kubelet configuration
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 110
podPidsLimit: 4096
serializeImagePulls: true

Node Capacity

Tune maxPods based on node resources. Higher values enable more pods but increase overhead.

Container Runtime¶

# containerd configuration
[plugins."io.containerd.grpc.v1.cri"]
  max_concurrent_downloads = 3
  snapshotter = "overlayfs"

Runtime Tuning

Optimize container runtime for faster image pulls and container startup.

Application Optimization¶

Resource Right-Sizing¶

# Use Vertical Pod Autoscaler recommendations
kubectl get vpa <vpa-name> -o yaml

# Analyze resource usage
kubectl top pods --all-namespaces

Right-Sizing

Right-sized resources improve utilization and reduce costs.

Image Optimization¶

# Use multi-stage builds
FROM builder AS build
# ... build steps ...

FROM alpine:latest
COPY --from=build /app /app
# Smaller images = faster pulls

Image Size

Smaller images reduce pull time and improve pod startup performance.

Troubleshooting¶

Performance Issues¶

High API Server Latency¶

# Check API server metrics
kubectl get --raw /metrics | grep apiserver_request_latencies

# Check etcd performance
ETCDCTL_API=3 etcdctl endpoint status

# Review API server logs
kubectl logs -n kube-system kube-apiserver-<node>

Slow Pod Startup¶

# Check image pull time
kubectl get events --field-selector involvedObject.name=<pod-name>

# Check node resources
kubectl describe node <node-name>

# Review kubelet logs
journalctl -u kubelet -f

Best Practices¶

Production Recommendations

Monitor performance metrics continuously
Establish performance baselines
Optimize based on actual bottlenecks
Test optimizations in non-production
Document performance tuning changes
Regular performance reviews

Next Chapter: HPA & VPA Deep Dive