WhatBrokeToday
FAQ
Today's Trends
highInvestigatingKubernetes

Kubernetes Scheduler Cache Loses Bound Pod After Failed Bind Response

  1. Investigating(current)
  2. Trending(upcoming)
  3. Workaround available(upcoming)
  4. Resolved(upcoming)

Executive summary

A deployment with required podAntiAffinity can end up with multiple replicas scheduled on the same node because the scheduler cache loses a successfully-bound pod following a failed bind response. This violates required podAntiAffinity rules even when alternative feasible nodes exist.

No verified workaround yet

None of the linked reports describe a mitigation. Follow this incident to hear when one appears.

Technical root cause

Under active investigation. No report has identified a root cause yet.

Evidence

1 report · 1 GitHubShow
  1. [1]
    GitHub First report
    Scheduler cache loses a successfully-bound pod after a failed bind response, violating required podAntiAffinity(opens GitHub issue in a new tab)

    What happened? A Deployment with 2 replicas and a required podAntiAffinity (labelSelector: k8s-app=example, topologyKey: kubernetes.io/hostname) ended up with both replicas on the same node. The affinity configuration is correct, and another feasible node existed at that moment. Timeline (UTC, from kube-scheduler and kube-apiserver logs on the same control-plane node): | time | event | |---|---| | T+0 | etcd, kube-apiserver and kube-scheduler restart on all control-plane nodes (scheduler…

    by Black-max12138 0upvotes 2comments

Timeline

  1. First report on GitHub issue

    Scheduler cache loses a successfully-bound pod after a failed bind response, violating required podAntiAffinity

    Source