Kubernetes apiserver Watch Cache Silently Drops Events with Identical ResourceVersions
Is it fixed? Not yet.
Ongoing since Nov 15, 19:15 UTC. First reported Nov 15, 19:15 UTC.
What happened
The kube-apiserver watch cache silently drops events that share the same resourceVersion when an etcd transaction touches multiple keys at once. The watch cache dispatches only the first of those events to open watchers and discards the rest due to a bug in dispatchEvent.
What caused it?
In pkg/apiserver/watchcache (dispatchEvent), events from the same etcd transaction sharing a resourceVersion are incorrectly filtered, causing only the first event to be dispatched while the rest are discarded.
No verified workaround yet
None of the linked reports describe a mitigation. This page updates when one appears.
Official fixTimeline
- Investigating(current)
- Workaround available(upcoming)
- Resolved(upcoming)
- First report on GitHub issue
apiserver watch cache silently drops events that share a resourceVersion
Source
Evidence
1 report · 1 GitHubShow
- [1]GitHub First reportapiserver watch cache silently drops events that share a resourceVersion(opens GitHub issue in a new tab)
What happened? An open watch served from the kube-apiserver watch cache silently drops events that share the same resourceVersion. When an etcd transaction touches multiple keys at once, all those changes land in etcd under the same resourceVersion. The watch cache dispatches only the first of those events to open watchers; the rest are discarded. The bug is in dispatchEvent (pkg/apiserver/watchcache): when the dispatcher processes events from the same transaction, it checks: if…
by theo-mrn 0upvotes 2comments
FAQ
What happened in the Kubernetes issue?
The kube-apiserver watch cache silently drops events that share the same resourceVersion when an etcd transaction touches multiple keys at once. The watch cache dispatches only the first of those events to open watchers and discards the rest due to a bug in dispatchEvent.
What is affected by the Kubernetes issue?
Kubernetes.
When did the Kubernetes issue start and end?
It started Nov 15, 19:15 UTC and is still ongoing (3 d 9 h so far). This page updates as it develops.
How do I get alerts for Kubernetes?
Subscribe to the Kubernetes RSS feed for every new Kubernetes incident, or connect an AI assistant to WhatBrokeToday over MCP and ask whether Kubernetes is down. Kubernetes feed
Where does this information come from?
WhatBrokeToday collects public status pages, developer community reports and security advisories, groups the ones about the same problem, and links every report under Evidence on this page. This one has 1 report from 1 independent source.