Kubernetes WebSocket Exec Heartbeat Times Out After 61 Seconds Under Backpressure
Is it fixed? Not yet.
Ongoing since Nov 17, 15:51 UTC. First reported Nov 17, 15:51 UTC.
What happened
Kubernetes kubectl exec over WebSocket fails with a 61-second I/O timeout when stdin or stdout is backpressured and the remote process reads slower than the client sends data. The issue affects WebSocket connections (v5.channel.k8s.io with ExtendWebSocketsToKubelet enabled) on server v1.36.2 and kubectl v1.37.2, whereas SPDY completes the same transfer successfully.
What caused it?
Under active investigation. No report has identified a root cause yet.
No verified workaround yet
None of the linked reports describe a mitigation. This page updates when one appears.
Timeline
- Investigating(current)
- Workaround available(upcoming)
- Resolved(upcoming)
- First report on GitHub issue
WebSocket exec: heartbeat times out after 61 s when stdin or stdout is backpressured
Source
Evidence
1 report · 1 GitHubShow
- [1]GitHub First reportWebSocket exec: heartbeat times out after 61 s when stdin or stdout is backpressured(opens GitHub issue in a new tab)
What happened: kubectl exec -i over WebSocket (v5.channel.k8s.io, ExtendWebSocketsToKubelet on) fails with next reader: read tcp …: i/o timeout 61 seconds after the last pong when the remote process reads stdin slower than the client sends it. SPDY completes the same transfer. Reproduce (v1.36.2 server, containerd 2.3.2, kubectl v1.37.2): head -c 16777216 /dev/zero | kubectl exec -i POD -- sh -c 'sleep 75; wc -c' 4 MiB and 6 MiB pass; 8 MiB and more fail at about 61 s. A consumer that reads…
by theorician 0upvotes 4comments
FAQ
What happened in the Kubernetes issue?
Kubernetes kubectl exec over WebSocket fails with a 61-second I/O timeout when stdin or stdout is backpressured and the remote process reads slower than the client sends data. The issue affects WebSocket connections (v5.channel.k8s.io with ExtendWebSocketsToKubelet enabled) on server v1.36.2 and kubectl v1.37.2, whereas SPDY completes the same transfer successfully.
What is affected by the Kubernetes issue?
Kubernetes (versions: v1.36.2 server, v1.37.2 kubectl).
When did the Kubernetes issue start and end?
It started Nov 17, 15:51 UTC and is still ongoing (1 d 12 h so far). This page updates as it develops.
How do I get alerts for Kubernetes?
Subscribe to the Kubernetes RSS feed for every new Kubernetes incident, or connect an AI assistant to WhatBrokeToday over MCP and ask whether Kubernetes is down. Kubernetes feed
Where does this information come from?
WhatBrokeToday collects public status pages, developer community reports and security advisories, groups the ones about the same problem, and links every report under Evidence on this page. This one has 1 report from 1 independent source.