Kubernetes Kubelet Stops Negotiating HTTP/2 When ReloadKubeletClientCAFile is Enabled
Is it fixed? Not yet.
Ongoing since Nov 11, 09:48 UTC. First reported Nov 11, 09:48 UTC.
What happened
Since Kubernetes 1.36, kubelets with --client-ca-file set fail to negotiate ALPN on the serving port (10250), forcing clients like the kube-apiserver to fall back to HTTP/1.1. This behavior occurs because the ReloadKubeletClientCAFile feature gate is enabled by default.
What caused it?
In NewMainKubelet (pkg/kubelet/kubelet.go), the implementation for ReloadKubeletClientCAFile overrides or misconfigures the TLS config handling such that ALPN is no longer negotiated.
No verified workaround yet
None of the linked reports describe a mitigation. This page updates when one appears.
Official fixTimeline
- Investigating(current)
- Workaround available(upcoming)
- Resolved(upcoming)
- First report on GitHub issue
kubelet stops negotiating HTTP/2 (no ALPN) when ReloadKubeletClientCAFile is enabled (default since 1.36)
Source
Evidence
1 report · 1 GitHubShow
- [1]GitHub First reportkubelet stops negotiating HTTP/2 (no ALPN) when ReloadKubeletClientCAFile is enabled (default since 1.36)(opens GitHub issue in a new tab)
What happened? Since 1.36, kubelets that have --client-ca-file set no longer negotiate ALPN on the serving port (10250). Every client, including the kube-apiserver, falls back to HTTP/1.1. The cause is in NewMainKubelet (pkg/kubelet/kubelet.go on master, L963-L972 on v1.36.4), which was added in #136762: if utilfeature.DefaultFeatureGate.Enabled(features.ReloadKubeletClientCAFile) { klet.clientCAManager = dynamiccertificates.NewDynamicServingCertificateController(kubeDeps.TLSConfig.Clone(),…
by kayiplar 1upvotes 2comments
FAQ
What happened in the Kubernetes issue?
Since Kubernetes 1.36, kubelets with --client-ca-file set fail to negotiate ALPN on the serving port (10250), forcing clients like the kube-apiserver to fall back to HTTP/1.1. This behavior occurs because the ReloadKubeletClientCAFile feature gate is enabled by default.
What is affected by the Kubernetes issue?
Kubernetes (versions: >= 1.36). Also affected at the same time: Bun.
When did the Kubernetes issue start and end?
It started Nov 11, 09:48 UTC and is still ongoing (7 d 18 h so far). This page updates as it develops.
How do I get alerts for Kubernetes?
Subscribe to the Kubernetes RSS feed for every new Kubernetes incident, or connect an AI assistant to WhatBrokeToday over MCP and ask whether Kubernetes is down. Kubernetes feed
Where does this information come from?
WhatBrokeToday collects public status pages, developer community reports and security advisories, groups the ones about the same problem, and links every report under Evidence on this page. This one has 1 report from 1 independent source.