WhatBrokeToday
GTA 6FAQ
Kubernetes

Kubernetes pvc-protection Causes Memory Spike via Full Pod LIST

medium

Is it fixed? Not yet.

Ongoing since Nov 19, 09:19 UTC. First reported Nov 19, 09:19 UTC.

What happened

When the pvc-protection controller fails to find a Pod referencing a terminating PVC in its informer index, it issues an unfiltered full Pod LIST for the namespace and retains the entire []v1.Pod copy. This results in significant memory spikes in large, high-churn namespaces.

What caused it?

The pvc-protection controller issues a full, unfiltered Pod LIST of a namespace and retains the entire []v1.Pod value slice when a Pod referencing a terminating PVC is missing from the informer index.

Source: github.com

No verified workaround yet

None of the linked reports describe a mitigation. This page updates when one appears.

Official fix

Timeline

  1. Investigating(current)
  2. Workaround available(upcoming)
  3. Resolved(upcoming)
  1. First report on GitHub issue

    pvc-protection: full Pod LIST per namespace in askAPIServer retains a []v1.Pod copy — memory spike in large, high-churn namespaces

    Source

Evidence

1 report · 1 GitHubShow
  1. [1]
    GitHub First report
    pvc-protection: full Pod LIST per namespace in askAPIServer retains a []v1.Pod copy — memory spike in large, high-churn namespaces(opens GitHub issue in a new tab)

    What happened? When the pvc-protection controller cannot find a Pod referencing a terminating PVC in its informer index, it issues a full, unfiltered Pod LIST of that PVC's namespace and keeps the whole result as a value slice of []v1.Pod: // pkg/controller/volume/pvcprotection/pvc_protection_controller.go type LazyLivePodList struct { cache []v1.Pod // value slice, not pointers controller *Controller } func (c *Controller) askAPIServer(..., lazyLivePodList *LazyLivePodList) (bool, error) { if…

    by Black-max12138 0upvotes 3comments

FAQ

What happened in the Kubernetes issue?

When the pvc-protection controller fails to find a Pod referencing a terminating PVC in its informer index, it issues an unfiltered full Pod LIST for the namespace and retains the entire []v1.Pod copy. This results in significant memory spikes in large, high-churn namespaces.

What is affected by the Kubernetes issue?

Kubernetes. Also affected at the same time: DigitalOcean.

When did the Kubernetes issue start and end?

It started Nov 19, 09:19 UTC and is still ongoing (1 d so far). This page updates as it develops.

How do I get alerts for Kubernetes?

Subscribe to the Kubernetes RSS feed for every new Kubernetes incident, or connect an AI assistant to WhatBrokeToday over MCP and ask whether Kubernetes is down. Kubernetes feed

Where does this information come from?

WhatBrokeToday collects public status pages, developer community reports and security advisories, groups the ones about the same problem, and links every report under Evidence on this page. This one has 1 report from 1 independent source.