WhatBrokeToday
FAQ
Today's Trends
highInvestigatingKubernetes

Kubernetes Kubelet Panics on PodCertificateRequest Tombstone Deletion

  1. Investigating(current)
  2. Trending(upcoming)
  3. Workaround available(upcoming)
  4. Resolved(upcoming)

Executive summary

The kubelet crashes and panics with an interface conversion error when a PodCertificateRequest deletion arrives as a tombstone after a relist. Users running affected versions experience kubelet instability.

No verified workaround yet

None of the linked reports describe a mitigation. Follow this incident to hear when one appears.

Official fix
Technical root cause

An unhandled type assertion in the PodCertificateRequest issuing manager fails when processing cache.DeletedFinalStateUnknown instead of a direct *v1.PodCertificateRequest object.

Evidence

1 report · 1 GitHubShow
  1. [1]
    GitHub First report
    kubelet panics when a PodCertificateRequest deletion arrives as a tombstone after a relist(opens GitHub issue in a new tab)

    What happened? The kubelet crashes with: panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.PodCertificateRequest [recovered, repanicked] goroutine 67 [running]: ... k8s.io/kubernetes/pkg/kubelet/podcertificate.NewIssuingManager.func3(...) pkg/kubelet/podcertificate/podcertificatemanager.go:303 k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnDelete(...) staging/src/k8s.io/client-go/tools/cache/controller.go:437…

    by SEPURI-SAI-KRISHNA 0upvotes 7comments

Timeline

  1. First report on GitHub issue

    kubelet panics when a PodCertificateRequest deletion arrives as a tombstone after a relist

    Source