WhatBrokeToday
FAQ
Today's Trends
highInvestigatingKubernetes

Kubernetes Kubelet Reports Zero Resource Request for Pods Using Pod-Level Resources

  1. Investigating(current)
  2. Trending(upcoming)
  3. Workaround available(upcoming)
  4. Resolved(upcoming)

Executive summary

The kubelet computes a resource request of zero for any resource not set at the pod level when pod-level resources are used. This causes eviction to incorrectly pick these pods ahead of higher priority pods.

No verified workaround yet

None of the linked reports describe a mitigation. Follow this incident to hear when one appears.

Official fix
Technical root cause

GetResourceRequestQuantity in pkg/api/v1/resource/helpers.go skips container-level resources whenever any pod-level resource is set, resulting in a zero resource request calculation.

Evidence

1 report · 1 GitHubShow
  1. [1]
    GitHub First report
    kubelet reports a resource request of zero for pods using pod-level resources, so eviction picks them ahead of higher priority pods(opens GitHub issue in a new tab)

    /kind bug /sig node What happened? The kubelet computes a resource request of zero for a pod that uses pod-level resources, for every resource that is not set at the pod level. GetResourceRequestQuantity in pkg/api/v1/resource/helpers.go asks PodRequests to skip container-level resources whenever any pod-level resource is set: hasPodLevelResources := utilfeature.DefaultFeatureGate.Enabled(features.PodLevelResources) && resourcehelper.IsPodLevelResourcesSet(pod) if rQuantity, ok :=…

    by SEPURI-SAI-KRISHNA 0upvotes 2comments

Timeline

  1. First report on GitHub issue

    kubelet reports a resource request of zero for pods using pod-level resources, so eviction picks them ahead of higher priority pods

    Source