Kubernetes just made two moves that will change what platform teams test for in CI and what they expect from node images. First: as of v1.35 the kubelet's failCgroupV1 behavior is true by default — kubelets will refuse to start on cgroup v1 nodes unless an operator explicitly opts into a fallback. Second: AI Conformance has been promoted from a temporary working group into a permanent SIG Architecture subproject, and a new Workload Conformance Working Group has been created to codify runtime expectations for workloads (especially AI).
Flipping failCgroupV1 to true is the kind of blunt tool the project needs. For years the ecosystem has limped along with a mix of cgroup v1 and v2 behavior across distros, container runtimes, and kernels. Setting failCgroupV1=true removes ambiguity: a kubelet on a cgroup v1 system will fail to start unless you deliberately override the behavior. That forces operators to either move nodes to cgroup v2, patch their OS images, or explicitly opt into the older behavior.
Why this matters now
cgroup v2 fixes long-standing resource-accounting edge cases that matter for real workloads: memory and swap accounting, unified hierarchy behavior, and more predictable OOM handling for mixed CPU/memory/GPU processes. AI workloads — long-running, memory-heavy, device-bound — are precisely the cases where differences in accounting and eviction behavior surface as silent correctness bugs. Turning the knob to fail fast is a pragmatic way to accelerate the platform upgrade path.
What operators need to know
- Default change: v1.35 sets failCgroupV1=true. If your node OS is still using cgroup v1, kubelet startup will fail by default. The temporary workaround is to configure the kubelet to allow the cgroup v1 fallback, but that’s a band-aid.
- Real fix: move node images to kernels/distros that enable cgroup v2 by default, and ensure your container runtime (containerd or CRI-O) and underlying OCI runtime (runc) and tooling are on cgroup-v2-compatible releases. The ecosystem has been preparing compatibility work, but mixed fleets will still surface issues during upgrades.
- Upcoming removal: the community has signaled a path toward removing the cgroup v1 fallback entirely in a future minor release. Treat any temporary flag change as short-term.
This is overdue — and the right call. The alternative was a perpetual compatibility layer that lets subtle resource-accounting bugs slip into production, especially for GPU- and memory-hungry AI jobs. Platforms that haven't modernized their node images will face noisy kubelet failures during upgrades rather than gentle surprises weeks later.
AI conformance: governance meets runtime reality
Moving AI Conformance into SIG Architecture and standing up a Workload Conformance WG signals something more than governance theatre: the project is treating AI workloads as first-class citizens that need deterministic, testable behavior across distributions. The Certified Kubernetes AI Conformance Program is intended to give distributions and managed providers a repeatable stamp that an AI workload will see predictable scheduling, device-plugin semantics, extended resource behavior, and eviction semantics.
That's good. The bad: certification programs can become checkbox exercises that miss emergent interactions (GPU NUMA, MIG, hugepage fragmentation, ephemeral device plugins). The WG's success will hinge on executable, per-minor tests that surface those system-level interactions — not just an API/annotation matrix. If they do it right, platform teams will have a smaller, clearer test surface to run in CI and in their upgrade gates. If they do it wrong, expect a new layer of marketing-level compatibility claims that still leave you debugging weird OOMs at 3am.
How these threads connect
Both changes — the cgroup v2 enforcement and the AI conformance push — converge on the same truth: predictable node and runtime behavior matters more when workloads are stateful, device-intensive, and latency-sensitive. If you're building or operating AI platforms on Kubernetes, treat the new conformance tests as mandatory CI jobs and accelerate your node OS modernization plan.
If you want to track where this is headed, read the Workload Conformance WG announcement and the notes about future cgroup v1 fallback removal; they’re the roadmap. My prediction: within 18 months, vendors who haven't passed a compact AI conformance suite or who ship cgroup v1 by default will be an operational liability for any team doing production AI. That's not fear-mongering — it's just the reality of predictable infrastructure at scale.