Migrate k8s-infra-prow-builds to cgroups v2

[BenTheElder]

We'll probably want to coordinate with test-infra-oncall to do this for the existing default cluster as well.

https://cloud.google.com/kubernetes-engine/docs/how-to/node-system-config#cgroup-mode-options

cgroups v2 is the future and eventually when kubernetes etc stops supporting v1 then we will need this to run things like KIND and local-up-cluster.sh

[BenTheElder]

We should also check what the EKS cluster is doing.

[bobbypage]

/cc

[BenTheElder]

Quick hacky test:

kubectl run --rm -it --privileged --env DOCKER_IN_DOCKER_ENABLED=true --image=gcr.io/k8s-staging-test-infra/krte:v20230421-ec4335b54b-master test -- sh -c 'curl -Lo ./kind "https://kind.sigs.k8s.io/dl/v0.18.0/kind-$(uname)-amd64" && chmod +x ./kind && ./kind create cluster; ./kind delete cluster'
If you don't see a command prompt, try pressing enter.
wrapper.sh] [SETUP] Done setting up Docker in Docker.
fatal: not a git repository (or any of the parent directories): .git
================================================================================
wrapper.sh] [TEST] Running Test Command: `sh -c curl -Lo ./kind "https://kind.sigs.k8s.io/dl/v0.18.0/kind-$(uname)-amd64" && chmod +x ./kind && ./kind create cluster; ./kind delete cluster` ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    97  100    97    0     0    487      0 --:--:-- --:--:-- --:--:--   487
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 6808k  100 6808k    0     0  10.8M      0 --:--:-- --:--:-- --:--:-- 10.8M
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.26.3) 🖼 
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
Deleting cluster "kind" ...
Deleted nodes: ["kind-control-plane"]
wrapper.sh] [TEST] Test Command exit code: 0
wrapper.sh] [CLEANUP] Cleaning up after Docker in Docker ...
Stopping Docker: docker.
wrapper.sh] [CLEANUP] Done cleaning up after Docker in Docker.
================================================================================
wrapper.sh] Exiting 0
Session ended, resume using 'kubectl attach test -c test -i -t' command when the pod is running
pod "test" deleted

KIND at the latest release is working fine with our e2e docker-in-docker on cgroups v2 GKE host (v1.26 node pool in a new test cluster, double checked that v2 is detected).

[xmudrii]

/cc
for EKS cluster

[ameukam]

IIUC, adding a new nodepool with GKE 1.26+ should give us cgroupv2 by default. prow-build got upgraded to 1.26. Maybe add a new nodepool ?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[xmudrii]

Still relevant kubernetes/test-infra#31572
/remove-lifecycle stale

[ameukam]

/sig k8s-infra
/sig testing
/priority backlog
/area infra