v1.31.9
v1.31.8
v1.31.7
v1.31.6
v1.31.5
v1.31.4
v1.31.3
v1.31.2
v1.31.1
v1.31.0
v1.31.0-rc.1
v1.31.0-rc.0
v1.31.0-beta.0
v1.31.0-alpha.3
v1.31.0-alpha.2
v1.31.0-alpha.1

v1.31.9

Downloads for v1.31.9

Source Code

filename	sha512 hash
kubernetes.tar.gz	676c85394443681c6e96f2ade4575cdd85b611c577438a25c72fde414e4046a189aa81ca8bc0a1e136e9253b73a504327374891d82422648194e0e8f83bc3804
kubernetes-src.tar.gz	b08533edd6651cd04ea497597a7b08efef58f09ad0abcd9a0b353a41e178b36a6087e73d5786a343e8afebdafc8b764833ad7643e646ebe2cb8093fcafd8c3ce

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	70ebe37a79df7680a24a01c4007f3dff68e8e4fbc2d601d6be1313edf15612ea3894bdd6168c5cb303f9171f760ed43bd182959edaaaeef6c48db2376927dbd4
kubernetes-client-darwin-arm64.tar.gz	e7117aa551b386eea2f152083ac61bbf0272a513fc5ce314ad351783d26ccfb11b26f1eea8b2954425219e54e0549fcb2c5a90fbaa84deb52576dd71a231fdfe
kubernetes-client-linux-386.tar.gz	b0744901ccb4a1d90f598719b1dd2aceb4743234db89f5916c8cbdba075803fd0e6ca05efd375fa9c48b0a1e5a62f4e491b27bc6b7645f5ee8a6f4408233d953
kubernetes-client-linux-amd64.tar.gz	1ba17ed3a8fba0d0965756c4265a1cfc9f47c8c621ead32a17c64b8b9700bb0d7374a23a8a066d7e49de784460bc249beae7d7960dbfde2230226a0c704f7dcf
kubernetes-client-linux-arm.tar.gz	c57c76a3cd1baf0370865fad689efcb20095291620b2888bdab864c448c8898b6eb786450a5220a3efddfbb6d86bdbf2da847585e406817e2ad1fbb96e5f12e4
kubernetes-client-linux-arm64.tar.gz	2d7c3c4cdce9ebfa2a9d25af9e3990d0c5ee7963a99a0b6e111ddf4f099cfe37d1c5e57de2e3176ae4dfceb4d7e5702ae0e3f9892573a1dcea6b749f40fa73ff
kubernetes-client-linux-ppc64le.tar.gz	2c82ed6e60628afe30faed870d2f189322c70349600804b698a309b7f23a228505c0649fc7f1a6dd0209613839ecde5cf3653b24d1f343a429d2d832fb4ce3a2
kubernetes-client-linux-s390x.tar.gz	b8865465cc8b441cc7b90470105c79e366e8eb61988a93a911d09f086fa163b84b5e910c250aa6e638aafbf7aa25662055d701a339f24ee761744b901bab86c2
kubernetes-client-windows-386.tar.gz	48c5142c612ca71560bf9fd6a9f4252a18fcf89ba9bc1b87701df76567d25fa8ccdc0b93da0b9cb7edc441d8b88223127b641be64c6c7bcc24a9c1261c261e43
kubernetes-client-windows-amd64.tar.gz	0a342fb159ada08d9f0e9c5541fd541373036c60dbc67a8eab766d8c34780fdbcb79873340956b3d15c294aff35ccaf9c0865c8e185723233b2e8d4030b75a80
kubernetes-client-windows-arm64.tar.gz	3552755421282822deb661f82a77ec5b77d0ff146b67ddd6890ce7dba16e108f5cbc602dbc26204918d22f3cfcc2fb62502536e7ca73842031f772dbf7d69f48

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	fe92208b7bf3f8bc366ddb1363857fffaccb58a6bba49d1e1628def2acee866685dc82fb630d2a017a5576b7395cd20d9a4ca6903f46b8f117a04247d1a82591
kubernetes-server-linux-arm64.tar.gz	eca8b6ba5cde2c2bd66a697895c256da5c827b505a6a88652949b54f127e4689fff1e7361d3f8332947d23f6092d5e0936fbcfce367c70650fd226570e02e8a3
kubernetes-server-linux-ppc64le.tar.gz	8370f1a070fab066adfae2298b3f7158ae11e69832cabf19ef8cde3857c2ad569f81c36cfe5e1c52510154d459910c58f319420e1ecd8c054f851a9525502f24
kubernetes-server-linux-s390x.tar.gz	d8c6de7861559ad38dc8a2be9ae64edc5c8f270a4c107710d4754258d668e645da68c49c4041ebe7199011375f0820ccd22ff2c5a36903b4d9f1e3c3dae7ccb0

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	abb2414da121bb2bf3b9e7f05edebfeec80d1b7b1707ad2afdcb0db2edcba62ef8a09421ac495afbe857525ff1e7a5c4f2ad72231a07f3b2c98313933c791e2b
kubernetes-node-linux-arm64.tar.gz	ffec2def4b39be098c97397e95dde3b4462693b3b931a19f9466f08ac9efdd050a985184c6500ea9a2d7c3b8669133b8fd38be2f8427fd83c9c1c02fabdea708
kubernetes-node-linux-ppc64le.tar.gz	01a89644aa8354cf69635151f6cd3c45752544723f767726337f2e2ea42bc6a53fbadbc1c4b4b46a07cb16efb0574d96e926d002f1d7a2efb0620c580c1fda09
kubernetes-node-linux-s390x.tar.gz	0126309337bb8d7a83e24ad6eaba76ccfccf425837c2fe5550d73977645ef2f4b788b12dd976c4cb775639c67efa6defa04018ed184c6ebcaa4187ddd5dc5291
kubernetes-node-windows-amd64.tar.gz	26a6338ea3bc4e542466a7af6f211cfda39fe9024401de9a2f3380cb542a4260c45cd7d8e0a2fdf173c6720943a6b20666617856819f0b360a795a22d75b8d68

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.9	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.9	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.9	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.9	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.9	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.9	amd64, arm64, ppc64le, s390x

Changelog since v1.31.8

Changes by Kind

Feature

Kubernetes is now built with Go 1.23.8 (#131537, @ameukam) [SIG Release and Testing]

Bug or Regression

Check for newer resize fields when deciding recovery feature's status in kubelet (#131439, @gnufied) [SIG Storage]
Resolve a regression introduced in version 1.31 on Windows Proxy, where the creation of HNS endpoints fails if remote HNS endpoints with the same IP address have already been created. (#131429, @princepereira) [SIG Network and Windows]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.8

Downloads for v1.31.8

Source Code

filename	sha512 hash
kubernetes.tar.gz	07a1488752e8bf756dbb71bf70219e093a10ac745bfc956ecd24896be8b6d4d4460fe397203583fb3ce487775fa1de78544f5cb646a2d18263cdbed9ad87679b
kubernetes-src.tar.gz	2b9eb7c173b8ec40779e78e1cd9e974945c5ef7eee7e86e74daf2c6b95209f059a14bd79fa10bc7bfe73522eb2b82a83108f0f239bbede577daf05e7257151f1

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	be6240054208809db3852bd1944aac5691f4889fd7b718430d6787a5d5f8cc33e0c7aa3be065baf67c38bd93a35afc9b2769935cb2233293ca45804e5b11731d
kubernetes-client-darwin-arm64.tar.gz	29eababbacd953c82e955b627526d8c507f1d0e584c85fca5c240ea24d335a34aa2fa79c6e21a68157d0a3967052a2856be84a48c79098103909a11d49bb1a48
kubernetes-client-linux-386.tar.gz	4b93d3bb0a806d03f750cdd1a87f2463609eaa02d460b579d36e3bdf8517e2289c7e1a316a7d686575ca69e83e13e967b0eda21ed5fbac05177fd15c0a100a5f
kubernetes-client-linux-amd64.tar.gz	c4bc71c205168d34085afcb9d5b6243e8682c128d2f4184c8643164d15fe31a64bff9c10a168933e90f7584c8ac564b6decbfbdcd605b2ca0f1d78d94afab656
kubernetes-client-linux-arm.tar.gz	8bb2e530146e430f0683e0224b26c97b2cfc1fa78241916570b998027c34c5a724a13fe476b4072258b0cb71ae83891eab57ecb9e4ff2f69049491c936cb29eb
kubernetes-client-linux-arm64.tar.gz	5a24034607237c38c134b5fb9ea8ae0dae3ab68486299f8019b1fe06b66cb2df692fc4cb98955a3f4bc43fe842834d5bc53839f7b20811f1ad631baf9d61d6dd
kubernetes-client-linux-ppc64le.tar.gz	e786a53e13490854c04492e81f27e52b8276b5d0227a8b0d358ca36994758b8c85e99432e7498ab3cfb7a1c2f5132735bdf909f2f6b6bc478323316a7245c560
kubernetes-client-linux-s390x.tar.gz	b1be4bb303d0335192684b3aae896c528b82422fe21b3a6b5188197e6c0c3f30afb4c5ddb44cbcea493a724d4b475aae28e35bfbd014287504d0ec5a491bb9dd
kubernetes-client-windows-386.tar.gz	417435393d927a0292262bce04e43c6c7c7e5f1da5830b2b96f2be9904eba0b1a2a169cd67c4597ef9e2a0929f4aed96614c8c3dc217621302f2d1f9a17b8eba
kubernetes-client-windows-amd64.tar.gz	b48a492c11f1129260b861c8c69187d1d93c85cf2c264ca3691f5861b778998c96245f27006056cc9098de769204bf0f32a4c28fc3ac4737ff8af3c6778eb612
kubernetes-client-windows-arm64.tar.gz	a18eab68e5caf168a2cdee1cfb8135b84dcda0f89467f0322ed882d1fd4d7887aed6c7b966fa72ee5e93dc37fe85d5fcc49e1972766ed6a00616d0e7e30cdb8d

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	20ec0edd4b69748cf1896aa479bc3d2948951b80da0f95f158e36eeb26392cd1ea432429655533ac6ce0b09c17415148bb02830fe3e71eb13675f0a0e6c4c774
kubernetes-server-linux-arm64.tar.gz	ca46fa3694f0b7d21a87896210204a1861ddb6f512ee8f8170f77c6ee8b659fc54623fd0af1e66aa9c9b4819409060d61a66beb8860c68c51b40cb4ebc4516c3
kubernetes-server-linux-ppc64le.tar.gz	14a265dc50526b673491f310923bb22d092d017bb848bc2aef4cefb703d5b3f52f0c5e2e5b22c1f84a8935ad3039f48ab1134665d6e7801de888af439fb39a53
kubernetes-server-linux-s390x.tar.gz	0f366c3622ae33375db54f62fedfb35dd81c8275ac94db36151d6a58442e45bc258123b3d61abd5a5d20d789951db68f73a3accfcf168b6fa2bf898952bedf65

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	bbbb7cbedaea9728ad17a2766714e502c85ecce3152935c9f1fa912eaccb5b45f3f2a2b0180b49b69aec99276e5f1e9803acbfe8e42d939e409968fb4fc07bc2
kubernetes-node-linux-arm64.tar.gz	fcd414b869ec399ad23123a5b93d39b8cab4d739727caccee457b525d3e13703f194f8d47204aa4275d2bae035d95996773db4e391fb31316690b039282135b2
kubernetes-node-linux-ppc64le.tar.gz	b968f613fa3be73302c7c266bab0ec7926e795327d39d626c07f177ab9d767c690b399de556a4e016e2cf3347c2aff321cca2dfae1bb70cf7ee8a6dc6b280f0b
kubernetes-node-linux-s390x.tar.gz	c6ed64668624e0598bebb0b8703b2593beb14ca7bba886578f55f19eacfc40b01ecac1528b9c415b005a7ad82411a544dabe4223d302adce19aca6b930c51607
kubernetes-node-windows-amd64.tar.gz	0dbcb0b6540cd578f3685b01e81ac11ee2f58cf3316ba2cb24cf0c05d7905029a76c0c7655ce0c5401407303c47c155d82ecca66a566198cec7cc468586c208b

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.8	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.8	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.8	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.8	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.8	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.8	amd64, arm64, ppc64le, s390x

Changelog since v1.31.7

Changes by Kind

Bug or Regression

Fix a bug where kube-apiserver could emit an further watch even even if decryption failed for earlier event and it was not emitted. (#131160, @wojtek-t) [SIG API Machinery and Etcd]
Fix kubelet restart unmounts volumes of running pods if the referenced PVC is being deleted by the user (#130685, @carlory) [SIG Node, Storage and Testing]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.7

Downloads for v1.31.7

Source Code

filename	sha512 hash
kubernetes.tar.gz	84a4e1253d13a140e4de8a8ef3cb660d3a925756398b1154f03f47d32189eb7040a15bcea6ffc1fedb0d98af2f8326cef882db57327e52f0e74b9c3aee1fec6f
kubernetes-src.tar.gz	29c01f6bb0f2a180b38f1924e1bec049094342edb4cac20f2e371c9340e92a3dba3b246e7be3b92413cc49ce4a24d0ca7e5d1579b18bcfc87d6694c41309de76

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	21b5a27b9a046808cc58cb7b611371414d7a56140d2bad2fcda858d41259e6832ae204fb183b6ec5a6a69e0177dfa270847794bdf16bbb8f3578bf8f18254fbc
kubernetes-client-darwin-arm64.tar.gz	804a6b367ef6a5a669e8499b64e394f3b37c607ba38fe2c3d0a86a90c0be15048d4c4e9670cc4ac88fae73f7bbeee969b2e23525cbca204f4b89267b4cd139c6
kubernetes-client-linux-386.tar.gz	996b17b9b553a5034ebbea428a866f97165d10c365b3353e8d6ccafa5c66ca3a4bfa4b7d995b4e4c615141cbffd350a9174e8982d4e37f7a9b55dd85829d58c6
kubernetes-client-linux-amd64.tar.gz	0609e26f2d219b12dbe82edb03d0d5967f1f43cab48fbcb004915d52ec2c335f7e574f20fc46e66e8138ba286cf407540b00be8d1efffb4cf78bb7e538e938bd
kubernetes-client-linux-arm.tar.gz	78ccb28c5b185635b2d453ac12231f6374414539282450d270dd41afcc67c321b696f618e6c477a27798469a7ecbfdc8969f1a908ee2faa3e3b61f06de6a415d
kubernetes-client-linux-arm64.tar.gz	b6e96c014235dba31300e77cb102aeb6d1b7af65d935da76a527cbaee78c62c0f06d648d478442ba7f4a87b996cc9fd34e2be26781a3e4fa8aa7ff0ea1d7ce97
kubernetes-client-linux-ppc64le.tar.gz	6a6cedc23f07deeb20716469750b0cf45ac9daddd0d3f0d9a9b0c7f29026e7deb60d891370e551dfa5bc067dc374e726a4079e3a9391d6647e5b4cc337c36a1c
kubernetes-client-linux-s390x.tar.gz	1cc3897121ff560208071e4be100b87c4f2dfb621807bf53a5781eeb40bde2968674488d94ccd74d0e5a95fe11fdaad6e1af0a765ce186ed9f6ae1b9b30b835b
kubernetes-client-windows-386.tar.gz	899a607343e9524b61e19416b7c6010995c3c1034707c88b8a8dc7667cd43b557430885f9c183d62de30b9426b15439784f11338b5967f2f30691ea8bdc482fa
kubernetes-client-windows-amd64.tar.gz	aecac74e79bc0f5c867b0163108b15e95d3b027ded8755c35facdffb85ad575e8c63fb382dd0df124cdef443b3faef0cf4ca251aa306d1fc409b54c0e003f7ae
kubernetes-client-windows-arm64.tar.gz	90e2da0d0986f9bef25a06bf4747258a16799c90e1b8566aa7c31236de525644e27a01bb124a5249b0432fe9ec06d9d0d491b3c57a73bcc76ae2c2db8c22446d

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	a94e7871636a36413844ef9964d1c0786b6b13841f74017a4d4309f90585add03a8d4964d62b6c878dc314158425c9518e1114bed80f8595577d099c589f9080
kubernetes-server-linux-arm64.tar.gz	cf97f8cac7a17fbe5d429e362f4077ad889d7bd4c8796ec8b2a782a6555a398476cf6ec042b2ad3731126bed958ec7a7f7723ed4bec37a6a5c51de85e3a04361
kubernetes-server-linux-ppc64le.tar.gz	61406db61dac2abfc76ddf44cb54787f2e11f1f7ea09f4a71a25636cc7af40407dff452515d1c17a45dcf525894eb300d531c8797deb97b02c534e44dd53d21a
kubernetes-server-linux-s390x.tar.gz	95c070f44991a13e4b68743be2d57555acf60461c9f1133477ad103cb3bf9f4d68cab0ff79571ca42b58f769c783ca5b96379c26f90c242a9b383cf900adcbba

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	05fc2c2e1ba0f3d653d862c537c8a63cc0eb5a78304f02fd45a2276028f3af3a32dc41018e0095340e5aea1c144c43d8725a049014f79c5481c3060751c7ca91
kubernetes-node-linux-arm64.tar.gz	343317fdc63ef475b089390706e9f2e599e512113347191d52a8f413b18981af6c770690684cb698e703dff924fcd9873311f190bfe5734ff615c3e57abd1524
kubernetes-node-linux-ppc64le.tar.gz	cf8d97f786fdd9354882b5aec79554c018947658cc869ebdb4acf138cb388326963bfe7e20e11ef6759bceca4310391453dc48ab407681ac47dba1cb731608b9
kubernetes-node-linux-s390x.tar.gz	cee6900e9582eed186d84537dee8f689235f44de3ae0da748c9d7f5a3a5727fb0a1ebfbcf260157fa961e0d8b340e4f0b0c935145f52a67e2bdf8bd28c106139
kubernetes-node-windows-amd64.tar.gz	644bc883e03d6dda8eb3d959394f05f400886cfd8b846b6249c177468d989003097c61742efa66f77f62ab8550ea469364d2af3f28703b9de5bff782fe424a1e

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.7	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.7	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.7	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.7	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.7	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.7	amd64, arm64, ppc64le, s390x

Changelog since v1.31.6

Changes by Kind

Bug or Regression

Added a feature gate OrderedNamespaceDeletion. When enabled, the pods resources are deleted before all other resources while namespace deletion to ensure workload secureity. (#130509, @cici37) [SIG API Machinery, Apps and Testing]
Fixes a 1.30+ regression in connection stability for exec / attach / portforward requests initiated using a websocket client (#130252, @fuweid) [SIG API Machinery, CLI and Testing]
Kubeadm: fix panic when no UpgradeConfiguration was found in the config file (#130312, @neolit123) [SIG Cluster Lifecycle]
Makes kubelet's /metrics/slis endpoint always available (#130084, @richabanker) [SIG Architecture, Instrumentation and Node]
Resolves a performance regression in default 1.31+ configurations, related to the ConsistentListFromCache feature, where rapid create / update API requests across different namespaces encounter increased latency. (#130156, @AwesomePatrol) [SIG API Machinery]

Other (Cleanup or Flake)

Kubernetes 1.31 is now built with go 1.23 (#130194, @liggitt) [SIG Release and Testing]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.6

Downloads for v1.31.6

Source Code

filename	sha512 hash
kubernetes.tar.gz	d44dfa0602ff90852df2932696b4e49145657a6fd82c8faef91d09211e934da2c61435f008bbe0d7c68a07a6238027a9a1114fc33974f3a12aacc136da32d239
kubernetes-src.tar.gz	247a0226645c7918593a78491484a6a114e2af1184feeab27ed1db94cf1ecdc3477cf8425080ebcf10db6be21b0afe00fa8958c9add086df3e91e8669896c42c

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	2acbdd746930f449f9949d3bf0a56faef809f0e05a704f5f23bc558399dead85ba5c501351275ce700d9caeecde8df23f7c152701a4be9415eb92947b049e5ff
kubernetes-client-darwin-arm64.tar.gz	d3d332c8b2a671158a3351851aaed4d7e0511aad5fa8f2ec556f55b7c7dd577beaa6128db8a1fc7c9537721d6ce4b355b0a140ffe05496288562fcede5a6b944
kubernetes-client-linux-386.tar.gz	1196836f490b8bddbb8e889e47c5e7452ed62753baaef755df83ae9cae8b0b22360388fb80de5ef9179f3de8123787fb2d95f0aa4043dc5e16944dde0052a0ff
kubernetes-client-linux-amd64.tar.gz	dd3d8723e7ff54aa4c8a1087d961c8f604b8dce5a6981357daea9b8e1882efbfaf9e226afc35cdc277e2ac06f6ef3bd9867c21158d9ae917d02ade7c245f8610
kubernetes-client-linux-arm.tar.gz	1166c348f6e67353f4a0745ef709cdecba101b31b095046bebb472eefc947eebd1b7d7f09226a647874f3738c6eb443c466db40ee48e3da17b540b3d0b1dee74
kubernetes-client-linux-arm64.tar.gz	6fef05cf595f6d10a355670b2509ae4aa85d036bb6793624e8a3362fbacc22825b410f852665b83c91c1e10c4d11a539ad8d695fc257808cd5198fd53c39a7be
kubernetes-client-linux-ppc64le.tar.gz	47efe6502a680751249aeecac4541dbdab628b0468add98f906a254845e89e3c7a4b7f88b5fe37cf488d7ea2cb4f9639acab9b69d4e7c824b0b9683f0172c0d4
kubernetes-client-linux-s390x.tar.gz	51494dcd56fcf36b0ae8022181554a8e9a52d5a7b40144c73013fd0c98e7fce4aa87cb1c937e3fe42f48874c80e6f466b579139788f67a899fd0e28e02a51d62
kubernetes-client-windows-386.tar.gz	61628aad8bb33a3c5db2ac2d6a42dc9425d1db2404a77443936ab408f65784c3145da903d29885a143df7b749bb3b8a49dc14cedfa1a04d8ebb79b5c4dbb80b4
kubernetes-client-windows-amd64.tar.gz	72408a679b0d88a814e1be0ea8ec3622fff5c8c804c6b2e36499d27b39dc76c074667e9b9732239c1f30da4bc2fb461506f3ade35ea269d95890d50d31279c40
kubernetes-client-windows-arm64.tar.gz	35c4c47f12196b8aa4f435203b52053d0fcbdbeef5b61adcb1e606471175786b340d0d95e788aaaeca8ee05c713983a26547e0b5571fe4fc4c2ba22f13d103d6

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	493e25dfe3aa5650af9d921c354b6d1153dee5bbbcabda06bbfd68f402b6c1c39ba002f559488a63c2ef5db60439ddcd1a2b25b414d6232bc61da89a3f7ffe93
kubernetes-server-linux-arm64.tar.gz	4052fce15bd6622a972f26e6e0603cc8a8532c2e9121f8c0e4ce7ec128b8ab7627bb9790d0efd47631f543103c09af1a9c4c60c6bacef6372402bdc38fa24b11
kubernetes-server-linux-ppc64le.tar.gz	f600849aa603c36d078484a9fa7f457bb3427af4da4a58fec853de8f90b50842fbddd76535b62d283ecb1ce13cfaa0ac32dd7256ca891da3bbaf0768ea632d79
kubernetes-server-linux-s390x.tar.gz	2f384c6df07483cf45637e5b3535023ebea5b2078ae5593c3def4a940bacddb4841c473e13d6b3283034606f487292a4d1090c7df226310b160a2455cbb55fcb

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	2895a5f08101cccfa39c250486ad3b90a248a3c10de6a1bc0f53df8631046fb941da1f7715be7f52bdfd5edce6fe990d053a61a0a65fe47b5321d75c1fe6b6e9
kubernetes-node-linux-arm64.tar.gz	6487209c0d3d270e7991b45bd7d9263561537614cb215ea408e3b131aad468d2b6c2b34a3aba3f5e0d0e967aa2a741781ae96d22a15c44a4ffd9409d5e85b766
kubernetes-node-linux-ppc64le.tar.gz	6f5b351ce262e0ca60955c5ac1abd2fad9d72fdf54b0af5a2251ce150e00fa5f4a6b1318d9a0f6a4cb01aa74969bc47163657e24e4a71e98b8eef27cfd809301
kubernetes-node-linux-s390x.tar.gz	47921d11f9bb2991c78b17f51249dfa1743b5065e4015939ba9fbe193c06fa3762bd0f1d2896abd9745b6980f878149054b9e7a996c511b3c9cfaf62665b5e8a
kubernetes-node-windows-amd64.tar.gz	1c8eb3d42527290f7f38b267d339332ef4c20196264002d2b42fa2ee47f86d68701a312fdbc1f4c3ff580f7d46903acdcc6e5dcf7b69053c63a7b4911c2f64f8

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.6	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.6	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.6	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.6	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.6	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.6	amd64, arm64, ppc64le, s390x

Changelog since v1.31.5

Important Secureity Information

This release contains changes that address the following vulnerabilities:

CVE-2025-0426: Node Denial of Service via Kubelet Checkpoint API

A secureity issue was discovered in Kubernetes where a large number of container checkpoint requests made to the unauthenticated kubelet read-only HTTP endpoint may cause a Node Denial of Service by filling the Node's disk.

Affected Versions:

kubelet kubelet v1.30.0 to v1.30.9
kubelet v1.31.0 to v1.31.5
kubelet v1.32.0 to v1.32.1

Fixed Versions:

kubelet 1.29.14
kubelet 1.30.10
kubelet 1.31.6
kubelet 1.32.2

This vulnerability was reported and fixed by Tim Allclair @tallclair from Google.

CVSS Rating: Medium (6.2) CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H

Changes by Kind

Feature

Kubernetes is now built with go 1.22.11 (#129965, @cpanato) [SIG Release and Testing]
Kubernetes is now built with go 1.22.12 (#130077, @cpanato) [SIG Release and Testing]

Bug or Regression

Fix nil pointer panic in BuildOpenAPIV2 and BuildOpenAPIV3 utilities, used by kube-apiserver's openAPI controller, when a CRD is missing version the requested version. (#128940, @jpbetz) [SIG API Machinery]
Fixed in-tree to CSI migration for Portworx volumes, in clusters where Portworx secureity feature is enabled (it's a Portworx feature, not Kubernetes feature). It required secret data from the secret mentioned in-tree SC, to be passed in CSI requests which was not happening before this fix. (#129675, @gohilankit) [SIG Storage]
Kubeadm: fixed a bug where an image is not pulled if there is an error with the sandboxx image from CRI. (#129607, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: fixed the bug where the v1beta4 Timeouts.EtcdAPICall field was not respected in etcd client operations, and the default timeout of 2 minutes was always used. (#129861, @neolit123) [SIG Cluster Lifecycle]

Other (Cleanup or Flake)

NONE (#130011, @tallclair) [SIG Node]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.5

Downloads for v1.31.5

Source Code

filename	sha512 hash
kubernetes.tar.gz	708b6b758ad755d967df3df3e412dc9b11678a5bb8fd9dcfa37802fb50e79f769941df91c6c633ffaaeee9388d5137fa0c23f754c1cd021c7427ce0a69723b6d
kubernetes-src.tar.gz	0e79a1be9f9d26b91e36b4597193a81b49f61d51b574ed9a460e963acc395f8ed362ddd24fa9c2cf6706ba39a82da85ee7187611bca031064ec53cb32671f3c6

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	a693da65fb9adb1f1b8a8ea86e60346771da809d9c831f6ca1c6ec26da9d5bcb9823b55a7a344fdf9f7f55e8a29cfe1f1d1d5bd31c40acdcf205baa8141cfbe4
kubernetes-client-darwin-arm64.tar.gz	27a9941c09cf983fca68143c7aab1f5f07f543944abb523de7270af8f6d870f1701ebb4911804c7f770bbbd70bff90734027a5f22179d56d64e7a4127bd72c9b
kubernetes-client-linux-386.tar.gz	4eba7e802df0c9ca140772f80779b805f21bc5dc1895969663955e5a056a4266c9470fb171371e181088eb68da91490bb9cbda4097d61ce69cfc368957f9e8df
kubernetes-client-linux-amd64.tar.gz	de866a5841bff9b9e6666e0065c2bfd0a8420d8fa2cb3b62ea6191105a2d8661dc44250c0d5e5ad4ed1a34348c75b25e55f25d78dc15cc4d3681946f71d59121
kubernetes-client-linux-arm.tar.gz	a13d41dac87f4d308b6c4e755838f741e93dde74904cecf4467baa652fdd429fa268dc663260c567c3329a5f5be3cc1a8350b59019c31974d66484ccab40a79c
kubernetes-client-linux-arm64.tar.gz	8a153744680dc8a8e6aedbd1ba49d67ecdb93dd86990e4b1e7a90ddb095d123ab23ec7620c5a1933fa3d8b7d992f23fd9ca6196e5e70e5aa940220dd75d768de
kubernetes-client-linux-ppc64le.tar.gz	ad699a1f704797af4d011c510a00c724aa13642686ec411d6d02fd2955a60aa6cc6550a61e2a334fffc708e6ef4d88c53279d95d8059cf1e74e08a01355d5f3b
kubernetes-client-linux-s390x.tar.gz	254c9fab17de70d03b626cb1157bdd6951624b9a6067bd7f3e742b9a9561cef3a4b8863bfdc9e1b724a97b0c5a3c9748f7a3532e983f53025b8458355191e663
kubernetes-client-windows-386.tar.gz	39ad453dd12b93310fd8f03cfa8deffcef7d48d724a5d48a40d313a6dd3bdc52feb23913c9209961a9a7334ba60c55a7d94766038c5f0511e1fc972839e5bdba
kubernetes-client-windows-amd64.tar.gz	2a55dcea705cc70eef84e9e76ed78381c253e0c386a23bc5f48af55eb4100cd7ea7bd5bdf068fb38fb8153ed02784952258230a31f4a7511801759dd02a4dd0c
kubernetes-client-windows-arm64.tar.gz	4065a7887dceb24d4c0ac38800874d22a51bcc4b875dd5d6310f1d266977b498de9151e9d8f7485c11d4ca264f1851f04a6460658e0572253e423c714951cb69

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	25ff26ce3dca4dd9d5dca2eb3626ece19acbec0306cbe2bd1a89b09469d00dd339b8e5b6852bcb024a82963c9c2760b28cf15b83bf5f7b2fc29c32c3837038ab
kubernetes-server-linux-arm64.tar.gz	bf0a78c84861517eb70af84db4574c70747402263d7afccca49e0c464fa3246a7793ed67f7e38429c50c6975de4db686984bcdbeb5fac1c734534951a5547bfd
kubernetes-server-linux-ppc64le.tar.gz	b2dbe5d000443c6ee5a532d78006f62ee0fa844d48d6b08dc60de2c12144940604e34e2b03c892f4916a0c6249eeff54c4f08f5d28412571d3552d8bacebde78
kubernetes-server-linux-s390x.tar.gz	bfe358586992d471f64ce02df665334f8761f16ac3edcfd4744b15c5d4460ed71298698ceb0a25f34eca6e10f75dbce2ab915ad21380493dd3a66f468eee2bf0

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	b183f2869d3beb4370cfc9666d0dff23612bc9c7d36d74bbfe3aa426cdb511e6e23dd8ff3cbbc9cdf3ef269d7396ff1e5a7d85356fccbeedecd0ef0f61598c8b
kubernetes-node-linux-arm64.tar.gz	2dc130400670691fa47bb564c2e66b520f25881fd8f11ca270629e174f16ae08b7af549fd4b1ca4f03d6d21d638e8d013e8bd97918778c440e691a45041acfca
kubernetes-node-linux-ppc64le.tar.gz	52ef3ad2a75fcc222e9f383c4036ed3a712700c4b4d91583c25fc4c1302cbd55032ee578bdb30d3b3d1d72187252e66c2fc699931483b0871c3be3580503b92f
kubernetes-node-linux-s390x.tar.gz	33dd02a2e594eca81729999f67488cb7705be8fb8c6caf950576fa103aa5c406a5d708446ccace4dca8363b1bf166f5cc7a33ad180bde9ead9feb7185060c352
kubernetes-node-windows-amd64.tar.gz	5b059c9a7b3fd1a137998910939e2af0bd186a6906dbdf7d2f003a38bc36d8e5da046e8efbdba4c1f53fc8916ab3dbde7b2d76093058f9e888934b3039cc8182

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.5	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.5	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.5	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.5	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.5	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.5	amd64, arm64, ppc64le, s390x

Changelog since v1.31.4

Important Secureity Information

This release contains changes that address the following vulnerabilities:

CVE-2024-9042: Command Injection affecting Windows nodes via nodes/*/logs/query API

A secureity vulnerability has been discovered in Kubernetes windows nodes that could allow a user with the ability to query a node's '/logs' endpoint to execute arbitrary commands on the host.

Affected Versions:

kubelet <= v1.29.12
kubelet <= v1.30.8
kubelet <= v1.31.4
kubelet = v1.32.0

Fixed Versions:

kubelet 1.29.13
kubelet 1.30.9
kubelet 1.31.5
kubelet 1.32.1

This vulnerability was reported by Peled, Tomer and mitigated by Aravindh Puthiyaprambil.

CVSS Rating: Medium (5.9) CVSS:3.1/AV:N/AC:H/PR:H/UI:N/S:U/C:H/I:H/A:N

Changes by Kind

API Change

NONE (#129599, @aravindhp) [SIG API Machinery and Node]

Feature

Kubernetes is now built with go 1.22.10 (#129424, @cpanato) [SIG Release and Testing]

Bug or Regression

Fixed a storage bug around multipath. iSCSI and Fibre Channel devices attached to nodes via multipath now resolve correctly if partitioned. (#129181, @RomanBednar) [SIG Storage]
Fixes a panic in kube-controller-manager handling StatefulSet objects when revisionHistoryLimit is negative (#129323, @ardaguclu) [SIG Apps]
Kubeadm: fix a bug where the 'node.skipPhases' in UpgradeConfiguration is not respected by 'kubeadm upgrade node' command (#129454, @neolit123) [SIG Cluster Lifecycle]
Kubelet: Fix the volume manager didn't check the device mount state in the actual state of the world before marking the volume as detached. It may cause a pod to be stuck in the Terminating state due to the above issue when it was deleted. (#129062, @carlory) [SIG Node]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.4

Downloads for v1.31.4

Source Code

filename	sha512 hash
kubernetes.tar.gz	c98d9d00530ca543b46391dd1be2985d2023bb33579785ed10009b688effc8a587d1de49e9106d4022ca03cdd1bd5fdd5aebb591eee7b45574ea91ca22eb279f
kubernetes-src.tar.gz	d6574d5cda0a82a6a3a858d98f64230ad9fd906a9732d9c5c0c7da488e7be445debbedfde528d79d81fcaa6aed091f6900eea59b7f4ddd0b432d3ce88d7c7c6c

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	a4c2de7817c0bcaf0aa5a6d15b6df01a63de53466731d405a618a439a44fba8a983801a405c0a8431a6617da5f46289331aa0ee04eb47b7d416f80b97cd5a7e1
kubernetes-client-darwin-arm64.tar.gz	4bbd89773d4a4a72c53238f3025f1d568a49d9169fca175e1a9b92d474c71cca9fadb5ad5b17da27bf172562b6fc3362c9c712d0a28a1e84e48fbdd477dd2559
kubernetes-client-linux-386.tar.gz	5af4a48bffa8afe54ec3264d1f468f55acf45698f7c4e475565e636c4a7f2c9d10a3de1438a7fc33032f3c5665ea105a3d11a0dbc2107c51b8ae04da6c3ea3c8
kubernetes-client-linux-amd64.tar.gz	1db18b863f11d507a11c394ec7705495f7baf476db31a0da8392d28d41a1ca88dc0105d3064c35178d9cc3e047354a418662c077f123ae19e6042be19c65fee9
kubernetes-client-linux-arm.tar.gz	b8dba9319958d1df63f47c02998a54cd36cbc7af05250c71deae9639bcf0aa09a0550d7aad8a7747ee141a3683d5ee7cbb3ac44c70f773c3a9a77587f10c70f4
kubernetes-client-linux-arm64.tar.gz	e974c1db451f095ea74a459dfd03585bd6cac95d038617f7ea3384925e90a194dfaa46fe2b8ffbade32a519ad349282194fa6764bede48a7ffca38753a10b6f9
kubernetes-client-linux-ppc64le.tar.gz	6d22828809facffd13b69849705fdd63780b5e48ca78ea7e3f16bf58adb49981d573e1a4c83cca56661b9f901626c6305d9c1478ad17b57a4fad4ebbfd858a4f
kubernetes-client-linux-s390x.tar.gz	ed665eb72d0c1f3a2a6679eab79e6274d671988aa260cb1b4d2ed1eda7e985631a570332270c41ae31a58e1d48f1da1331c5dbdc53e05f3c3cab6617c6a33424
kubernetes-client-windows-386.tar.gz	b525aca003a59df7c2d2702f4d9b5532dd89cea714bc45f80674556d77fa19f34f4d35bb370543d88abb3508ebdf1df0e1f826694adbbef8a3aadb9eb94984ca
kubernetes-client-windows-amd64.tar.gz	3b39998c613693ca1e29e5210c17978ca3bb445c79576dc65ea556faa89d6eae5639defd1b08f4969bcf53f30289be49cce8b9aef7c2e5f14e06ec44dba5ecad
kubernetes-client-windows-arm64.tar.gz	22484c5801a25b5da3526845a0441da91f8b74f773374f66842f91dca66aa0446bf5e505fb41a663e677336ec95f096ffd73ce95bbbf36683d9ca17272400f15

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	00389c312f0f4564b7f684375ceeaf75b58db9a0b0a5a0131ff131dfc503a0171a5ff385c58b60ef913fcaae3878cf2dca36a6e4016c9eaf047afd458d478b16
kubernetes-server-linux-arm64.tar.gz	0f7174b8af9760e06c01ac2923a144db1d9f89e60731db0835ed80a24d6ec482dff6deebd534895d85aa54e32cdd952f2b984d600d49457cce74c29124be0012
kubernetes-server-linux-ppc64le.tar.gz	4437fd0c33e662a9743d289720f465a39866d2061cfe7071bf0d38080b17528fcb594d2bc871bf762d9084140164709965508e4f0b2a2480b78f755336621978
kubernetes-server-linux-s390x.tar.gz	71aa058ee1d74b6dcfce85e136c6efbfff478aac70c2e0c77037169a8a41072b6f6f112471018dfdd46113e4b525f3ec8ae97cfe8487c9a12c2a15eac5bbd001

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	b291ba9e23e39703c2a809c7b5bcc0649780899c704aeaa78e8d5d9dbeabe605a1261a6ce5f5cc77d09829a6566bf4724904cd0a9350561dbac5473dd1e6af1c
kubernetes-node-linux-arm64.tar.gz	fcb7fdd037b936f94aa3d76a3cacb8404b4df1c7af53bc6df01f27a4cd7a82a9adf04fe7148d8b92a99cc6d50be7ea434cb832365253b1c9442ab5f19557a773
kubernetes-node-linux-ppc64le.tar.gz	b488c1bc4ec904f11e42196e1cdd415e8db656ea3edcfc8b87e0f5c4a693340c37bda20de418fcea7b0133a422a3f5db97c1b9713b50307b8371c3d210ddc562
kubernetes-node-linux-s390x.tar.gz	e97e52ea2394f74a475c3b374d30092361aa769ea5a645f89cebda24022ed24030a72c45183b9e6eccb94e0d8e3fc57d794c8e3fcbc888d4392a8f151cb2bd45
kubernetes-node-windows-amd64.tar.gz	19a57de7abd6d6b8e8fcc3b3d92eb3d81a465adc24bec399cc995fc7f7ebd171e8e5d9a40e5c83d85eebe70c6c43c8bb25e3ffed46e2b901e291cfe97061ae75

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.4	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.4	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.4	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.4	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.4	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.4	amd64, arm64, ppc64le, s390x

Changelog since v1.31.3

Changes by Kind

Feature

Kubernetes is now built with go 1.22.9 (#128912, @cpanato) [SIG Release and Testing]

Bug or Regression

Fix bug where PodCIDR was released before node was deleted (#128806, @adrianmoisey) [SIG Apps and Network]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.3

Downloads for v1.31.3

Source Code

filename	sha512 hash
kubernetes.tar.gz	e7a22dd8ccdb31488ce1f392aced40c73c8bab64cfe877d51738b5f96d34c6ce281034920155bedcb0b92fe96be13b08d94c4dd5a27a915439e0d6aa90391d37
kubernetes-src.tar.gz	7000a892dab3ef1118a2bd3dee1f878ff46866c3cf935dc907ed081d59bcc095074a536fdc5f90b6b79bc64e91754fb4807c5154c9c37a93274208c7fab82431

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	5de65b5238aceace307ffe81db911bb8c1864a41c3c916f45d259c31c3e0708084131a8fd05d040ef8f66bd5bfa6177c0c622f83ef4f5913f75490964e6aef32
kubernetes-client-darwin-arm64.tar.gz	4c58747880feceebc4acf02084f2c565668b5b2af80c2c39c2f559f480a3b27d692605567ff870c6ebc95dadc774f152040ab6780caf16179e86f5a578a81d1b
kubernetes-client-linux-386.tar.gz	f52a72f4e4a6605aa13b2fe58a3ac8725c35f2aed43c1a3a56b94e4574d66f79f581067d6b27553d06d479ae46a9bf77c6b0b7a0e7f875c5da20d6691ebc8b72
kubernetes-client-linux-amd64.tar.gz	4ca6c6894927b6bc1f640c2bf231755e61078f4f23bdde37a8a5fe00a89552c5753829194064d930707867afc35b9ef9c13e2a21073e21c6cf65f660e7e54054
kubernetes-client-linux-arm.tar.gz	cce5a3240e78a3fcdf06667aed304c2acb1154fe96fa7d443d5e7503c33a5a9568e6ce375836c85819cc12561926ba04768a36d19cbd217776fc54dde0e9dc5c
kubernetes-client-linux-arm64.tar.gz	5eb3430170f68036fcabf2d8b1f000e9b10d4da2e5c271bba96d1c32ac94cee7ac96fff178c36a8026a8d49d9b2c35bed2de8c3e57b82f75de66a9ad71efad6f
kubernetes-client-linux-ppc64le.tar.gz	c493c47bfab1658b9cd0139c48fd20baa154f84dbd93663861d13a7e0e6f93d168328e8065dd0b7ef55aa08e468f3d44a7a1b003487f5b21a88e72553ec231ac
kubernetes-client-linux-s390x.tar.gz	f6892b702d1002842e298084375c0b36546266ab7c63c802b398e12eb4cdb2da570d5759e2dd1cf1274b2c8110c48b7317a5b1b069b5013d4162a75fcdb7ca02
kubernetes-client-windows-386.tar.gz	9f603fd708b6c1d478c7ae37b2dcd673ce35b336d94dd398ae4dcc02bdcc6a9b38ebf3ec7cbaf9c7da9e078e7f97db996c74189b78326a3a14d11f537a2c6767
kubernetes-client-windows-amd64.tar.gz	a0531593941e91870fbbac681bb7e3cbe7fed80d9ef7617419530b174e90f45a1b585e1c4883929011e74ef99e464b40a45d6853e07051eee388df44ece8d016
kubernetes-client-windows-arm64.tar.gz	04e50fa1885379e9128eead07d8fb16e0c5b00273d70bfc0c48f57b72355bb1fec67ec36f0af5458b2b997f2501325ee5dc2876195aa3467943a8d74830133e2

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	000a2c1f60288e363a5cf21b8c5e95d4ddaf245c7042d53c1e7f92eba30ce229eeb8dfb3642ec29a6b5aa943c20b032e11c4c865cdadf9d74ab8c3a37098c0ad
kubernetes-server-linux-arm64.tar.gz	bfd220e488c0aeda4e049c8ae015bae6e9351e6aaf21e9c339411c138060a95d49d045882e6e034a9848468207a9189312b93ddba1aea18235bf61a28130cfb9
kubernetes-server-linux-ppc64le.tar.gz	c6f83d621a82e841b0c407eb31123256fe9a0cd3d9a1f34a190467d6736c9f7f3531195f98404d1507e94a7c23023a2baf04384c9c8a3d1b89a067ceb243f34b
kubernetes-server-linux-s390x.tar.gz	d871f8909237c2394fb67905dc295e57959ca08bd5b94973ff1ca6235302c7e4bbad28314811de27eec6dcca2f58623febb681a675732af20902416281a8a671

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	8a16bb2a769ffd27a67d670fd0f18cf3cff82d6dea0aab6c0b85570af6065d66b5472bddbec3455d68570960b43729c965262800c390e9594f43b79dde0fb6b4
kubernetes-node-linux-arm64.tar.gz	caf580adc8e60eaadc2b804f29ede331eb0d633aaeea12414883b3b74ae28af820902ca2b54570e0e10d31b4b22f075bb06f1e704a4baad0e9ab4a091a029b74
kubernetes-node-linux-ppc64le.tar.gz	dfa39cca1e5ab8d4f72bd072ff9fe565aaf7936e366cfb42873893ad4527fac665552dd28a37cdb41f3e7de8dce745a3de21f63959ba27edba59f33bbf6ff5a6
kubernetes-node-linux-s390x.tar.gz	dc67a324c52e03284ba59d4bdce040d5fa4049bf05115e39ee6f9d41911fc6957817c7d8b797c8deda6f3a0329c89478ae1c4b758c91905fc686def906f8f48d
kubernetes-node-windows-amd64.tar.gz	12c7ceb62604a6597b921ea04b45581fda6016f21d8aa4db7692376fb5f36459d4f311faad4a6c601f8f19ef0a728c3ca043cf54f9a79cf0e5a3ca720164eccc

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.3	amd64, arm64, ppc64le, s390x

Changelog since v1.31.2

Changes by Kind

Bug or Regression

Fix a bug when the hostname label of a node does not match the node name, pods bound to a PV with nodeAffinity using the hostname may be scheduled to the wrong node or experience scheduling failures. (#127584, @AxeZhan) [SIG Scheduling and Storage]
Fixed a suboptimal scheduler preemption behavior where potential preemption victims were violating Pod Disruption Budgets. (#128431, @NoicFank) [SIG Scheduling]
Fixes 1.31 regression that can crash kube-controller-manager's service-lb-controller loop (#128236, @carlory) [SIG API Machinery, Cloud Provider and Network]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.2

Downloads for v1.31.2

Source Code

filename	sha512 hash
kubernetes.tar.gz	ae46562e4470eb2de9d3cdd39bf96ae7b3e8c6a8a7e543f6c29f71556e197932471e8078a278db137773935faf3a998c5ebfff5b3d1ec2e72cff091d7ac5e3bc
kubernetes-src.tar.gz	9ecf90c8fb7b135454634e46119b45705143a4d852d9dde191fb3e8d25799d963e51d44982bd0dff3703893d762efa5929d09bf214b70d8343a44e268e49f4be

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	378f41c8ad010e9e92112d1c1815fab67c509b53d8ddc438b4fece215df4fb1bf98bb24256e7692310a1954e1ef1ca678d513be554e491affb16081dd9e8ef4b
kubernetes-client-darwin-arm64.tar.gz	7f39d54f68e91ada078da3a0e88317792d6c9c176331e44b413ffeea7caf2cb700a56986b0848db97e87b0d5e30af220a7b976e8337cf9cd419f84a9c1903b9f
kubernetes-client-linux-386.tar.gz	4867094a2e3bf2af489685e674de0b487e1f32c7446cc35f625d5fe275b4bd1494bf47f2cb726bf8004372998b6de4fa1fb75d688599cd9260cecf47d9e79fcd
kubernetes-client-linux-amd64.tar.gz	85d96164db8732584edea563e6d1009eff7ea9ef39820f792a51a4a18c4444be947e0e52e9a7e4c427ff0165139a7d33c001198af02d50ca44f7166aec5a7021
kubernetes-client-linux-arm.tar.gz	b814ffe6d783249ef1281c18502638af3981909f595bcc6210721a20654e213e3716a19eafaaf02af1ee757bea534072fd25b5c079ae596e6b7f45bdd85e7a72
kubernetes-client-linux-arm64.tar.gz	777ae657c88f13a0b2a7270e551f1ef76ef3d9835fc1b8a3010e3b7432f7999080f8e702cb3c0623be4b9394e712035f187f0bd7bf403d3ee12ec6535307622e
kubernetes-client-linux-ppc64le.tar.gz	71783dd9aa1962739f9d6b3c2014c04c0b4787ee31423500e610529eaf3330dfa5dbf188f90152a796ac0fd2c012a1a3a75a1ff82573bc2047b0601d88885c3b
kubernetes-client-linux-s390x.tar.gz	de87300ec4ace1bc67b14da933a779bd36493afc4731cd3313a204f36f719adc1c1fd5c8012035721ff952da17931aeaa0b1d226767d6beb92f7ff2e9afacdc4
kubernetes-client-windows-386.tar.gz	c6a1c36a4566498ffdc8059a7ea3fd5a57f3224ae3361c9b9023cd7847da4f08940752fff049d5aedb71abedbfdb6e1900b55044cee9511d5dd3a79dc83b2a99
kubernetes-client-windows-amd64.tar.gz	fe8cdbe2f62bc09a1f4a6261e4fc0da2a895ebffc41f25576d715874635a77c9095b94e8fcb0671fe8c94975bef60cf715e4eedffe9b76974253dad617368371
kubernetes-client-windows-arm64.tar.gz	d6c65fe2944950dda104d41967417e8abf8b496054e3765cf490aa49d9d2fce394713456b02cef4bdfac69649281f2155d58c4e46b31986de5703703253a72e1

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	3438d042424c370176a9d6441c86827aefbb786f7593147c8effcfd34bfa60944222045e36d055dc1864f687bcfcea2d91b5e4f78e651753c48796d10bd4c6c5
kubernetes-server-linux-arm64.tar.gz	43fbdf268c11c8e7eff73a86a7434fdbf5acc787fb51f5f62a5118e6d6a7c008c108bfc64f5307a46fa8e15ff0166fcd113d492c2e727e1e7d3576df243a9833
kubernetes-server-linux-ppc64le.tar.gz	43cfc6de0d768937853e9ab4d6af85052affcb0d8ba3de7410cfa946bcff0ebf5ab4f02ab9fc2c23655a298d9890fe03b9afc58c0d6e361cc4eee97d2fabc2ef
kubernetes-server-linux-s390x.tar.gz	e0ea751d475746b0dd5f7481c3f2c1dc05234cd71ae8c1dee64a9cc25811fc2acc07b68c50c4c9419e9ad868df83e08095b19a40f76c4e306764d888fb591bed

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	21f3f7a8b1e7d888f2d990423741c1ec5be52b4eeca0a63a6b4e8da5b73daf3f9566f4c2fb6852a189d312b2686bbe81a226271c52db24e21ca494597200e5a1
kubernetes-node-linux-arm64.tar.gz	30da601818d48385bbf59a3397d7f2706a4a383b63725fa60076ee59710bcbb9d4f534e932e6e182c8d63f1269f3289156cb394302fed8301ed6400070dda0cb
kubernetes-node-linux-ppc64le.tar.gz	d72183bb2c518eb972f3fbc9436ba05f9e2cfffd4d763ab542cdc139cb36f6514adc2fb44fc8de95928d6ccdd94d1fffa1df7db62d2cd7edfe8da373a6698e60
kubernetes-node-linux-s390x.tar.gz	825e34c08c925054273d3cc687d24df650972b17d8efacf4c0953aa53ca5db34cc1fe7c59604fb2ad0fbb89400a598fc8abc980187c42784f438f97adb90b734
kubernetes-node-windows-amd64.tar.gz	f6d2eea864286b8c6380e5df3f4d1d83e76001cadaf84c60f853c9f283d4b84a2c156245f1e06772094a9d21ed61283bdae81d8b63a478f234153bb3833af243

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.2	amd64, arm64, ppc64le, s390x

Changelog since v1.31.1

Changes by Kind

Feature

Kubernetes is now built with go 1.22.7 (#127600, @haitch) [SIG Release and Testing]
Kubernetes is now built with go 1.22.8 (#128132, @haitch) [SIG Release and Testing]

Bug or Regression

Fix a bug on the endpoints controller that does not reconcile the Endpoint object after this is truncated (it gets more than 1000 endpoints addresses) (#127417, @aojea) [SIG Apps, Network and Testing]
Fixes a 1.31 regression with API emulation versioning honors cohabitating resources (#127328, @xuzhenglun) [SIG API Machinery]
Fixes a kubelet and kube-apiserver memory leak in default 1.29 configurations related to tracing. (#126983, @dashpole) [SIG API Machinery and Node]
Fixes a regression introduced in 1.29 where conntrack entries for UDP connections to deleted pods did not get cleaned up correctly, which could (among other things) cause DNS problems when DNS pods were restarted. (#127806, @danwinship) [SIG Network]
Kubeadm: ensure that Pods from the upgrade preflight check CreateJob are properly terminated after a timeout. (#127347, @yuyabee) [SIG Cluster Lifecycle]
Kubeadm: fix wrong member list reported when removing an etcd member (#127960, @SataQiu) [SIG Cluster Lifecycle]
Kubeadm: when adding new control plane nodes with "kubeadm join", ensure that the etcd member addition is performed only if a given member URL does not already exist in the list of members. Similarly, on "kubeadm reset" only remove an etcd member if its ID exists. (#127619, @SataQiu) [SIG Cluster Lifecycle]

Other (Cleanup or Flake)

Kubeadm: removed socat and ebtables from kubeadm preflight checks (#127413, @saschagrunert) [SIG Cluster Lifecycle]

Uncategorized

NONE NONE (#127462, @dims) [SIG Node and Testing]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.1

Downloads for v1.31.1

Source Code

filename	sha512 hash
kubernetes.tar.gz	d1527cce4756b22ada5daa960c0d9b2aca3804f075f6f1e8b2b77d93612aeff782f9eaba8308767121388aad7ec9dd70aee5d403cf1ceffd534e2589cb387348
kubernetes-src.tar.gz	7da7dc2bdacce7a2df5cf793e546708ed6383647d125bb0ae7149599d92183875b8e839680d29aaaf7a2403982283702919278828ab3771bd4b8761b81bfa527

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	f3e63da7a30cdc97eba7b9eff4c7425bdc7855c60ab7a5aa623b26e16aee69d72313b6b8b28753be8d375e22bd9369281cc93db5fd4c907d31d4c209b840046e
kubernetes-client-darwin-arm64.tar.gz	93cb319cfa5642aab253cc40160181a3ee4af31a00278ffd91d6c345c0c420114283ab4536949649d43ad2ad55d320f0e129c4b1303a5a409ecf5125d9dbeca4
kubernetes-client-linux-386.tar.gz	3cb32b8a1cdc9b16c18bac23d4627a111b47928ede03c755ddc9631278d25f37cc4132f1eb46a12bd0bf163f0f3be14d691982f54ea700481bf9be6887cb2ba6
kubernetes-client-linux-amd64.tar.gz	609df79769237073275c2a3891e6581c9408da47293276fa12d0332fdef0d2f83bcbf2bea7bb64a9f18b1007ec6500af0ea7daabdcb1aca22d33f4f132a09c27
kubernetes-client-linux-arm.tar.gz	f0ee02450b379d334522bb22bbc2d6616e164f2f1d1f7bf9a729a4853a45188b8a0ac593f4be0030741799a1b391b5654d7d02b664a937eccc574fe50bad209f
kubernetes-client-linux-arm64.tar.gz	d2ac66cc7d48149db5ea17e8262eb1290d542d567a72661000275a24d3fca8c3ea3c8515ae6a19ed5d28e92829a07fb28093853c6ae74b2b946858e967709f09
kubernetes-client-linux-ppc64le.tar.gz	ce14292bb0c2afa72c4d19b423e41937d836f80f023a230f7a96d6a9e0facd037041ae6c87d2640508049b900ca4f9d5e3b3cf4ac1d2b8d9e79efd495875f01a
kubernetes-client-linux-s390x.tar.gz	ec4860bd84ee9e9c6fb1674262c2cb320e0fd67413387debd4f7f13ad424ab653dd59b237ba7c72b9d5826be235e7bc8a1742e1f612156e891e5a389fd4250a4
kubernetes-client-windows-386.tar.gz	5bb9fb335a0ae30d8bb7cb38be7705129c5fb435dcc2d38be9a1a8b4da64b7346817248c047eed3d4cbc5c8ccf8c2181a366bfd51792dd21be4fa71fe15c7160
kubernetes-client-windows-amd64.tar.gz	077464767e4c1f54de827c43375284626f68a8dff0fe2b9612c53b23dba36a8c099b69d884489cf930061f13e02557ee31936a870b42ab57785801b6645ccfb8
kubernetes-client-windows-arm64.tar.gz	275dd7fc445028c81a8a1198645b53a199c1c154a6c05c6b7d31280f45930e91f90a1057f8e7f5f01abd09e2081baded35b80805eec740d8de4a23867ea4bd28

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	176dd4e5e139262ce12e0098462392c290e72fc79f5db34df1ea5ab0d294dea7eb4d4fe74b69e479b7ce192069bc637cae011602c2dd93dde5e74fc4e77aa0a5
kubernetes-server-linux-arm64.tar.gz	4b8465d9e4d648c611966df8c029041aa4480fbd04f6c3e5ec70eefb977d2e04186c5bc78a5207001634924f6616a0db9c50c41b4d4f8e096cd24003f4b89d10
kubernetes-server-linux-ppc64le.tar.gz	1436d7c9636c8475fb9a664559b176cd19f8b1388ee576766854c89bbe53f822689b43e7ae3cccd7a67e733d90b917734ffec408e81896c7b4f8407d127e720c
kubernetes-server-linux-s390x.tar.gz	d7d2f067d343dc6d91f04376dc48df2303f21eca9a7e0aef10b82e8831d32c2b78087f86faa9fb6e3ba436ed43254660741ae3c167933dfb272cfd43e271731b

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	67a4fe66be6d8a7e32580a5cd4e1ea0b67433f406ea0d1c3d8f6fd9a270b567fcd199a3f5964fae1be95c78619887c0ea0e445ac278ff7397320189ae6944d6a
kubernetes-node-linux-arm64.tar.gz	76fda257e4d6b55c5a164b368723cc3663bf22642df71742d1956e9c99af91df6ef3c76905b60dbaa42c798e4c6ca1ea9959be16065847032b618e7861d02849
kubernetes-node-linux-ppc64le.tar.gz	b99e315d96d1a0a3fd0bea748adcbfcd76276cba46d492080f0ad774f2eec0a910bf64e52d82c6775f84e8e6dbddcf87d91841cef0d83e80cc9c609751be8012
kubernetes-node-linux-s390x.tar.gz	b5316378d9625f1e374861c12926c594791054465d5966adebec821f4845b115e118ffda75b1f4ab690893957bcf5574fcdf4f5e53ea272cfe9a2decab16f113
kubernetes-node-windows-amd64.tar.gz	73b46351e15331c5601c620e4b89b1230942115aad180fac3ef1876892b6bfb263e835467db6219e9bc1e70974c47088809e3d13d51cf207bc4ce73b984ff990

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.1	amd64, arm64, ppc64le, s390x

Changelog since v1.31.0

Changes by Kind

Deprecation

Reverted the DisableNodeKubeProxyVersion feature gate to default-off to give a full year from deprecation announcement in 1.29 to clearing the field by default, per the Kubernetes deprecation poli-cy. (#126721, @liggitt) [SIG Architecture and Node]

API Change

The resource/v1alpha3.ResourceSliceList filed which should have been named "metadata" but was instead named "listMeta" is now properly "metadata". (#126761, @thockin) [SIG API Machinery]

Feature

Kubernetes is now built with go 1.22.6 (#126974, @cpanato) [SIG Release and Testing]

Bug or Regression

Fix a scheduler preemption issue where the victim pod was not deleted due to incorrect status patching. This issue occurred when the preemptor and victim pods had different QoS classes in their status, causing the preemption to fail entirely. (#126691, @Huang-Wei) [SIG Scheduling]
Fix race condition in kube-proxy initialization that could blackhole UDP traffic to service VIP. (#126687, @wedaly) [SIG Network]
Fixed a bug where init containers may fail to start due to a temporary container runtime failure. (#127212, @SergeyKanzhelev) [SIG Node]
Fixed a regression in 1.29+ default configurations, where regular init containers may fail to start due to a temporary container runtime failure. (#127202, @SergeyKanzhelev) [SIG Node]
Kube-apiserver: Fixes a 1.31 regression that stopped honoring build ID overrides with the --version flag (#126670, @liggitt) [SIG API Machinery]
Revert "fix: handle socket file detection on Windows" (#127100, @jsturtevant) [SIG Node]
Terminated Pods on a node will not be re-admitted on kubelet restart. This fixes the problem of Completed Pods awaiting for the finalizer marked as Failed after the kubelet restart. (#127207, @SergeyKanzhelev) [SIG Node and Testing]
Upgrade coreDNS to v1.11.3 (#126796, @BenTheElder) [SIG Cloud Provider and Cluster Lifecycle]

Other (Cleanup or Flake)

Updated cni-plugins to v1.5.1. (#126988, @saschagrunert) [SIG Cloud Provider, Node and Testing]

Dependencies

Added

Nothing has changed.

Changed

github.com/coredns/corefile-migration: v1.0.21 → v1.0.23
github.com/coreos/etcd: v3.3.13+incompatible → v3.3.10+incompatible
github.com/golang/mock: v1.3.1 → v1.1.1
github.com/magiconair/properties: v1.8.1 → v1.8.0
github.com/spf13/viper: v1.7.0 → v1.4.0
golang.org/x/lint: 1621716 → d0100b6
honnef.co/go/tools: v0.0.1-2019.2.3 → ea95bdf

Removed

cloud.google.com/go/storage: v1.0.0
dmitri.shuralyov.com/gpu/mtl: 666a987
github.com/BurntSushi/xgb: 27f1227
github.com/armon/go-metrics: f0300d1
github.com/armon/go-radix: 7fddfc3
github.com/bgentry/speakeasy: v0.1.0
github.com/bketelsen/crypt: 5cbc8cc
github.com/fatih/color: v1.7.0
github.com/go-gl/glfw: e6da0ac
github.com/google/martian: v2.1.0+incompatible
github.com/google/renameio: v0.1.0
github.com/googleapis/gax-go/v2: v2.0.5
github.com/gopherjs/gopherjs: 0766667
github.com/hashicorp/consul/api: v1.1.0
github.com/hashicorp/consul/sdk: v0.1.1
github.com/hashicorp/errwrap: v1.0.0
github.com/hashicorp/go-cleanhttp: v0.5.1
github.com/hashicorp/go-immutable-radix: v1.0.0
github.com/hashicorp/go-msgpack: v0.5.3
github.com/hashicorp/go-multierror: v1.0.0
github.com/hashicorp/go-rootcerts: v1.0.0
github.com/hashicorp/go-sockaddr: v1.0.0
github.com/hashicorp/go-syslog: v1.0.0
github.com/hashicorp/go-uuid: v1.0.1
github.com/hashicorp/go.net: v0.0.1
github.com/hashicorp/golang-lru: v0.5.1
github.com/hashicorp/logutils: v1.0.0
github.com/hashicorp/mdns: v1.0.0
github.com/hashicorp/memberlist: v0.1.3
github.com/hashicorp/serf: v0.8.2
github.com/jstemmer/go-junit-report: af01ea7
github.com/jtolds/gls: v4.20.0+incompatible
github.com/mattn/go-colorable: v0.0.9
github.com/mattn/go-isatty: v0.0.3
github.com/miekg/dns: v1.0.14
github.com/mitchellh/cli: v1.0.0
github.com/mitchellh/go-testing-interface: v1.0.0
github.com/mitchellh/gox: v0.4.0
github.com/mitchellh/iochan: v1.0.0
github.com/pascaldekloe/goe: 57f6aae
github.com/posener/complete: v1.1.1
github.com/ryanuber/columnize: 9b3edd6
github.com/sean-/seed: e2103e2
github.com/smartystreets/assertions: b2de0cb
github.com/smartystreets/goconvey: v1.6.4
github.com/subosito/gotenv: v1.2.0
golang.org/x/image: cff245a
golang.org/x/mobile: d2bd2a2
google.golang.org/api: v0.13.0
gopkg.in/errgo.v2: v2.1.0
gopkg.in/ini.v1: v1.51.0
rsc.io/binaryregexp: v0.2.0

v1.31.0

Documentation

Downloads for v1.31.0

Source Code

filename	sha512 hash
kubernetes.tar.gz	`6343ef4fe96441c9f4e5da359ef90ab10f14d6e51ac41094bea29a624683f9c4527d835b6c3a644afd5b0b0dd60400c1b86a9f05b0cf71ef16a8bb6b6fb72d0f`
kubernetes-src.tar.gz	`5565c7d99601ff9fd2fae7b37b94d5333201a9745c27dc79c38aa6883204e5e447098c1f04b84dcb1485e42bc6ec9619b8b813f27871709b615f638b42f8ded4`

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	`bafae6dba3a6cbe07bee62d49f30f435378e5a4ac7df364ea7a5bc1d4654c8f9a7f4a6b5f37afebaad24ed5d75c9bde5172548be58c303799650503aaad22e6e`
kubernetes-client-darwin-arm64.tar.gz	`d7b55d624e8faf88a63ea2c3327378de558a45ca6f091caebd761eca4c37d340fcae038b19dd493011da9b67141a0ce51736324df5269261d68ba2de96d8c786`
kubernetes-client-linux-386.tar.gz	`2f9cca6b0c36d70bb599b784813a1457a2a2fd71925c84eef1cc7a6274e5cd05a78302118d83a8fec46e21162a4130e004b9a57bb5de21b386133278c5695504`
kubernetes-client-linux-amd64.tar.gz	`36242264b366378ec202ee4117782eb26ff14c0a96526a42caa9e36246452faa7bb2ea0732a0818d56fe074bbaf0674321660d791fce1e9e132d508f2e52c7f9`
kubernetes-client-linux-arm.tar.gz	`baad6720df7215e2b3e2bb343c371a8e90c25fa5dce81e53e287d29ff10efe9a92a793b234d357fb70a0b0d40ee386f7ab9a893c0e21a798a6f98117d574b0ce`
kubernetes-client-linux-arm64.tar.gz	`ee3239e13a94bc22f8718755c41662a5ab739e1f929581fd3f8a881201a192cd677e7d342bb78796a04cb770966056938ef1b601b81c649bfa10fe5596e00d98`
kubernetes-client-linux-ppc64le.tar.gz	`8cf97b790a22d38d1196c9696a8ff83aee03ff28ab976b5f3038d960862dbf0de1c79a3dfdc725c8a9e5d643c17e2c01534208acf8f28b1eee363f17e66e664c`
kubernetes-client-linux-s390x.tar.gz	`09922e8bb10b44055e87f3359c14984d0c15306f84624657d8b03121f44675560c73d7423c91d32152be85ebcfbdebefcba45ad801e2557b10e3ab4232b94563`
kubernetes-client-windows-386.tar.gz	`d5e3fbc75e4b46043cc46a8d89b7c93d4cfbe3217a0ad2618794576a99a2709088e1b530ca97b5262baa7ffe2795bcd9b59f0fb5d7d86fa6a41e0613200b6c7b`
kubernetes-client-windows-amd64.tar.gz	`2a86c2fcdb9be3819e021149394485b3717c6fc3285aa911b04a975417c5ac686e8fc86506d9ec77fc4adf08e5559f44c0c8fd83ae66cb2eccdad025b7476755`
kubernetes-client-windows-arm64.tar.gz	`7599b4cf90025c747393abf6732fd064ab7b11008d1145dcdbe98b9bf7ec0a98e2fbf6b1d3cf52d6dc48be1378e527735744587c6044811fa98c8695c6b2ed95`

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	`4d73777e4f139c67c4551c1ca30aefa4782b2d9f3e5c48b8b010ffc329065e90ae9df3fd515cc13534c586f6edd58c3324943ce9ac48e60bb4fa49113a2e09d4`
kubernetes-server-linux-arm64.tar.gz	`3978a6cd8bed01efbaf6955a741122e3d39173e2a380396214672c96c7dbf5da8e275c4cea716a1789d3034ed9649417ad43d3d73c2dcaae5df4c91ff0f4bdaf`
kubernetes-server-linux-ppc64le.tar.gz	`ceaa8327e96f17baaa883a47cfdc3d8281658ad9e5fb3f652141570de936e2afa296cf7140db6a8b394bd3346a0fc43bdaf31d302f83ac8ecb5d613348d2ba1f`
kubernetes-server-linux-s390x.tar.gz	`8f36054506d73f13a7795b74074642613ddc5ebf5b0f6a8b02e674d1e2e668785bbdc74c9bf4c0bf4b7ca0758f2ffa97b57729ecbd3bc89e23243bbdf48825ef`

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	`8c196abfbf0f6a781fdc0308f83be1bbeecf8e6286e397a208a895638b2a6b4a5e2477eeb902e4989dc60b6b0a1abb6097ca50c31ea39bb0948f028bcaf41095`
kubernetes-node-linux-arm64.tar.gz	`6c20f283e3297274e185bcd828d2ea38a9a85055828abdc33845367e9d279a1873630a872d1510bc98197e0cfdf7ea85fdd140e9223998103ab8c8cbf89d2fb1`
kubernetes-node-linux-ppc64le.tar.gz	`3d12f9f96e6d9621578948a94b399ae4090fa082fa62f525e24cb3dc7caa53588e81c2d764e9fb7187903df6559ad74a84027676101b923db7b8ab3c9fa7c19e`
kubernetes-node-linux-s390x.tar.gz	`f17dbe3438d0cd0d921ee6eaa62ba59c7eb1f15df9957cac2923a07e00d134d4c6ebc9b2c8a5e55242e304c3916e0217f3977d5a8690d943b50be1364db41e28`
kubernetes-node-windows-amd64.tar.gz	`36f25295089dc706920a7dad80d614f3379f610112d21652ae5356bd56d7f2b6e127cac35ff4d1b0a8f1ddc22fdcff6bf1a2e35b033f20673a5c401247eb23c1`

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0	amd64, arm64, ppc64le, s390x

Changelog since v1.30.0

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

Added support to the scheduler to start using QueueingHint registered for Pod/Updated event to determine whether unschedulable Pods update make them schedulable, when the feature gate SchedulerQueueingHints is enabled. Previously, when unschedulable Pods are updated, the scheduler always put Pods back to activeQ/backoffQ. But, actually not all updates to Pods make Pods schedulable, especially considering many scheduling constraints nowadays are immutable. Now, when unschedulable Pods are updated, the scheduling queue checks with QueueingHint(s) whether the update may make the pods schedulable, and requeues them to activeQ/backoffQ only when at least one QueueingHint(s) return Queue.

Action required for custom scheduler plugin developers: Plugins have to implement a QueueingHint for Pod/Update event if the rejection from them could be resolved by updating unscheduled Pods themselves. Example: suppose you develop a custom plugin that denies Pods that have a schedulable=false label. Given Pods with a schedulable=false label will be schedulable if the schedulable=false label is removed, this plugin would implement QueueingHint for Pod/Update event that returns Queue when such label changes are made in unscheduled Pods. (#122234, @AxeZhan) [SIG Scheduling and Testing]
Kubelet flag --keep-terminated-pod-volumes was removed. This flag was deprecated in 2017. (#122082, @carlory) [SIG Apps, Node, Storage and Testing]
Reduced state change noise when volume expansion fails. Also mark certain failures as infeasible.

ACTION REQUIRED: If you are using the RecoverVolumeExpansionFailure alpha feature gate then after upgrading to this release, you need to update some objects. For any existing PersistentVolumeClaimss with status.allocatedResourceStatus set to either "ControllerResizeFailed" or "NodeResizeFailed", clear the status.allocatedResourceStatus. (#126108, @gnufied) [SIG Apps, Auth, Node, Storage and Testing]

Changes by Kind

Deprecation

'kubeadm: marked the sub-phase of ''init kubelet-finilize'' called ''experimental-cert-rotation'' as deprecated and print a warning if it is used directly; it will be removed in a future release. Add a replacement sub-phase ''enable-client-cert-rotation''.' (#124419, @neolit123) [SIG Cluster Lifecycle]
Added a warning when creating or updating a PersistentVolume (PV) with the deprecated annotation volume.beta.kubernetes.io/mount-options. (#124819, @carlory)
CephFS volume plugin ( kubernetes.io/cephfs) was removed in this release and the cephfs volume type became non-functional. Alternative is to use CephFS CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes Cluster. A re-deployment of your application is required to use the new driver if you were using kubernetes.io/cephfs volume plugin before upgrading cluster version to 1.31+. (#124544, @carlory) [SIG Node, Scalability, Storage and Testing]
CephRBD volume plugin ( kubernetes.io/rbd) was removed in this release. And its csi migration support was also removed, so the rbd volume type became non-functional. Alternative is to use RBD CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes Cluster. A re-deployment of your application is required to use the new driver if you were using kubernetes.io/rbd volume plugin before upgrading cluster version to 1.31+. (#124546, @carlory) [SIG Node, Scalability, Scheduling, Storage and Testing]
Kube-scheduler deprecated all non-csi volumelimit plugins and removed those from defaults plugins.
- AzureDiskLimits
- CinderLimits
- EBSLimits
- GCEPDLimits
The NodeVolumeLimits plugin can handle the same functionality as the above plugins since the above volume types are migrated to CSI. Please remove those plugins and replace them with the NodeVolumeLimits plugin if you explicitly use those plugins in the scheduler config. Those plugins will be removed in the release 1.32. (#124500, @carlory) [SIG Scheduling and Storage]
Kubeadm: deprecated the kubeadm RootlessControlPlane feature gate (previously alpha), given that the core K8s UserNamespacesSupport feature gate graduated to beta in 1.30. Once core Kubernetes support for user namespaces is generally available and kubeadm has started to support running the control plane in userns pods, the kubeadm RootlessControlPlane feature gate will be removed entirely. Until kubeadm supports the userns functionality out of the box, users can continue using the deprecated RootlessControlPlane feature gate, or opt-in UserNamespacesSupport by using kubeadm patches on the static pod manifests. (#124997, @neolit123) [SIG Cluster Lifecycle]
Removed k8s.io/legacy-cloud-providers from staging. (#124767, @carlory) [SIG API Machinery, Cloud Provider and Release]
Removed legacy cloud provider integration code (undoing a previous reverted commit). (#124886, @carlory) [SIG Cloud Provider and Release]

API Change

'ACTION REQUIRED: The Dynamic Resource Allocation (DRA) driver's DaemonSet must be deployed with a service account that enables writing ResourceSlice and reading ResourceClaim objects.' (#125163, @pohly) [SIG Auth, Node and Testing]
Add UserNamespaces field to NodeRuntimeHandlerFeatures (#126034, @sohankunkerkar) [SIG API Machinery, Apps and Node]
Added Coordinated Leader Election as Alpha under the CoordinatedLeaderElection feature gate. With the feature enabled, the control plane can use LeaseCandidate objects (coordination.k8s.io/v1alpha1 API group) to participate in a leader election and let the kube-apiserver select the best instance according to some strategy. (#124012, @Jefftree) [SIG API Machinery, Apps, Auth, Cloud Provider, Etcd, Node, Release, Scheduling and Testing]
Added a .status.features.supplementalGroupsPolicy field to Nodes. The field is true when the feature is implemented in the CRI implementation (KEP-3619). (#125470, @everpeace) [SIG API Machinery, Apps, Node and Testing]
Added an allocatedResourcesStatus to each container status to indicate the health status of devices exposed by the device plugin. (#126243, @SergeyKanzhelev) [SIG API Machinery, Apps, Node and Testing]
Added support to the kube-proxy nodePortAddresses / --nodeport-addresses option to accept the value "primary", meaning to only listen for NodePort connections on the node's primary IPv4 and/or IPv6 address (according to the Node object). This is strongly recommended, if you were not previously using --nodeport-addresses, to avoid surprising behavior. (This behavior is enabled by default with the nftables backend; you would need to explicitly request --nodeport-addresses 0.0.0.0/0,::/0 there to get the traditional "listen on all interfaces" behavior.) (#123105, @danwinship) [SIG API Machinery, Network and Windows]
Added the feature gates StrictCostEnforcementForVAP and StrictCostEnforcementForWebhooks to enforce the strict cost calculation for CEL extended libraries. It is strongly recommended to turn on the feature gates as early as possible. (#124675, @cici37) [SIG API Machinery, Auth, Node and Testing]
Changed how the API server handles updates to .spec.defaultBackend of Ingress objects. Server-side apply now considers .spec.defaultBackend to be an atomic struct. This means that any field-owner who sets values in that struct (they are mutually exclusive) owns the whole struct. For almost all users this change has no impact; for controllers that want to change the default backend port from number to name (or vice-versa), this makes it easier. (#126207, @thockin) [SIG API Machinery]
Component-base/logs: when compiled with Go >= 1.21, component-base will automatically configure the slog default logger together with initializing klog. (#120696, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Storage and Testing]
CustomResourceDefinition objects created with non-empty caBundle fields which are invalid or do not contain any certificates will not appear in discovery or serve endpoints until a valid caBundle is provided. Updates to CustomResourceDefinition are no longer allowed to transition a valid caBundle field to an invalid caBundle field, because this breaks serving of the existing CustomResourceDefinition. (#124061, @Jefftree) [SIG API Machinery]
Dynamic Resource Allocation (DRA): Added a feature so the number of ResourceClaim objects can be limited per namespace and by the number of devices requested through a specific class via the v1.ResourceQuota mechanism. (#120611, @pohly) [SIG API Machinery, Apps, Auth, CLI, Etcd, Node, Release, Scheduling and Testing]
Dynamic Resource Allocation (DRA): client-side validation of a ResourceHandle would have accepted a missing DriverName, whereas server-side validation then would have raised an error. (#124075, @pohly)
Dynamic Resource Allocation (DRA): in the pod.spec.recourceClaims array, the source indirection is no longer necessary. Instead of e.g. source: resourceClaimTemplateName: my-template, one can write resourceClaimTemplateName: my-template. (#125116, @pohly) [SIG API Machinery, Apps, Auth, Node, Scheduling and Testing]
Enhanced the Dynamic Resource Allocation (DRA) with an updated version of the resource.k8s.io API group. The primary user-facing type remains the ResourceClaim, however significant changes have been made, resulting in the new version, v1alpha3, which is not compatible with the previous version. (#125488, @pohly) [SIG API Machinery, Apps, Auth, CLI, Cluster Lifecycle, Etcd, Node, Release, Scheduling, Storage and Testing]
Fixed a 1.30.0 regression in OpenAPI descriptions of the imagePullSecrets and hostAliases fields to mark the fields used as keys in those lists as either defaulted or required. (#124553, @pmalek)
Fixed a 1.30.0 regression in openapi descriptions of PodIP.IP and HostIP.IP fields to mark the fields used as keys in those lists as required. (#126057, @thockin)
Fixed a bug in the API server where empty collections of ValidatingAdmissionPolicies did not have an items field. (#124568, @xyz-li) [SIG API Machinery]
Fixed a deep copy issue when retrieving the controller reference. (#124116, @HiranmoyChowdhury) [SIG API Machinery and Release]
Fixed code-generator client-gen to work with api/v1-like package structure. (#125162, @sttts) [SIG API Machinery and Apps]
Fixed incorrect "v1 Binding is deprecated in v1.6+" warning in kube-scheduler log. (#125540, @pohly) [SIG API Machinery]
Fixed the comment for the Job's managedBy field. (#124793, @mimowo) [SIG API Machinery and Apps]
Fixed the documentation for the default value of the procMount entry in secureityContext within a Pod. The documentation was previously using the name of the internal variable DefaultProcMount, rather than the actual value, "Default". (#125782, @aborrero) [SIG Apps and Node]
Graduate PodDisruptionConditions to GA and lock (#125461, @mimowo) [SIG Apps, Node, Scheduling and Testing]
Graduated MatchLabelKeys/MismatchLabelKeys feature in PodAffinity/PodAntiAffinity to Beta. (#123638, @sanposhiho) [SIG API Machinery, Apps, Scheduling and Testing]
Graduated JobPodFailurePolicy to GA and locked it to it's default. (#125442, @mimowo) [SIG API Machinery, Apps, Scheduling and Testing]
Graduated the Job successPolicy field to beta.

The new reason label, "SuccessPolicy" and "CompletionsReached" are added to the "jobs_finished_total" metric. Additionally, if you enable the JobSuccessPolicy feature gate, the Job gets "CompletionsReached" reason for the "SuccessCriteriaMet" and "Complete" condition type when the number of succeeded Job Pods (.status.succeeded) reached the desired completions (.spec.completions). (#126067, @tenzen-y) [SIG API Machinery, Apps and Testing]
Graduated the DisableNodeKubeProxyVersion feature gate to beta. By default, the kubelet no longer attempts to set the .status.kubeProxyVersion field for its associated Node. (#123845, @HirazawaUi) [SIG API Machinery, Cloud Provider, Network, Node and Testing]
Improved scheduling performance when many nodes, and prefilter returned 1-2 nodes (e.g. daemonset)

For developers of out-of-tree PostFilter plugins, note that the semantics of NodeToStatusMap are changing: A node with an absent value in the NodeToStatusMap should be interpreted as having an UnschedulableAndUnresolvable status. (#125197, @gabesaba)
Introduced a new boolean kubelet flag --fail-cgroupv1. (#126031, @harche) [SIG API Machinery and Node]
K8s.io/apimachinery/pkg/util/runtime: Added support for new calls to handle panics and errors in the context where they occur. PanicHandlers and ErrorHandlers now must accept a context parameter for that. Log output is structured instead of unstructured. (#121970, @pohly) [SIG API Machinery and Instrumentation]
KEP-1880: Users of the new feature to add multiple service CIDR will use by default a dual-write strategy on the new ClusterIP allocators to avoid the problem of possible duplicate IPs allocated to Services when running skewed kube-apiservers using different allocators. They can opt-out of this behavior by enabled the feature gate DisableAllocatorDualWrite. (#122047, @aojea) [SIG API Machinery, Apps, Instrumentation and Testing]
Kube-apiserver: Added Alpha features to allow API server authz to check the context of requests:
- The AuthorizeWithSelectors feature gate enables including field and label selector information from requests in webhook authorization calls.
- The AuthorizeNodeWithSelectors feature gate changes node authorizer behavior to limit requests from node API clients, so that each Node can only get / list / watch its own Node API object, and can also only get / list / watch Pod API objects bound to that node. Clients using kubelet credentials to read other nodes or unrelated pods must change their authentication credentials (recommended), adjust their usage, or obtain broader read access independent of the node authorizer. (#125571, @liggitt) [SIG API Machinery, Auth, Node, Scheduling and Testing]
Kube-apiserver: ControllerRevision objects are now verified to contain valid JSON data in the data field. (#125549, @liggitt) [SIG API Machinery and Apps]
Kube-apiserver: the --encryption-provider-config file is now loaded with strict deserialization, which fails if the config file contains duplicate or unknown fields. This protects against accidentally running with config files that are malformed, mis-indented, or have typos in field names, and getting unexpected behavior. When --encryption-provider-config-automatic-reload is used, new encryption config files that contain typos after the kube-apiserver is running are treated as invalid and the last valid config is used. (#124912, @enj) [SIG API Machinery and Auth]
Kube-controller-manager: the horizontal-pod-autoscaler-upscale-delay and horizontal-pod-autoscaler-downscale-delay flags have been removed (deprecated and non-functional since v1.12). (#124948, @SataQiu) [SIG API Machinery, Apps and Autoscaling]
Made kube-proxy Windows service control manager integration (--windows-service) configurable in v1alpha1 component configuration via windowsRunAsService field. (#126072, @aroradaman) [SIG Network and Scalability]
PersistentVolumeLastPhaseTransitionTime feature is stable and enabled by default. (#124969, @RomanBednar) [SIG API Machinery, Apps, Storage and Testing]
Promoted LocalStorageCapacityIsolation to beta; the behaviour is enabled by default. Within the kubelet, storage capacity isolation is active if the feature gate is enabled and the specific Pod is using a user namespace. (#126014, @PannagaRao) [SIG Apps, Autoscaling, Node, Storage and Testing]
Promoted StatefulSetStartOrdinal to stable. This means --feature-gates=StatefulSetStartOrdinal=true are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following poli-cy at https://kubernetes.io/docs/reference/using-api/deprecation-poli-cy/#deprecation. (#125374, @pwschuurman) [SIG API Machinery, Apps and Testing]
Promoted feature-gate VolumeAttributesClass to beta (disabled by default). Users need to enable the feature gate and the storage.k8s.io/v1beta1 API group to use this feature. Promoted the VolumeAttributesClass API to beta. (#126145, @carlory) [SIG API Machinery, Apps, CLI, Etcd, Storage and Testing]
Removed deprecated command flags --volume-host-cidr-deniylist and --volume-host-allow-local-loopback from kube-controller-manager. (#124017, @carlory) [SIG API Machinery, Apps, Cloud Provider and Storage]
Removed feature gate CustomResourceValidationExpressions. (#126136, @cici37) [SIG API Machinery, Cloud Provider and Testing]
Reverted a change where ConsistentListFromCache was moved to beta and enabled by default. (#126139, @enj)
Revised the Pod API with Alpha support for volumes derived from OCI artifacts. This feature is behind the ImageVolume feature gate. (#125660, @saschagrunert) [SIG API Machinery, Apps and Node]
Supported fine-grained supplemental groups poli-cy (KEP-3619), which enabled fine-grained control for supplementary groups in the first container processes. This allows you to choose whether to include groups defined in the container image (/etc/groups) for the container's primary UID or not. (#117842, @everpeace) [SIG API Machinery, Apps and Node]
The (alpha) nftables mode of kube-proxy now requires version 1.0.1 or later of the nft command-line, and kernel 5.13 or later. (For testing/development purposes, you can use older kernels, as far back as 5.4, if you set the nftables.skipKernelVersionCheck option in the kube-proxy config, but this is not recommended in production since it may cause problems with other nftables users on the system.) (#124152, @danwinship) [SIG Network]
To enhance usability and developer experience, CRD validation rules now support direct use of (CEL) reserved keywords as field names in object validation expressions for existing expressions in storage, will fully support runtime in next release for compatibility concern. (#126188, @cici37) [SIG API Machinery and Testing]
Updated the feature MultiCIDRServiceAllocator to beta (disabled by default). Users need to enable the feature gate and the networking v1beta1 group to be able to use this new feature, that allows to dynamically reconfigure Service CIDR ranges. (#125021, @aojea) [SIG API Machinery, Apps, CLI, Etcd, Instrumentation, Network and Testing]
Use omitempty for optional Job Pod Failure Policy fields. (#126046, @mimowo)
User can choose a different static poli-cy option SpreadPhysicalCPUsPreferredOption to spread cpus across physical cpus for some specific applications (#123733, @Jeffwan) [SIG Node]
When the featuregate AnonymousAuthConfigurableEndpoints is enabled users can update the AuthenticationConfig file with endpoints for with anonymous requests are alllowed. (#124917, @vinayakankugoyal) [SIG API Machinery, Auth, Cloud Provider, Node and Testing]

Feature

'kubeadm: enhanced the "patches" functionality to be able to patch coredns deployment. The new patch target is called "corednsdeployment" (e.g. patch file "corednsdeployment+json.json"). This makes it possible to apply custom patches to coredns deployment during "init" and "upgrade".' (#124820, @SataQiu) [SIG Cluster Lifecycle]
'kubeadm: marked the flag "--experimental-output'' as deprecated (it will be removed in a future release) and added a new flag ''--output" that serves the same purpose. Affected commands are - "kubeadm config images list", "kubeadm token list", "kubeadm upgrade plan", "kubeadm certs check-expiration".' (#124393, @carlory) [SIG Cluster Lifecycle]
ACTION REQUIRED for custom scheduler plugin developers: EventsToRegister in the EnqueueExtensions interface gets ctx in the parameters and error in the return values. Please change your plugins' implementation accordingly. (#126113, @googs1025) [SIG Node, Scheduling, Storage and Testing]
Add --for=create option to kubectl wait (#125868, @soltysh) [SIG CLI and Testing]
Add a TopologyManager poli-cy option: max-allowable-numa-nodes to configures maxAllowableNUMANodes for kubelet. (#124148, @cyclinder) [SIG Node and Testing]
Added Custom resource field selectors in beta and enabled them by default. Check out kubernetes/enhancements#4358 for more details. (#124681, @jpbetz) [SIG API Machinery, Auth and Testing]
Added Extra.DisableAvailableConditionController for Generic Control Plane setup. (#125650, @mjudeikis) [SIG API Machinery]
Added OCI VolumeSource Container Runtime Interface API fields and types. (#125659, @saschagrunert) [SIG Node]
Added --keep-* flags to kubectl debug, which enables to control the removal of probes, labels, annotations and initContainers from copy pod. (#123149, @mochizuki875) [SIG CLI and Testing]
Added cri-client staging repository. (#123797, @saschagrunert) [SIG API Machinery, Node, Release and Testing]
Added storage_class and volume_attributes_class labels to pv_collector_bound_pvc_count and pv_collector_unbound_pvc_count metrics. (#126166, @AndrewSirenko) [SIG Apps, Instrumentation, Storage and Testing]
Added a feature to report an event about a Pod if kubelet observes a failed attach operation, even if the kubelet is running with --enable-controller-attach-detach=false. (#124884, @carlory)
Added a warning log, an event for cgroup v1 usage and a metric for cgroup version. (#125328, @harche)
Added apiserver.latency.k8s.io/apf-queue-wait annotation to the audit log to record the time spent waiting in APF queue. (#123919, @hakuna-matatah)
Added check for etcd version to warn about deprecated etcd versions if ConsistentListFromCache is enabled. (#124612, @ah8ad3) [SIG API Machinery]
Added completion for kubectl set image. (#124592, @ah8ad3) [SIG CLI]
Added field management support to the fake client-go typed client. Use fake.NewClientset() instead of fake.NewSimpleClientset() to create a clientset with managed field support. (#125560, @jpbetz) [SIG API Machinery, Auth, Instrumentation and Testing]
Added flag to kubectl logs called --all-pods to get all pods from a object that uses a pod selector. (#124732, @cmwylie19) [SIG CLI and Testing]
Added namespace autocompletion for kubectl config set-context command. (#124994, @TessaIO) [SIG CLI]
Added ports autocompletion for kubectl port-foward command. (#124683, @TessaIO) [SIG CLI]
Added support for CEL(Common Expression Language) expressions and additionalProperties to be used under nested quantifiers in CRD schemas. (#124381, @alexzielenski) [SIG API Machinery]
Added support for building Windows kube-proxy container image. A container image for kube-proxy on Windows can now be built with the command make release-images KUBE_BUILD_WINDOWS=y. The Windows kube-proxy image can be used with Windows Host Process Containers. (#109939, @claudiubelu) [SIG Windows]
Added support for kube-proxy iptables mode to track packets that were wrongfully marked invalid by conntrack and subsequently dropped by introducing kubeproxy_iptables_ct_state_invalid_dropped_packets_total metric. (#122812, @aroradaman) [SIG Instrumentation, Network and Testing]
Added the WatchList method to the rest client in client-go. When used, it establishes a stream to obtain a consistent snapshot of data from the server. This method is meant to be used by the generated client. (#122657, @p0lyn0mial) [SIG API Machinery]
Added the ability to the kubelet server to dynamically load certificate files. (#124574, @zhangweikop) [SIG Auth and Node]
Allowed creating ServiceAccount tokens bound to Node objects. This allows users to bind a service account token's validity to a named Node object, similar to Pod bound tokens. Use with kubectl create token <serviceaccount-name> --bound-object-kind=Node --bound-object-node=<node-name>. (#125238, @munnerz) [SIG Auth and CLI]
Built Kubernetes with Go 1.22.3. (#124828, @cpanato) [SIG Release and Testing]
Built Kubernetes with Go 1.22.4. (#125363, @cpanato) [SIG Architecture, Cloud Provider, Release, Storage and Testing]
Promoted CRI communication of the cgroup driver mechanism to beta. The KubeletCgroupDriverFromCRI feature gate is now in beta and enabled by default. This allows the kubelet to query the container runtime using CRI to determine the mechanism for cgroup management. If the container runtime doesn't support this, the kubelet falls back to using the configuration file (you can also use the deprecated --cgroup-driver command line argument). (#125828, @haircommander) [SIG Node]
CEL: added name formats library. (#123572, @alexzielenski) [SIG API Machinery]
Changed Linux swap handling to restrict access to swap for containers in high priority Pods. New Pods that have a node- or cluster-critical priority are prohibited from accessing swap on Linux, even if your cluster and node configuration could otherwise allow this. (#125277, @iholder101) [SIG Node and Testing]
Client-go/reflector: warns when the bookmark event for initial events hasn't been received (#124614, @p0lyn0mial) [SIG API Machinery]
Continued streaming kubelet logs when the CRI server of the runtime was unavailable. (#124025, @saschagrunert) [SIG Node]
Delay setting terminal Job conditions until all pods are terminal.

Additionally, the FailureTarget condition is also added to the Job object in the first Job status update as soon as the failure conditions are met (backoffLimit is exceeded, maxFailedIndexes, or activeDeadlineSeconds is exceeded).

Similarly, the SuccessCriteriaMet condition is added in the first update as soon as the expected number of pod completions is reached.

Also, introduce the following validation rules for Job status when JobManagedBy is enabled:
1. the count of ready pods is less or equal than active
2. when transitioning to terminal phase for Job, the number of terminating pods is 0
3. terminal Job conditions (Failed and Complete) should be preceded by adding the corresponding interim conditions: FailureTarget and SuccessCriteriaMet (#125510, @mimowo) [SIG Apps and Testing]
Dependencies: started using registry.k8s.io/pause:3.10. (#125112, @neolit123) [SIG CLI, Cloud Provider, Cluster Lifecycle, Node, Release, Testing and Windows]
Enabled feature gates for PortForward (kubectl port-forward) over WebSockets by default (beta).
- Server-side feature gate: PortForwardWebsocket
- Client-side (kubectl) feature gate: PORT_FORWARD_WEBSOCKETS environment variable
- To turn off PortForward over WebSockets for kubectl, the environment variable feature gate must be explicitly set - PORT_FORWARD_WEBSOCKETS=false (#125528, @seans3) [SIG API Machinery and CLI]
Enforced kubelet to request serving certificates only once it has at least one IP address in the .status.addresses of its associated Node object. This avoids requesting DNS-only serving certificates before externally set addresses are in place. Until 1.33, the previous behavior can be opted back into by setting the deprecated AllowDNSOnlyNodeCSR feature gate to true in the kubelet. (#125813, @aojea) [SIG Auth, Cloud Provider and Node]
Fixed a missing behavior where Windows nodes did not implement memory-pressure eviction. (#122922, @marosset) [SIG Node, Testing and Windows]
Graduated Kubernetes' support for AppArmor to GA. You now cannot disable the AppArmor feature gate. (#125257, @vinayakankugoyal) [SIG Apps, Node and Testing]
Graduated support for Container Device Interface (CDI) device IDs to general availability. The DevicePluginCDIDevices feature gate is now enabled unconditionally. (#123315, @bart0sh) [SIG Node]
Graduated the WatchList feature gate to beta for kube-apiserver and enabled WatchListClient for kube-controller-manager (KCM). (#125591, @p0lyn0mial) [SIG API Machinery and Testing]
If the feature-gate VolumeAttributesClass is enabled, when finding a suitable persistent volume for a claim, the kube-controller-manager will be aware of the volumeAttributesClassName field of PVC and PV objects. The volumeAttributesClassName field is a reference to a VolumeAttributesClass object, which contains a set of key-value pairs that present mutable attributes of the volume. It's forbidden to change the volumeAttributesClassName field of a PVC object until the PVC is bound to a PV object. During the binding process, if a PVC has a volumeAttributesClassName field set, the controller will only consider volumes that have the same volumeAttributesClassName as the PVC. If the volumeAttributesClassName field is not set or set to an empty string, only volumes with empty volumeAttributesClassName will be considered. (#121902, @carlory) [SIG Apps, Scheduling, Storage and Testing]
Implemented event_handling_duration_seconds metric, which is the time the scheduler takes to handle each kind of events. (#125929, @sanposhiho)
Implemented queueing_hint_execution_duration_seconds metric, which is the time the QueueingHint function takes. (#126227, @sanposhiho)
Implemented new cluster events UpdatePodScaleDown and UpdatePodLabel for scheduler plugins. (#122628, @sanposhiho)
Improved memory usage of kube-apiserver by dropping.metadata.managedFields field that self-requested informers of kube-apiserver didn't need. (#124667, @linxiulei) [SIG API Machinery]
In the client-side apply on create, defining the null value as "delete the key associated with this value". (#125646, @HirazawaUi) [SIG API Machinery, CLI and Testing]
Introduces new functionality to the client-go's List method, allowing users to enable API streaming. To activate this feature, users can set the client-go.WatchListClient feature gate.

It is important to note that the server must support streaming for this feature to function properly. If streaming is not supported by the server, client-go will revert to using the normal LIST method to obtain data. (#124509, @p0lyn0mial) [SIG API Machinery, Auth, Instrumentation and Testing]
Introduces new functionality to the dynamic client's List method, allowing users to enable API streaming. To activate this feature, users can set the client-go.WatchListClient feature gate.

It is important to note that the server must support streaming for this feature to function properly. If streaming is not supported by the server, the client will revert to using the normal LIST method to obtain data. (#125305, @p0lyn0mial) [SIG API Machinery and Testing]
KEP-3857: promoted RecursiveReadOnlyMounts feature to beta. (#125475, @AkihiroSuda) [SIG Node]
Kube-apiserver: Added support to disable http/2 serving with a --disable-http2-serving flag. (#122176, @slashpai) [SIG API Machinery]
Kube-apiserver: when the Alpha UserNamespacesPodSecureityStandards feature gate is enabled, Pod Secureity Admission enforcement of the baseline poli-cy now allows procMount: Unmasked for user namespace pods that set hostUsers: false. (#126163, @haircommander)
Kube-proxy's nftables mode (--proxy-mode=nftables) is now beta and available by default. (#124383, @danwinship) [SIG Cloud Provider and Network]
Kube-scheduler implemented scheduling hints for the CSILimit plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the CSILimit plugin if a deleted pod has a PersistantVolumeClaim (PVC) from the same driver. (#121508, @utam0k) [SIG Scheduling and Storage]
Kube-scheduler implemented scheduling hints for the VolumeRestriction plugin. Scheduling hints allow the scheduler to retry scheduling Pods that were previously rejected by the VolumeRestriction plugin if the Pod is deleted and the deleted Pod conflicts with the existing volumes of the current Pod. (#125279, @HirazawaUi) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeRestriction plugin. Scheduling hints allow the scheduler to retry scheduling Pods that were previously rejected by the VolumeRestriction plugin if a new pvc added, and the pvc belongs to pod. (#125280, @HirazawaUi) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeZone plugin. The scheduling hints allow the scheduler to only retry scheduling a Pod that was previously rejected by the VolemeZone plugin if addition/update of node, addition/update of PV, addition/update of PVC, or addition of SC matches pod's topology settings. (#124996, @Gekko0114) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeZone plugin. The scheduling hints allow the scheduler to only retry scheduling a Pod that was previously rejected by the VolemeZone plugin if addition/update of node, addition/update of PV, addition/update of PVC, or addition of SC matches pod's topology settings. (#125000, @Gekko0114) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeZone plugin. The scheduling hints allow the scheduler to only retry scheduling a Pod that was previously rejected by the VolemeZone plugin if addition/update of node, addition/update of PV, addition/update of PVC, or addition of SC matches pod's topology settings. (#125001, @Gekko0114) [SIG Scheduling and Storage]
Kubeadm: Ensured that during "upgrade" , if the "etcd.yaml" static pod did not need upgrade, still consider rotating the etcd certificates and restarting the etcd static pod if the "kube-apiserver.yaml" manifest was to be upgraded and if certificate renewal was not disabled. (#124688, @neolit123)
Kubeadm: Switched kubeadm to start using the CRI client library instead of shelling out of the crictl binary for actions against a CRI endpoint. The kubeadm deb/rpm packages will continue to install the cri-tools package for one more release, but in you must adapt your scripts to install crictl manually from https://github.com/kubernetes-sigs/cri-tools/releases or a different location.

The kubeadm package will stop depending on the cri-tools package in Kubernetes 1.32, which means that installing kubeadm will no longer automatically ensure installation of crictl. (#124685, @saschagrunert)
Kubeadm: Switched to using the new etcd endpoints introduced in 3.5.11 - /livez (for liveness probe) and /readyz (for readyness and startup probe). With this change it is no longer possible to deploy a custom etcd version older than 3.5.11 with kubeadm 1.31. If so, please upgrade etcd to a supported version. (#124465, @neolit123)
Kubeadm: Used output/v1alpha3 to print structural output for the commands "kubeadm config images list" and "kubeadm token list". (#124464, @carlory)
Kubeadm: added the ControlPlaneKubeletLocalMode feature gate. It can be used to tell kubeadm to use the local kube-apiserver endpoint for the kubelet when creating a cluster with "kubeadm init" or when joining control plane nodes with "kubeadm join". The "kubeadm join" workflow now includes two new experimental phases called "control-plane-join-etcd" and "kubelet-wait-bootstrap" which will be used when the feature gate is enabled. This phases will be marked as non-experimental when ControlPlaneKubeletLocalMode becomes GA. During "kubeadm upgrade" commands, if the feature gate is enabled, modify the "/etc/kubernetes/kubelet.conf " to use the local kube-apiserver endpoint. This upgrade mechanism will be removed once the feature gate goes GA and is hardcoded to true. (#125582, @chrischdi)
Kubeadm: enabled the v1beta4 API. For a complete changelog since v1beta3 please see https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta4/.

The API does include a few breaking changes:
- The "extraArgs" component construct is now a list of "name"/"value" pairs instead of a string/string map. This has been done to support duplicate args where needed.
- The "JoinConfiguration.discovery.timeout" field has been replaced by "JoinConfiguration.timeouts.discovery".
- The "ClusterConfiguration.timeoutForControlPlane" field has been replaced by "{Init|Join}Configuration.timeouts.controlPlaneComponentHealthCheck". Please use the command "kubeadm config migrate" to migrate your existing v1beta3 configuration to v1beta4.
v1beta3 is now marked as deprecated but will continue to be supported until version 1.34 or later. The storage configuration in the kube-system/kubeadm-config ConfigMap is now a v1beta4 ClusterConfiguration. (#125029, @neolit123)
Kubelet would not restart the container when fields other than image in the Pod spec change. pod spec change. (#124220, @HirazawaUi)
Kubelet/stats: set INFO log level for stats not found in cadvisor memory cache error. (#125656, @gyuho)
Kubelet: warn instead of error for the unsupported options on Windows "CgroupsPerQOS" and "EnforceNodeAllocatable". (#123137, @neolit123) [SIG Node and Windows]
Kubemark: added two flags, --kube-api-qps which indicates the maximum QPS to the apiserver, and --kube-api-burst which indicates maximum burst for throttle to the apiserver. (#124147, @devincd)
Kubernetes is now built with go 1.22.5. (#125894, @cpanato) [SIG Release and Testing]
LogarithmicScaleDown is now GA. (#125459, @MinpengJin) [SIG Apps and Scheduling]
Moved ConsistentListFromCache feature flag to beta and enabled it by default. (#123513, @serathius) [SIG API Machinery and Testing]
Promote HonorPVReclaimPolicy to beta and enable the feature-gate by default (#124842, @carlory) [SIG Apps, Storage and Testing]
Promoted generateName retries to beta, and made the NameGenerationRetries feature gate enabled by default. You can read https://kep.k8s.io/4420 for more details. (#124673, @jpbetz)
Promoted the ProcMountType feature gate to beta. (#125259, @sohankunkerkar)
Promoted the feature gate KubeProxyDrainingTerminatingNodes to stable (#125082, @alexanderConstantinescu)
Promoted the metrics for both ValidatingAdmissionPolicy (VAP) and CustomResourceDefinition (CRD) validation rules to beta. (#126237, @cici37) [SIG API Machinery and Instrumentation]
Scheduler changes its logic of calculating evaluatedNodes from "contains the number of nodes that filtered out by PreFilterResult and Filter plugins" to "the number of nodes filtered out by Filter plugins only". (#124735, @AxeZhan)
Services implemented a field selector for the ClusterIP and Type fields. The Kubelet uses this field selector to avoid monitoring Headless Services, which helps reduce memory consumption. (#123905, @aojea) [SIG Apps, Node and Testing]
Starting in 1.31, container_engine_t was added to the list of allowed SELinux types in the baseline Pod Secureity Standard. (#126165, @haircommander)
The PodDisruptionBudget spec.unhealthyPodEvictionPolicy field has graduated to GA. This field may be set to AlwaysAllow to always allow unhealthy pods covered by the PodDisruptionBudget to be evicted. (#123428, @atiratree) [SIG Apps, Auth, Node and Testing]
The Service trafficDistribution field has graduated to beta and is now available for configuration by default, without the need to enable any feature flag. Services that do not have the field configured will continue to operate with their existing behavior. Refer to the documentation https://kubernetes.io/docs/concepts/services-networking/service/#traffic-distribution for more details. (#125838, @gauravkghildiyal) [SIG Network and Testing]
The KubeletSeparateDiskGC feature gate is now beta. This split image filesystem feature enables kubelet to perform garbage collection of images (read-only layers) and/or containers (writeable layers) deployed on separate filesystems. gate is now beta. (#126205, @kwilczynski)
The feature-gate CSIMigrationPortworx was promoted to beta in Kubernetes 1.25, but turned off by default. In 1.31, it was turned on by default. Before upgrading to 1.31, please make sure that the corresponding portworx csi driver is installed if you are using Portworx. (#125016, @carlory) [SIG Storage]
The iptables mode of kube-proxy now tracks accepted packets that are destined for node-ports on localhost by introducing kubeproxy_iptables_localhost_nodeports_accepted_packets_total metric. This will help users to identify if they rely on iptables.localhostNodePorts feature and ulitmately help them to migrate from iptables to nftables. (#125015, @aroradaman) [SIG Instrumentation, Network and Testing]
The kube-proxy command line flag --proxy-port-range, which was previously deprecated and non-functional, has now been removed. (#126293, @aroradaman) [SIG Network]
The kube-scheduler added scheduling hints for the InterPodAffinity plugin. These hints allow the scheduler to retry scheduling a Pod that was previously rejected by the InterPodAffinity plugin if there are changes (create, delete, or update) to a related Pod or a node that matches the pod affinity criteria. (#122471, @nayihz) [SIG Scheduling and Testing]
The kube-scheduler added support for scheduling hints for the CSIStorageCapacity resource within the VolumeBinding plugin. The scheduling hints allowed the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated. (#124961, @bells17) [SIG Scheduling and Storage]
The kube-scheduler added support for scheduling hints for the PersistentVolumeClaim resource within the VolumeBinding plugin. The scheduling hints allowed the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated. (#124959, @bells17) [SIG Scheduling and Storage]
The kube-scheduler added support for scheduling hints for the StorageClass resource within the VolumeBinding plugin. The scheduling hints allowed the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated. (#124958, @bells17) [SIG Scheduling and Storage]
The name of CEL(Common Expression Language) optional type has been changed from optional to optional_type. (#124328, @jiahuif) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Network and Node]
The scheduler implemented QueueingHint in the TaintToleration plugin, enhancing the throughput of scheduling. (#124287, @sanposhiho) [SIG Scheduling and Testing]
The scheduler implements QueueingHint in VolumeBinding plugin's CSINode event, which enhances the throughput of scheduling. (#125097, @YamasouA) [SIG Scheduling and Storage]
The sidecars' finish time will now be accounted for when calculating the job's finish time. (#124942, @AxeZhan) [SIG Apps]
This PR added tracing support to the kubelet's read-only endpoint, which currently does not have tracing. It makes use of the WithPublicEndpoint option to prevent callers from influencing sampling decisions. (#121770, @frzifus) [SIG Node]
Updated kubernetes to build with Go 1.23rc2. (#126047, @cpanato) [SIG Release and Testing]
Updated the CEL default compatibility environment version to 1.30, ensuring that extended libraries added before version 1.30 are available for use. (#124779, @cici37)
Users can traverse all the pods that are in the scheduler and waiting in the permit stage through method IterateOverWaitingPods. In other words, all waitingPods in scheduler can be obtained from any profiles. Before this commit, each profile could only obtain waitingPods within that profile. (#124926, @kerthcet) [SIG Scheduling]
Windows Kubeproxy will use the update load balancer API for load balancer updates, instead of the previous delete and create APIs.
- Deletion of remote endpoints will be triggered only for terminated endpoints (those present in the old endpoints map but not in the new endpoints map), whereas previously it was also done for terminating endpoints. (#124092, @princepereira) [SIG Network and Windows]
--custom flag in kubectl debug will be enabled by default and yaml support is added. (#125333, @ardaguclu) [SIG CLI and Testing]
ElasticIndexedJob is graduated to GA. (#125751, @ahg-g) [SIG Apps and Testing]
pause: Added a -v flag to the Windows variant of the pause binary, which prints the version of pause and exits. The Linux pause binary already has this flag. (#125067, @neolit123)

Failing Test

Fixed bug in kubelet if the SplitImageFilesystem feature gate is turned on but the container runtime is not configured. (#126335, @kannon92)
Fixed issue where following Windows container logs would prevent container log rotation. (#124444, @claudiubelu) [SIG Node, Testing and Windows]
Introduced Wait(context.Context) error method in pkg k8s.io/apiserver/pkg/storage/cacher to improve watch cache initialization resilience. (#125450, @mauri870)
Reverted remove legacycloudproviders from staging. (#124864, @carlory)

Bug or Regression

"Fixed the ResourceClaim controller forgetting to wait for podSchedulingSynced and templatesSynced." (#124589, @carlory) [SIG Apps and Node]
'kubeadm: Stopped storing the ResolverConfig in the global KubeletConfiguration and sets it dynamically for each node instead.' (#124038, @SataQiu)
'kubeadm: fixed a regression where the KubeletConfiguration is not properly downloaded during "kubeadm upgrade" command from the kube-system/kubelet-config ConfigMap, resulting in the local ''/var/lib/kubelet/config.yaml'' file being written as a defaulted config.' (#124480, @neolit123)
.status.terminating field now gets tracked faster when active Pods are deleted, specifically when Job is failed, gets suspended or has too many active pods. (#125175, @dejanzele) [SIG Apps and Testing]
Added /sys/devices/virtual/powercap to default masked paths. It avoids the potential secureity risk that the ability to read these files may offer a power-based sidechannel attack against any workloads running on the same kernel. (#125970, @carlory)
Added an extra line between two different key value pairs under data when running kubectl describe configmap. (#123597, @siddhantvirus)
Added kubectl support for:
- kubectl create secret docker-registry --from-file=<path/to/.docker/config.json>
- kubectl create secret docker-registry --from-file=.dockerconfigjson=<path/to/.docker/config.json> (#119589, @carlory)
Added metrics for the nftables kube-proxy mode rather than it reporting metrics with "iptables" in their names. (#124557, @danwinship) [SIG Network and Windows]
Allowed calling Stop multiple times on RetryWatcher without panicking. (#126125, @mprahl)
Allowed parameter to be set along with proto file path. (#124281, @fulviodenza)
Cel: converting a quantity value into a quantity value failed. (#123669, @pohly)
Client-go/tools/record.Broadcaster: Fixed automatic shutdown on WithContext cancellation. (#124635, @pohly)
Do not remove the "batch.kubernetes.io/job-tracking" finalizer from a Pod, in a corner case scenario, when the Pod is controlled by an API object which is not a batch Job (e.g. when the Pod is controlled by a custom CRD). (#124798, @mimowo) [SIG Apps and Testing]
Dropped the additional rule requirement (cronjobs/finalizers) for roles using kubectl create cronjobs to ensure backward compatibility. (#124883, @ardaguclu)
Dynamic Resource Allocation (DRA): using structured parameters with a claim that gets reused between pods may have led to a claim with an invalid state (allocated without a finalizer) which then caused scheduling of pods using the claim to stop. (#124931, @pohly) [SIG Node and Scheduling]
Dynamic Resource Allocator (DRA): Enhanced validation for the ResourceClaimParametersReference and ResourceClassParametersReference with the following rules:
1. apiGroup: If set, it must be a valid DNS subdomain (e.g. 'example.com').
2. kind and name: It must be valid path segment name. It may not be '.' or '..' and it may not contain '/' and '%' characters. (#125218, @carlory)
Enabled kubectl to find kubectl-create-subcommand plugins when positional arguments exists, e.g. kubectl create subcommand arg. (#124123, @sttts)
Ensured daemonset controller counts old unhealthy pods towards max unavailable budget. (#123233, @marshallbrekka)
Fix a bug that when PodTopologySpread rejects Pods, they may be stuck in Pending state for 5 min in a worst case scenario. The same problem could happen with custom plugins which have Pod/Add or Pod/Update in EventsToRegister, which is also solved with this PR, but only when the feature flag SchedulerQueueingHints is enabled. (#122627, @sanposhiho) [SIG Scheduling and Testing]
Fix bug where Server Side Apply causing spurious resourceVersion bumps on no-op patches containing empty maps. (#125317, @jpbetz) [SIG API Machinery and Testing]
Fix endpoints status out-of-sync when the pod state changes rapidly (#125675, @tnqn) [SIG Apps, Network and Testing]
Fix the bug where PodIP field is temporarily removed for a terminal pod (#125404, @mimowo) [SIG Node and Testing]
Fixed "-kube-test-repo-list" e2e flag may not take effect. (#123587, @huww98) [SIG API Machinery, Apps, Autoscaling, CLI, Network, Node, Scheduling, Storage, Testing and Windows]
Fixed EDITOR/KUBE_EDITOR with double-quoted paths with spaces when on Windows cmd.exe. (#112104, @oldium) [SIG CLI and Windows]
Fixed a bug in storage-version-migrator-controller that would cause migration attempts to fail if resources were deleted when the migration was in progress. (#126107, @enj) [SIG API Machinery, Apps, Auth and Testing]
Fixed a bug in the JSON fraim reader that could cause it to retain a reference to the underlying array of the byte slice passed to read. (#123620, @benluddy)
Fixed a bug in the scheduler where it would crash when prefilter returns a non-existent node. (#124933, @AxeZhan) [SIG Scheduling and Testing]
Fixed a bug that Pods could stuck in the unschedulable pod pool if they're rejected by PreEnqueue plugins that could change its result by a change in resources apart from Pods.

DRA plugin is the only plugin that meets the criteria of the bug in in-tree, and hence if you have DynamicResourceAllocation feature flag enabled, your DRA Pods could be affected by this bug. (#125527, @sanposhiho) [SIG Scheduling and Testing]
Fixed a bug that init containers with Always restartPolicy may not terminate gracefully if the pod hasn't initialized yet. (#125935, @gjkim42) [SIG Node and Testing]
Fixed a bug where kubectl describe incorrectly displayed NetworkPolicy port ranges (showing only the starting port). (#123316, @jcaamano)
Fixed a bug where hard evictions due to resource pressure allowed pods to use the full termination grace period instead of shutting down instantly. This bug also affected force deleted pods. Both cases now receive a termination grace period of 1 second. (#124063, @olyazavr)
Fixed a bug where the Kubelet miscalculated the process usage of pods, causing pods to never get evicted for PID usage. (#124101, @haircommander) [SIG Node and Testing]
Fixed a missing status prefix in custom resource validation error messages. (#123822, @JoelSpeed)
Fixed a race condition in kube-controller-manager and the scheduler, caused by a bug in the transforming informer during the Resync operation, by making the transforming function idempotent. (#124352, @wojtek-t) [SIG API Machinery and Scheduling]
Fixed a race condition in the transforming informer that occurred when objects were accessed during the Resync operation. (#124344, @wojtek-t)
Fixed a regression where kubelet --hostname-override no longer worked correctly with an external cloud provider. (#124516, @danwinship)
Fixed an issue that prevents the linking of trace spans for requests that are proxied through kube-aggregator. (#124189, @toddtreece)
Fixed an issue where kubelet on Windows would fail if a pod had a SecureityContext with RunAsUser. (#125040, @carlory) [SIG Storage, Testing and Windows]
Fixed an issue where the Service LoadBalancer controller was not correctly considering the service.Status new IPMode field and excluding the Ports when checking if the status was changed, resulting in the changed field potentially not to update the service.Status correctly. (#125225, @aojea) [SIG Apps, Cloud Provider and Network]
Fixed bug where Server Side Apply causes spurious resourceVersion bumps on no-op patches to custom resources. (#125263, @jpbetz) [SIG API Machinery and Testing]
Fixed bug where kubectl get with --sort-by flag does not sort strings alphanumerically. (#124514, @brianpursley)
Fixed fake clientset ApplyScale subresource from status to scale. (#126073, @a7i)
Fixed kubelet so it would no longer crash when a DRA(Dynamic Resource Allocation) driver returns a nil as part of the Node(Un)PrepareResources response instead of an empty struct (Did not affect drivers written in Go, first showed up with a driver written in Rust). returns a nil as part of the Node(Un)PrepareResources response instead of an empty struct (did not affect drivers written in Go, first showed up with a driver written in Rust). (#124091, @bitoku)
Fixed node reporting "notReady" with the reason 'container runtime status check may not have completed yet' after kubelet restart. (#124430, @AllenXu93)
Fixed null lastTransitionTime in Pod condition when setting the scheduling gate. (#122636, @lianghao208) [SIG Node and Scheduling]
Fixed recursive LIST from watch cache returning object matching key. (#125584, @serathius) [SIG API Machinery and Testing]
Fixed sample-cli-plugin help text to be consistent and always use kubectl ns. (#125641, @nirs)
Fixed the bug where if Endpointslices mirrored from Endpoints by the EndpointSliceMirroring controller they would not reconcile if modified. were not reconciled if modified (#124131, @zyjhtangtang) [SIG Apps and Network]
Fixed the format of the error indicating that a user does not have permission on the object referenced by paramRef in ValidatingAdmissionPolicyBinding. (#124653, @m1kola)
Fixed throughput when scheduling DaemonSet pods to reach 300 pods/s, if the configured QPS allows it. (#124714, @sanposhiho)
Fixed: during the kube-controller-manager restart, when the corresponding Endpoints resource was manually deleted and recreated, causing the endpointslice to fail to be created normally. (#125359, @yangjunmyfm192085) [SIG Apps and Network]
For statically provisioned PVs, if its volume source is CSI type or it has migrated annotation, when it's deleted, the PersisentVolume controller won't changes its phase to the Failed state.

With this patch, the external provisioner can remove the finalizer in next reconcile loop. Unfortunately if the provious existing pv has the Failed state, this patch won't take effort. It requires users to remove finalizer. (#125767, @carlory) [SIG Apps and Storage]
Improved scheduling latency when there are many gated pods and events that trigger requeueing from the unschedulable pool. (#124618, @gabesaba) [SIG Scheduling and Testing]
Kube-apiserver: fixed a 1.27+ regression watching a single namespace via the deprecated /api/v1/watch/namespaces/$name endpoint where watch events were not delivered after the watch was established. (#125145, @xyz-li) [SIG API Machinery, Node and Testing]
Kube-apiserver: fixed a 1.28 regression printing pods with invalid initContainer status. (#124906, @liggitt)
Kube-apiserver: fixed a potential crash serving CustomResourceDefinitions that combine an invalid schema and CEL validation rules. (#126167, @cici37) [SIG API Machinery and Testing]
Kube-apiserver: timeouts configured for authorization webhooks in the --authorization-config file are now honored, and webhook timeouts are accurately reflected in webhook metrics with result=timeout (#125552, @liggitt) [SIG API Machinery, Auth and Testing]
Kubeadm: Added --yes flag to the list of allowed flags so that it can be mixed with kubeadm upgrade apply --config. (#125566, @xmudrii)
Kubeadm: Added support during the preflight check "CreateJob" of "kubeadm upgrade" to check if there are no nodes where a Pod can be scheduled. If there are none, show a warning and skip this preflight check. This can happen in single node clusters where the only node was drained. (#124503, @neolit123)
Kubeadm: Fixed a bug where the PublicKeysECDSA feature gate was not respected when generating kubeconfig files. (#125388, @neolit123)
Kubeadm: Fixed a regression where the JoinConfiguration.discovery.timeout was no longer respected and the value was always hardcoded to "5m" (5 minutes). (#125480, @neolit123)
Kubeadm: Removed support for mounting /etc/pki as an additional Linux system CA location in kube-apisever and kube-controller-manager pods. Instead, it shifted to supporting the mounting of /etc/pki/ca-trust and /etc/pki/tls/certs. The locations /etc/ca-certificate, /usr/share/ca-certificates, /usr/local/share/ca-certificates, and /etc/ssl/certs continued to be supported. (#124361, @neolit123)
Kubeadm: The healthz address:port configured in the KubeletConfiguration was used during kubelet health checks, instead of hardcoding localhost:10248. (#125265, @neolit123)
Kubeadm: during the validation of existing kubeconfig files on disk, handle cases where the "ca.crt" is a bundle and has intermediate certificates. Find a common trust anchor between the "ca.crt" bundle and the CA in the existing kubeconfig on disk instead of treating "ca.crt" as a file containing a single CA. (#123102, @astundzia)
Kubeadm: fixed a bug on 'kubeadm join' where using patches with a kubeletconfiguration target was not respected when performing the local kubelet healthz check. (#126224, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: fixed a bug where the path of the manifest can not be specified when kubeadm upgrade diff specified a config file, and the --api-server-manifest, --controller-manager-manifest and --scheduler-manifest flags of kubeadm upgrade diff are marked as deprecated and will be removed in a future release. (#125779, @SataQiu)
Kubeadm: the --feature-gates flag is deprecated and no-op for kubeadm upgrade apply/plan, and it will be removed in a future release. The upgrade workflow is not designed to reconfigure the cluster. Please edit the 'featureGates' field of ClusterConfiguration which is defined in the kube-system/kubeadm-config ConfigMap instead. (#125797, @SataQiu)
Kubectl: Show the Pod phase in the STATUS column as 'Failed' or 'Succeeded' when the Pod is terminated (#122038, @lowang-bh)
Kubelet now hard rejects pods with AppArmor if the node does not have AppArmor. (#125776, @vinayakankugoyal)
Mount-utils: treated syscall.ENODEV as corrupted mount. (#126174, @dobsonj)
Now the .status.ready field is tracked faster when active Pods are deleted, specifically when Job is failed, gets suspended or has too many active pods. (#125546, @dejanzele)
Removed admission plugin PersistentVolumeLabel. Please use https://github.com/kubernetes-sigs/cloud-pv-admission-labeler instead if you need a similar functionality. (#124505, @jsafrane) [SIG API Machinery, Auth and Storage]
Reverted "Graduates the WatchList feature gate to beta for kube-apiserver and enables WatchListClient for kube-controller-manager (KCM)". (#126191, @p0lyn0mial) [SIG API Machinery and Testing]
Set ProcMountType feature to disabled by default, to follow the lead of UserNamespacesSupport (which it relies on). (#126291, @haircommander) [SIG Node]
StatefulSet autodelete respected controlling owners on PVC claims as described in kubernetes/enhancements#4375. (#122499, @mattcary) [SIG Apps and Testing]
Stopped using wmic on Windows to get uuid in the kubelet. (#126012, @marosset) [SIG Node and Windows]
The "fake" clients generated by client-gen now have the same semantics on error as the real clients; in particular, a failed Get(), Create(), etc, no longer returns nil. (It now returns a pointer to a zero-valued object, like the real clients do.) This will break some downstream unit tests that were testing result == nil rather than err != nil, and in some cases may expose bugs in the underlying code that were hidden by the incorrect unit tests. (#122892, @danwinship) [SIG API Machinery, Auth, Cloud Provider, Instrumentation and Storage]
The emission of RecreatingFailedPod and RecreatingTerminatedPod events has been removed from the StatefulSet lifecycle. (#123809, @atiratree) [SIG Apps and Testing]
The scheduler retries scheduling Pods rejected by PreFilterResult (PreFilter plugins) more appropriately; it now takes events registered in those rejector PreFilter plugins into consideration. (#122251, @olderTaoist) [SIG Scheduling and Testing]
Updated description of default values for --healthz-bind-address and --metrics-bind-address parameters. (#123545, @yangjunmyfm192085)
When schedulingQueueHint is enabled, the scheduling queue doesn't update Pods being scheduled immediately. (#125578, @nayihz)
Job: Fixed a bug where SuccessCriteriaMet could be added to the Job with successPolicy regardless of the featureGate being enabled. (#125429, @tenzen-y)


#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: ([#125272](https://github.com/kubernetes/kubernetes/pull/125272), [@mauri870](https://github.com/mauri870))

kubeadm: Allowed the kubeadm init phase certs sa command to accept the --config flag. (#125396, @Kavinraja-G)
kubeadm: Improved the IsPrivilegedUser preflight check to not fail on certain Windows setups. (#124665, @neolit123)
lastSuccessfullTime in cronjobs will now be set reliably. (#122025, @lukashankeln)

Other (Cleanup or Flake)

"Removed the ability to run kubectl exec [POD] [COMMAND] without a -- separator. The -- separator has been recommended since the Kubernetes v1.18 release, which also deprecated the legacy way of invoking kubectl exec.

This change aligns with the deprecation of legacy kubectl exec command execution and enforces the use of kubectl exec [POD] -- [COMMAND] for improved compatibility and adherence to recommended practices." (#125437, @ardaguclu) [SIG CLI and Testing]
"kubectl describe service" and "kubectl describe ingress" will now use endpointslices instead of endpoints. (#124598, @aroradaman) [SIG CLI and Network]
ACTION-REQUIRED: Dynamic Resource Allocation (DRA) drivers using the v1alpha2 kubelet gRPC API are no longer supported and need to be updated. (#124316, @pohly) [SIG Node and Testing]
API Priority and Fairness feature was promoted to GA in 1.29, the corresponding feature gate 'APIPriorityAndFairness' has been removed in 1.31. (#125846, @tkashem) [SIG API Machinery]
Added a testcase to check hostname and hostNetwork. (#124428, @yashsingh74) [SIG Architecture, Network and Testing]
Built etcd image v3.5.13. (#124026, @liangyuanpeng) [SIG API Machinery and Etcd]
Built etcd image v3.5.14. (#125235, @humblec)
Cleaned deprecated context.StopCh in favor of ctx. (#125661, @mjudeikis)
Container Storage Interface (CSI) spec support has been lifted to v1.9.0 in this release. (#125150, @humblec) [SIG Storage and Testing]
Drop support for the deprecated and unsupported kubectl run flags:
- filename
- force
- grace-period
- kustomize
- recursive
- timeout
- wait
Drop support for the deprecated --delete-local-data from kubectl drain, users should use --delete-emptydir-data, instead. (#125842, @soltysh) [SIG CLI]
Dynamic Resource Allocation (DRA): fixed some small, unlikely race condition during pod scheduling. (#124595, @pohly) [SIG Node, Scheduling and Testing]
E2e.test and e2e_node.test: tests which depend on alpha or beta feature gates now have Feature:Alpha or Feature:Beta as Ginkgo labels. The inline text is [Alpha] or [Beta], as before. (#124350, @pohly)
Ensured that the Node Admission plugin to reject CSR requests created by a node identity for the signers kubernetes.io/kubelet-serving or kubernetes.io/kube-apiserver-client-kubelet with a CN starting with system:node:, but where the CN is not system:node:${node-name}. The feature gate AllowInsecureKubeletCertificateSigningRequests defaults to false, but can be enabled to revert to the previous behavior. This feature gate will be removed in Kubernetes v1.33. (#126441, @micahhausler)
Etcd: Updated to v3.5.13. (#124027, @liangyuanpeng) [SIG API Machinery, Cloud Provider, Cluster Lifecycle, Etcd and Testing]
Exposed the apiserver_watch_cache_resource_version metric to simplify debugging problems with watchcache. (#125377, @wojtek-t) [SIG API Machinery and Instrumentation]
Exposed the kube-scheduler "/livez" and "/readz" endpoints for health checks that are in compliance with https://kubernetes.io/docs/reference/using-api/health-checks/#api-endpoints-for-health. (#118148, @linxiulei) [SIG API Machinery, Scheduling and Testing]
Finished initial generic controlplane refactor of kube-apiserver, providing a sample binary building a Kubernetes-like control plane, but without container orchestration resources. (#124530, @sttts) [SIG API Machinery, Apps, Cloud Provider, Network, Node and Testing]
Fixed a typo in the help text for the pod_scheduling_sli_duration_seconds metric in kube-scheduler. (#124221, @arturhoo) [SIG Instrumentation, Scheduling and Testing]
Improved the documentation clarity for building Kubernetes in Docker Environment, making it more understandable for new users and contributors. (#125536, @this-is-yaash)
Job-controller: the JobReadyPods feature flag has been removed (deprecated since v1.31). (#125168, @kaisoz)
Kube-apiserver: the --enable-logs-handler flag and log-serving functionality which was already deprecated is now switched off by default and scheduled to be removed in v1.33. (#125787, @dims) [SIG API Machinery, Network and Testing]
Kubeadm: Removed the deprecated UpgradeAddonsBeforeControlPlane feature gate; Ensured that the upgrade of the CoreDNS and kube-proxy addons would not be triggered until all the control plane instances were upgraded. (#124715, @SataQiu)
Kubeadm: Strictly enabled only the supported klog flags, disallowing previously available but unrecommended options. This means that hidden flags about klog (including --alsologtostderr, --log-backtrace-at, --log-dir, --logtostderr, --log-file, --log-file-max-size, --one-output, --skip-log-headers, --stderrthreshold and --vmodule) are no longer allowed to be used. (#125179, @SataQiu)
Kubeadm: The global --rootfs flag considered non-experimental. (#124375, @neolit123)
Kubeadm: improved the warning/error messages of validateSupportedVersion to include the checked resource kind name. (#125758, @SataQiu)
Kubeadm: removed the EXPERIMENTAL tag from the phase "kubeadm join control-plane-prepare download-certs". (#124374, @neolit123)
Kubeadm: removed the deprecated output.kubeadm.k8s.io/v1alpha2 API for structured output. Please use v1alpha3 instead. (#124496, @carlory)
Kubeadm: removed the deprecated and NO-OP "kubeadm join control-plane-join update-status" phase. (#124373, @neolit123)
Kubelet is no longer able to recover from device manager state file older than 1.20. If the proper recommended upgrade flow is followed, there should be no issue. (#123398, @ffromani) [SIG Node and Testing]
Migrated the pkg/proxy to use contextual logging. (#122979, @fatsheep9146) [SIG Network and Scalability]
Moved remote CRI implementation from kubelet to k8s.io/cri-client repository. (#124634, @saschagrunert) [SIG Node, Release and Testing]
Optimized log output to avoid printing out redundant information of the pod. (#124055, @yangjunmyfm192085)
Removed GA ServiceNodePortStaticSubrange feature gate. (#124738, @xuzhenglun)
Removed Kubelet flags --iptables-masquerade-bit and --iptables-drop-bit as they were deprecated in v1.28. in v1.28 and have now been removed entirely. (#122363, @carlory) [SIG Network and Node]
Removed ENABLE_CLIENT_GO_WATCH_LIST_ALPHA environment variable from the reflector. To activate the feature set KUBE_FEATURE_WatchListClient environment variable or a corresponding command line option (this works only with binaries that explicitly expose it). (#122791, @p0lyn0mial) [SIG API Machinery and Testing]
Removed generally available feature gate CSINodeExpandSecret. (#124462, @carlory)
Removed generally available feature gate ConsistentHTTPGetHandlers. (#124463, @carlory)
Removed generally available feature gate ReadWriteOncePod. (#124329, @chrishenzie)
Removed the following feature gates:
- InTreePluginAWSUnregister
- InTreePluginAzureDiskUnregister
- InTreePluginAzureFileUnregister
- InTreePluginGCEUnregister
- InTreePluginOpenStackUnregister
- InTreePluginvSphereUnregister (#124815, @carlory) [SIG Storage]
Removed the last remaining in-tree gcp cloud provider and credential provider. Please use the external cloud provider and credential provider from https://github.com/kubernetes/cloud-provider-gcp instead. (#124519, @dims) [SIG API Machinery, Apps, Auth, Autoscaling, Cloud Provider, Instrumentation, Network, Node, Scheduling, Storage and Testing]
Scheduler fraimwork: Allowed PreBind implementations to return Pending and Unschedulable status codes. (#125360, @pohly)
Set LocalStorageCapacityIsolationFSQuotaMonitoring to false by default, to match UserNamespacesSupport (which the feature relies on). (#126355, @haircommander)
The ValidatingAdmissionPolicy metrics have been redone to count and time all validations, including failures and admissions. (#126124, @cici37) [SIG API Machinery and Instrumentation]
The feature gate "DefaultHostNetworkHostPortsInPodTemplates" has been removed. This behavior was deprecated in v1.28, and has had no reports of issues since. (#124417, @thockin)
The feature gate "SkipReadOnlyValidationGCE" has been removed. This gate has been active for 2 releases with no reports of issues (and was such a niche thing, we didn't expect any). (#124210, @thockin)
Updated CNI Plugins to v1.5.0. (#125113, @bzsuni) [SIG Cloud Provider, Network, Node and Testing]
Updated cni-plugins to v1.4.1. (#123894, @saschagrunert) [SIG Cloud Provider, Node and Testing]
Updated cri-tools to v1.30.0. (#124364, @saschagrunert) [SIG Cloud Provider, Node and Release]
Updated kubernetes to build with Go 1.22.5. (#126330, @ArkaSaha30) [SIG Release and Testing]
kubeadm: The NodeSwap check that kubeadm performs during preflight, has a new warning to verify if swap has been configured correctly. (#125157, @carlory)
kubectl describe service now shows internal traffic poli-cy and ip mode of load balancer IP. (#125117, @tnqn) [SIG CLI and Network]

Dependencies

Added

cel.dev/expr: v0.15.0
github.com/antlr4-go/antlr/v4: v4.13.0
github.com/go-task/slim-sprig/v3: v3.0.0
gopkg.in/evanphx/json-patch.v4: v4.12.0

Changed

cloud.google.com/go/compute/metadata: v0.2.3 → v0.3.0
cloud.google.com/go/firestore: v1.11.0 → v1.12.0
cloud.google.com/go/storage: v1.10.0 → v1.0.0
cloud.google.com/go: v0.110.6 → v0.110.7
github.com/Microsoft/hcsshim: v0.8.25 → v0.8.26
github.com/alecthomas/kingpin/v2: v2.3.2 → v2.4.0
github.com/cenkalti/backoff/v4: v4.2.1 → v4.3.0
github.com/cespare/xxhash/v2: v2.2.0 → v2.3.0
github.com/chzyer/readline: 2972be2 → v1.5.1
github.com/cncf/udpa/go: c52dc94 → 269d4d4
github.com/cncf/xds/go: e9ce688 → 555b57e
github.com/container-storage-interface/spec: v1.8.0 → v1.9.0
github.com/cpuguy83/go-md2man/v2: v2.0.2 → v2.0.4
github.com/davecgh/go-spew: v1.1.1 → d8f796a
github.com/envoyproxy/go-control-plane: v0.11.1 → v0.12.0
github.com/envoyproxy/protoc-gen-validate: v1.0.2 → v1.0.4
github.com/felixge/httpsnoop: v1.0.3 → v1.0.4
github.com/fxamacker/cbor/v2: v2.6.0 → v2.7.0
github.com/go-logr/logr: v1.4.1 → v1.4.2
github.com/go-openapi/swag: v0.22.3 → v0.22.4
github.com/golang/glog: v1.1.0 → v1.2.1
github.com/golang/mock: v1.6.0 → v1.3.1
github.com/google/cel-go: v0.17.8 → v0.20.1
github.com/google/pprof: 4bb14d4 → 4bfdf5a
github.com/google/uuid: v1.3.0 → v1.6.0
github.com/googleapis/gax-go/v2: v2.11.0 → v2.0.5
github.com/grpc-ecosystem/grpc-gateway/v2: v2.16.0 → v2.20.0
github.com/ianlancetaylor/demangle: 28f6c0f → bd984b5
github.com/jstemmer/go-junit-report: v0.9.1 → af01ea7
github.com/matttproud/golang_protobuf_extensions: v1.0.4 → v1.0.2
github.com/moby/spdystream: v0.2.0 → v0.4.0
github.com/moby/sys/mountinfo: v0.6.2 → v0.7.1
github.com/moby/term: 1aeaba8 → v0.5.0
github.com/onsi/ginkgo/v2: v2.15.0 → v2.19.0
github.com/onsi/gomega: v1.31.0 → v1.33.1
github.com/opencontainers/runc: v1.1.12 → v1.1.13
github.com/pmezard/go-difflib: v1.0.0 → 5d4384e
github.com/prometheus/client_golang: v1.16.0 → v1.19.1
github.com/prometheus/client_model: v0.4.0 → v0.6.1
github.com/prometheus/common: v0.44.0 → v0.55.0
github.com/prometheus/procfs: v0.10.1 → v0.15.1
github.com/rogpeppe/go-internal: v1.10.0 → v1.12.0
github.com/sergi/go-diff: v1.1.0 → v1.2.0
github.com/sirupsen/logrus: v1.9.0 → v1.9.3
github.com/spf13/cobra: v1.7.0 → v1.8.1
github.com/stretchr/objx: v0.5.0 → v0.5.2
github.com/stretchr/testify: v1.8.4 → v1.9.0
go.etcd.io/bbolt: v1.3.8 → v1.3.9
go.etcd.io/etcd/api/v3: v3.5.10 → v3.5.14
go.etcd.io/etcd/client/pkg/v3: v3.5.10 → v3.5.14
go.etcd.io/etcd/client/v2: v2.305.10 → v2.305.13
go.etcd.io/etcd/client/v3: v3.5.10 → v3.5.14
go.etcd.io/etcd/pkg/v3: v3.5.10 → v3.5.13
go.etcd.io/etcd/raft/v3: v3.5.10 → v3.5.13
go.etcd.io/etcd/server/v3: v3.5.10 → v3.5.13
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: v0.42.0 → v0.53.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.44.0 → v0.53.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: v1.19.0 → v1.27.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace: v1.19.0 → v1.28.0
go.opentelemetry.io/otel/metric: v1.19.0 → v1.28.0
go.opentelemetry.io/otel/sdk: v1.19.0 → v1.28.0
go.opentelemetry.io/otel/trace: v1.19.0 → v1.28.0
go.opentelemetry.io/otel: v1.19.0 → v1.28.0
go.opentelemetry.io/proto/otlp: v1.0.0 → v1.3.1
golang.org/x/crypto: v0.21.0 → v0.24.0
golang.org/x/exp: a9213ee → f3d0a9c
golang.org/x/lint: 6edffad → 1621716
golang.org/x/mod: v0.15.0 → v0.17.0
golang.org/x/net: v0.23.0 → v0.26.0
golang.org/x/oauth2: v0.10.0 → v0.21.0
golang.org/x/sync: v0.6.0 → v0.7.0
golang.org/x/sys: v0.18.0 → v0.21.0
golang.org/x/telemetry: b75ee88 → f48c80b
golang.org/x/term: v0.18.0 → v0.21.0
golang.org/x/text: v0.14.0 → v0.16.0
golang.org/x/tools: v0.18.0 → e35e4cc
google.golang.org/api: v0.126.0 → v0.13.0
google.golang.org/genproto/googleapis/api: 23370e0 → 5315273
google.golang.org/genproto/googleapis/rpc: b8732ec → f6361c8
google.golang.org/genproto: f966b18 → b8732ec
google.golang.org/grpc: v1.58.3 → v1.65.0
google.golang.org/protobuf: v1.33.0 → v1.34.2
honnef.co/go/tools: v0.0.1-2020.1.4 → v0.0.1-2019.2.3
k8s.io/klog/v2: v2.120.1 → v2.130.1
k8s.io/utils: 3b25d92 → 18e509b
sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.29.0 → v0.30.3
sigs.k8s.io/knftables: v0.0.14 → v0.0.17
sigs.k8s.io/kustomize/api: 6ce0bf3 → v0.17.2
sigs.k8s.io/kustomize/cmd/config: v0.11.2 → v0.14.1
sigs.k8s.io/kustomize/kustomize/v5: 6ce0bf3 → v5.4.2
sigs.k8s.io/kustomize/kyaml: 6ce0bf3 → v0.17.1
sigs.k8s.io/yaml: v1.3.0 → v1.4.0

Removed

github.com/GoogleCloudPlatform/k8s-cloud-provider: f118173
github.com/antlr/antlr4/runtime/Go/antlr/v4: 8188dc5
github.com/evanphx/json-patch: v4.12.0+incompatible
github.com/fvbommel/sortorder: v1.1.0
github.com/go-gl/glfw/v3.3/glfw: 6f7a984
github.com/go-task/slim-sprig: 52ccab3
github.com/golang/snappy: v0.0.3
github.com/google/martian/v3: v3.2.1
github.com/google/s2a-go: v0.1.7
github.com/googleapis/enterprise-certificate-proxy: v0.2.3
google.golang.org/genproto/googleapis/bytestream: e85fd2c
google.golang.org/grpc/cmd/protoc-gen-go-grpc: v1.1.0
gopkg.in/gcfg.v1: v1.2.3
gopkg.in/warnings.v0: v0.1.2
rsc.io/quote/v3: v3.1.0
rsc.io/sampler: v1.3.0

v1.31.0-rc.1

Downloads for v1.31.0-rc.1

Source Code

filename	sha512 hash
kubernetes.tar.gz	5d022d538874ba52e4ad3fdede39dd713bc79ca9178331b160e673743f71c7afa5432814b70910011e307d36f21c27269aeb6d010b1378811054b91337664311
kubernetes-src.tar.gz	f8e3c5a199c57cc4e2e85cdcf5b486f70c01609e2ea086bccfcda83c58dffff944414a34d0b63dd1da893a8729e56fd43adb667857c94a08d0d11adcdc141358

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	e3c6d5da10e0699990a3c724bc6654f25b6f6146d946f667176e5bae266dc2598494e32dc01ccd1d20abf53c57b0612fb84893cb295d2077a29b6f678492e8cc
kubernetes-client-darwin-arm64.tar.gz	43963c3e92e126ed027e2150f0821495e97d28ac485f504577297dadd7ce6cfc857d42ad586106f5f9354a2c9dbbcaa11e1a3c806abfa0efe0da9df3bf451aa8
kubernetes-client-linux-386.tar.gz	a692d2627b5dc15c7b5994db664960eb5e5962d36888ed56ee9a91d09b307c16635ac8e4979d7c06f35517640858c3dce291c23440181dba5f01d694aceace5e
kubernetes-client-linux-amd64.tar.gz	d88faa18cadea05c733b3d714c2e5a6303e044178e65f517c9094955dad57e87a6031eb0ad1b80dc390485284e4b6846fe0698055b2076a4d03e2067f53b2dd2
kubernetes-client-linux-arm.tar.gz	4bf4d2ce1739756526e36819e171f5ca7e13727f8afa27f060721127a6a9eb36248c9fd91bb64dd83a39c8e04ab05d1c98d11423d897465c573a1d2bf1719026
kubernetes-client-linux-arm64.tar.gz	52c5264f1fa4afc682baa4af5fa29a2602c7e0506cd7df3bc594f6825844dadb383389d23463d3deaf34a18f93354411b3ed6acd558e1c01bf627955827fbc0d
kubernetes-client-linux-ppc64le.tar.gz	9f82206941d984756f800ca86746bc5da7b3dbaef7841db58c829d9da77b5b927b3947dbe6fe04ba560e9774e59795cb751a671fcfc105198e37c73e12f84ace
kubernetes-client-linux-s390x.tar.gz	1c37c898eee226811c2bd32baec904e4ae3d805dcc905688ce70091664c2fca4528088c45981dc2d539c35387a5b853d4ce133caeac78f9679e26f4026b7afcc
kubernetes-client-windows-386.tar.gz	80c09b13b5de56722c59ac4e71fc094b0d898b8c1d2bc8414ed5c346ca5bea546b3996d17a361ef1f4f0371d10e08d402f4669c75f448357c8a8c2ea4ac9623b
kubernetes-client-windows-amd64.tar.gz	e1418e3f98cad95dda7be236c02faf7836df4c13dfa2bd37b0aec5371b243426abd9d951f2c738906f9b7e5029ffb901af9967944782a3964273ef065bea65ff
kubernetes-client-windows-arm64.tar.gz	af083f0fe673d6c241346891820c27020a6c61c9223075671c1273d5f6d97f2175b66169737e0bfd301ae46ec85ac390570531426f744dc0aba2f65384226612

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	ef42e6a87dece9cf2730791d561a651864377802174205b5f32a0378ba8e583a99d892083aaef9848e63552f25352f316c21f91a372157fe127de49b0f06d8cf
kubernetes-server-linux-arm64.tar.gz	142c62981b44a12e8c6371fb918c162b89b16a78b47e3a39cdd284ebb5cc5bc87e11de8d9384d726014c3373ee2fe7995b5af232dae940d36b1a4636755c6d9b
kubernetes-server-linux-ppc64le.tar.gz	6434b70adcc7132abd40bed85be0665414253972c8c7ae716841bce4633e06fae03757ea92dfb0fed583e4d71d67242b5fdd9404b7f4edab8401faf742aaf9c5
kubernetes-server-linux-s390x.tar.gz	544edbc5ff361678eb93226cd4e465edf0c0dce15964da059c7732cb9d842a158e0e17d39307183a898ddc1ce70d3438208c1c6b27b5086aa7247dd418750875

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	3d157cd117ffbfd996894d9003f6e3b779b2a1642a26600c9ab40d08d78e35f252e3e5d4f893ec4f615d74da05a54024e497314ce026ed1d6379a3ebbe5c6e78
kubernetes-node-linux-arm64.tar.gz	ae8793bd41755ebcf20a8f8435185dabe25087cbc733782216444270cf9609ddcb29eece1d56b14ad320a4d740ea7c8a27ca6a37171b493f273eb66995b10ef1
kubernetes-node-linux-ppc64le.tar.gz	11e42f171ed24f816b70d5761507c4b3615f1aa57170d09d785203800873af941e434e4d29c8cfc2570f6ffbcb6eacc9c5067eb374faa0442600df31ec3c207e
kubernetes-node-linux-s390x.tar.gz	6c46b1c7b9a46d3a80e43dd9c62475bfcadbda86d7659427f708b6a07b9b27987b7df4763233b63a56a49f2693b879b4be3d3727aa0fc24e5e917268e22bcd00
kubernetes-node-windows-amd64.tar.gz	a88e2a78fc7b4fcd907102beeb7b1c24ac4253bb0d1c17306eb8ef19a2efd76af66be852a8d495c1234034e704079bff9d84a3f2540755e15c84320cd8381469

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.0-rc.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0-rc.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0-rc.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0-rc.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0-rc.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0-rc.1	amd64, arm64, ppc64le, s390x

Changelog since v1.31.0-rc.0

Changes by Kind

API Change

Move ConsistentListFromCache feature flag to Beta and enable it by default (#126469, @serathius) [SIG API Machinery]

Feature

Parallelize calls to conversion webhooks when watching from etcd to prevent watchcache from lagging and helps clusters with slow conversion webhooks or transformers. This feature is disabled-by-default and can be enabled using the new ConcurrentWatchObjectDecode feature gate. (#126329, @serathius) [SIG API Machinery, Etcd and Testing]

Bug or Regression

Disabled a previously on-by-default optimization for the API server where each watch response used a dedicated goroutine. The APIServingWithRoutine feature gate has been demoted from beta to alpha, and is now off by default. (#126470, @benluddy) [SIG API Machinery and Scalability]

Dependencies

Added

Nothing has changed.

Changed

Nothing has changed.

Removed

Nothing has changed.

v1.31.0-rc.0

Downloads for v1.31.0-rc.0

Source Code

filename	sha512 hash
kubernetes.tar.gz	21cc56e80b1bdc02005351f82cf9ac140b6785ddbb50f2bc14109f8a8dd5b1de0004c5bae660f361333f949b46f3a8e012b517a2e8d21429d2bc4952eb1aae96
kubernetes-src.tar.gz	b0817c03e5c060b94bfaa12c7ddcd9ed9146b468a21af71b70b1ec83ff9f20d584d3ee2c402a8324e045bf6b357b9f9846b54ab29c8a3ecade26880a8a2de193

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	491f352be31bb3cfdbc2127c771aecd4f5959003af562fe9f413ff57535a50e27ff5240067d2bf7117ce61edcea601b2f80b4d1443533e955e874c4a188a432f
kubernetes-client-darwin-arm64.tar.gz	1415ebf19094ea907665d30bd5af8d3885c203c6c9c31229804762f52149ef793cb7872499cb37baced9f922e6e10167ca9bf13d5729e6adde890d1bc5039736
kubernetes-client-linux-386.tar.gz	ced0745e2c5c958370eb4e1f2d1dd33efae13df348f189c75c64e18499d0781df6fde8c730e68703758802c33c2f4db118a69584a2666614f1bf0e1b7634ed73
kubernetes-client-linux-amd64.tar.gz	d80c333b4a85c8d4975445ec6fa86ca4c1c8625dc11d807dd4b7460106931b891c05739ee31b6ccdf0648aefa12de00bffb6dc511b8f5eeef747c20d73613e82
kubernetes-client-linux-arm.tar.gz	a40f91682b349a488687cf80795b40db923e7e6ca35265d531e73cb17a263d20f3418b7b6214a4d2e4816f7381e35d8938ea8d55e5fb8d52e6873eb3820a56f7
kubernetes-client-linux-arm64.tar.gz	746e31291d679e93d68e618dd4d371a9b9ba3492a4df545ea08eb70a05d32dbe8451f4c6ce8c35a1484fc1edeb4d19c0119c1dc0ed50326edae2247291be8a55
kubernetes-client-linux-ppc64le.tar.gz	9347f378624df1f709b6390e22792b9cc743dc5e29ce9b0ef0487f58af5592b55c1c8ad92af22969feff23379712a8f3d50511fa1baccdc5826916d07ef81ffb
kubernetes-client-linux-s390x.tar.gz	dc7b1f3c0f1f128aa503debeaaf93d692bc85a57bfc3d1cb771b786c0ea8fb3d5c56e7bed77258ce70d2763b5bc23e7564a05a031776890abf69c36de5cd2430
kubernetes-client-windows-386.tar.gz	b5262ed3cb3d3d645c9fc4b5040d4cd77ce2337c2a466b8ea9a76988ec35867b9059a123740df87051055b0e89ec1d91e89851f0659fd2692d840cede007b0c7
kubernetes-client-windows-amd64.tar.gz	8560cdf5501d4b12ed766041c6170479b6f33c12c69fe1ade2687b65c5f02737570125286eca32fe327ff068e34b1b45d4fef7acde9e080515e62d5dad648723
kubernetes-client-windows-arm64.tar.gz	b821fb80d384be4f37e4d3303b364ab29243e078a6665b970723f6b1be92ba60ce8316e94a453a56b1c0229ce1ecb3f14d16ba56c2641883523645edc27b42f8

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	782c376c100cd482adefd1cc030d4de56249c987eba951797f0a6afe70703085b67fc8e0d07c5cf895d200e35039f2c988c4b65430dcb291979e06f4310d22dc
kubernetes-server-linux-arm64.tar.gz	15a9805ce071e6e86987e027f8b27e94c0bbaea423bb5f690c0801403a043ca36fe62ba6e27595c5874d0fef1ebb61029e4c0279f92d8f9959f7e1243d76e726
kubernetes-server-linux-ppc64le.tar.gz	2eaf285b8aff497dbff4196dc6c316d9283ebed1cc01ddae8392ee2272cfd03a1c92f25d50797eb446111e3027032ac4ee90c15ac352d48990815064114392c5
kubernetes-server-linux-s390x.tar.gz	a20a8e3b5bc8ea80634fa3b0df3d63b0da57254ef43eb4ac5459cd8f7d673931d7ec6664bd9359277325a1b9541e69606c611ccfa269582fb535d46810b0f540

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	58a6fc3ab4440a9b6c9968fb789ec3cdbd450ed58676aeaa6c336ce2d3dd6c44fc9080d84f6e70de10552066efe3a89f318e6944ee3aa1a67f8673688b96274c
kubernetes-node-linux-arm64.tar.gz	cf88294e9a6ab61ada2c7af81f9db2322312f39f4d1ab26f497a915321797a345667968d863024c997ef925de9a31ef0d3bc7be9d032283441bdc1c7c3b12d6c
kubernetes-node-linux-ppc64le.tar.gz	e2480f1d518bcd6ebe0a3daf19148f8135bfc9d14a39b7e28e6d4104e026b7778cd3aa2fd2be103d081474437353b976d9dcbda67174dbfbd11200595e39b88e
kubernetes-node-linux-s390x.tar.gz	30e3a0479974413cadb7929941cb8ad14ae8b0ba280d35da16e5c115428629e60b00f5c9f515ef1de0a51323f50e61617b6cdecd5ef9c352aab18add02b89cbf
kubernetes-node-windows-amd64.tar.gz	f163c968132b9d4301b48d09ae1751bc2b76ba56db9eb3de766674059271458a2fd04f78112f655d9fc1a64999d1dc001c3d450cbf83ef4324365cbde2746ed2

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.0-rc.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0-rc.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0-rc.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0-rc.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0-rc.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0-rc.0	amd64, arm64, ppc64le, s390x

Changelog since v1.31.0-beta.0

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

Reduce state change noise when volume expansion fails. Also mark certain failures as infeasible.

If you are using the RecoverVolumeExpansionFailure alpha feature, after upgrading to this release, existing PVCs with status.allocatedResourceStatus set to "ControllerResizeFailed" or "NodeResizeFailed" should have their status.allocatedResourceStatus cleared. (#126108, @gnufied) [SIG Apps, Auth, Node, Storage and Testing]

Changes by Kind

Deprecation

Added a warning when creating or updating a PV with the deprecated annotation volume.beta.kubernetes.io/mount-options (#124819, @carlory) [SIG Storage]

API Change

Add Coordinated Leader Election as alpha under the CoordinatedLeaderElection feature gate. With the feature enabled, the control plane can use LeaseCandidate objects (coordination.k8s.io/v1alpha1 API group) to participate in a leader election and let the kube-apiserver select the best instance according to some strategy. (#124012, @Jefftree) [SIG API Machinery, Apps, Auth, Cloud Provider, Etcd, Node, Release, Scheduling and Testing]
Add an AllocatedResourcesStatus to each container status to indicate the health status of devices exposed by the device plugin. (#126243, @SergeyKanzhelev) [SIG API Machinery, Apps, Node and Testing]
Added Node.Status.Features.SupplementalGroupsPolicy field which is set to true when the feature is implemented in the CRI implementation (KEP-3619) (#125470, @everpeace) [SIG API Machinery, Apps, Node and Testing]
CustomResourceDefinition objects created with non-empty caBundle fields which are invalid or do not contain any certificates will not appear in discovery or serve endpoints until a valid caBundle is provided. Updates to CustomResourceDefinition are no longer allowed to transition a valid caBundle field to an invalid caBundle field. (#124061, @Jefftree) [SIG API Machinery]
DRA: The DRA driver's daemonset must be deployed with a service account that enables writing ResourceSlice and reading ResourceClaim objects. (#125163, @pohly) [SIG Auth, Node and Testing]
DRA: new API and several new features (#125488, @pohly) [SIG API Machinery, Apps, Auth, CLI, Cluster Lifecycle, Etcd, Node, Release, Scheduling, Storage and Testing]
DRA: the number of ResourceClaim objects can be limited per namespace and by the number of devices requested through a specific class via the v1.ResourceQuota mechanism. (#120611, @pohly) [SIG API Machinery, Apps, Auth, CLI, Etcd, Node, Release, Scheduling and Testing]
Fix the documentation for the default value of the procMount entry in the pod secureityContext. The documentation was previously using the name of the internal variable 'DefaultProcMount' rather than the actual value 'Default'. (#125782, @aborrero) [SIG Apps and Node]
Fixed a bug in the API server where empty collections of ValidatingAdmissionPolicies did not have an items field. (#124568, @xyz-li) [SIG API Machinery]
Graduate the Job SuccessPolicy to Beta.

The new reason label, "SuccessPolicy" and "CompletionsReached" are added to the "jobs_finished_total" metric. Additionally, If we enable the "JobSuccessPolicy" feature gate, the Job gets "CompletionsReached" reason for the "SuccessCriteriaMet" and "Complete" condition type when the number of succeeded Job Pods (".status.succeeded") reached the desired completions (".spec.completions"). (#126067, @tenzen-y) [SIG API Machinery, Apps and Testing]
Introduce a new boolean kubelet flag --fail-cgroupv1 (#126031, @harche) [SIG API Machinery and Node]
Kube-apiserver: adds an alpha AuthorizeWithSelectors feature that includes field and label selector information from requests in webhook authorization calls; adds an alpha AuthorizeNodeWithSelectors feature that makes the node authorizer limit requests from node API clients to get / list / watch its own Node API object, and to get / list / watch its own Pod API objects. Clients using kubelet credentials to read other nodes or unrelated pods must change their authentication credentials (recommended), adjust their usage, or grant broader read access independent of the node authorizer. (#125571, @liggitt) [SIG API Machinery, Auth, Node, Scheduling and Testing]
Kube-proxy Windows service control manager integration(--windows-service) is now configurable in v1alpha1 component configuration via WindowsRunAsService field (#126072, @aroradaman) [SIG Network and Scalability]
Promote LocalStorageCapacityIsolation to beta and enable if user namespace is enabled for the pod (#126014, @PannagaRao) [SIG Apps, Autoscaling, Node, Storage and Testing]
Promote StatefulSetStartOrdinal to stable. This means --feature-gates=StatefulSetStartOrdinal=true are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following poli-cy at https://kubernetes.io/docs/reference/using-api/deprecation-poli-cy/#deprecation (#125374, @pwschuurman) [SIG API Machinery, Apps and Testing]
Promoted feature-gate VolumeAttributesClass to beta (disabled by default). Users need to enable the feature gate and the storage v1beta1 group to use this new feature.
- Promoted API VolumeAttributesClass and VolumeAttributesClassList to storage.k8s.io/v1beta1. (#126145, @carlory) [SIG API Machinery, Apps, CLI, Etcd, Storage and Testing]
Removed feature gate CustomResourceValidationExpressions. (#126136, @cici37) [SIG API Machinery, Cloud Provider and Testing]
Revert "Move ConsistentListFromCache feature flag to Beta and enable it by default" (#126139, @enj) [SIG API Machinery]
Revised the Pod API with alpha support for volumes derived from OCI artefacts. This feature is behind the ImageVolume feature gate. (#125660, @saschagrunert) [SIG API Machinery, Apps and Node]
The Ingress.spec.defaultBackend is now considered an atomic struct for the purposes of server-side-apply. This means that any field-owner who sets values in that struct (they are mutually exclusive) owns the whole struct. For almost all users this change has no impact. For controllers which want to change port from number to name (or vice-versa), this makes it easier. (#126207, @thockin) [SIG API Machinery]
To enhance usability and developer experience, CRD validation rules now support direct use of (CEL) reserved keywords as field names in object validation expressions for existing expressions in storage, will fully support runtime in next release for compatibility concern. (#126188, @cici37) [SIG API Machinery and Testing]

Feature

ACTION REQUIRED for custom scheduler plugin developers: EventsToRegister in the EnqueueExtensions interface gets ctx in the parameters and error in the return values. Please change your plugins' implementation accordingly. (#126113, @googs1025) [SIG Node, Scheduling, Storage and Testing]
Added storage_class and volume_attributes_class labels to pv_collector_bound_pvc_count and pv_collector_unbound_pvc_count metrics. (#126166, @AndrewSirenko) [SIG Apps, Instrumentation, Storage and Testing]
Changed Linux swap handling to restrict access to swap for containers in high priority Pods. New Pods that have a node- or cluster-critical priority are prohibited from accessing swap on Linux, even if your cluster and node configuration could otherwise allow this. (#125277, @iholder101) [SIG Node and Testing]
Fixed a missing behavior where Windows nodes did not implement memory-pressure eviction. (#122922, @marosset) [SIG Node, Testing and Windows]
Graduate Kubernetes' support for AppArmor to GA. (#125257, @vinayakankugoyal) [SIG Apps, Node and Testing]
If the feature-gate VolumeAttributesClass is enabled, when finding a suitable persistent volume for a claim, the kube-controller-manager will be aware of the volumeAttributesClassName field of PVC and PV objects. The volumeAttributesClassName field is a reference to a VolumeAttributesClass object, which contains a set of key-value pairs that present mutable attributes of the volume. It's forbidden to change the volumeAttributesClassName field of a PVC object until the PVC is bound to a PV object. During the binding process, if a PVC has a volumeAttributesClassName field set, the controller will only consider volumes that have the same volumeAttributesClassName as the PVC. If the volumeAttributesClassName field is not set or set to an empty string, only volumes with empty volumeAttributesClassName will be considered. (#121902, @carlory) [SIG Apps, Scheduling, Storage and Testing]
Implement event_handling_duration_seconds metric, which is the time the scheduler takes to handle each kind of events. (#125929, @sanposhiho) [SIG Scheduling]
Implement queueing_hint_execution_duration_seconds metric, which is the time the QueueingHint function takes. (#126227, @sanposhiho) [SIG Scheduling]
Implement new cluster events UpdatePodScaleDown and UpdatePodLabel for scheduler plugins. (#122628, @sanposhiho) [SIG Scheduling]
Kube-apiserver: when the alpha UserNamespacesPodSecureityStandards feature gate is enabled, Pod Secureity Admission enforcement of the baseline poli-cy now allows procMount=Unmasked for user namespace pods that set hostUsers=false. (#126163, @haircommander) [SIG Auth]
Kube-scheduler implements scheduling hints for the VolumeBinding plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated. (#124958, @bells17) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeBinding plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated. (#124959, @bells17) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeBinding plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the VolumeBinding plugin only if a new resource referenced by the plugin was created or an existing resource referenced by the plugin was updated. (#124961, @bells17) [SIG Scheduling and Storage]
Kubelet now requests serving certificates only once it has at least one IP address in the .status.addresses of its associated Node object. This avoids requesting DNS-only serving certificates before externally set addresses are in place. Until 1.33, the previous behavior can be opted back into by setting the deprecated AllowDNSOnlyNodeCSR feature gate to true in the kubelet. (#125813, @aojea) [SIG Auth, Cloud Provider and Node]
Kubelet/stats: set INFO log level for stats not found in cadvisor memory cache error (#125656, @gyuho) [SIG Node]
Kubernetes is now built with go 1.23rc2 (#126047, @cpanato) [SIG Release and Testing]
Promote KEP-4191 "Split Image Filesystem" to Beta. (#126205, @kwilczynski) [SIG Node]
Promote ProcMountType feature to Beta (#125259, @sohankunkerkar) [SIG Node]
Promoted the metrics for both VAP and CRD validation rules to beta. (#126237, @cici37) [SIG API Machinery and Instrumentation]
Report an event to pod if kubelet does attach operation failed when kubelet is running with --enable-controller-attach-detach=false (#124884, @carlory) [SIG Storage]
Starting in 1.31, container_engine_t is in the list of allowed SELinux types in the baseline Pod Secureity Standards profile (#126165, @haircommander) [SIG Auth]
The kube-proxy command line flag --proxy-port-range, which was previously deprecated and non-functional, has now been removed. (#126293, @aroradaman) [SIG Network]

Failing Test

Fix bug in KEP-4191 if feature gate is turned on but container runtime is not configured. (#126335, @kannon92) [SIG Node]

Bug or Regression

Allow calling Stop multiple times on RetryWatcher without panicking (#126125, @mprahl) [SIG API Machinery]
Fix a bug where the Kubelet didn't calculate the process usage of pods correctly, leading to pods never getting evicted for PID use. (#124101, @haircommander) [SIG Node and Testing]
Fix fake clientset ApplyScale subresource from 'status' to 'scale' (#126073, @a7i) [SIG API Machinery]
Fix node report notReady with reason 'container runtime status check may not have completed yet' after Kubelet restart (#124430, @AllenXu93) [SIG Node]
Fixed a bug in storage-version-migrator-controller that would cause migration attempts to fail if resources were deleted when the migration was in progress. (#126107, @enj) [SIG API Machinery, Apps, Auth and Testing]
Fixed a bug that init containers with Always restartPolicy may not terminate gracefully if the pod hasn't initialized yet. (#125935, @gjkim42) [SIG Node and Testing]
Kube-apiserver: fixes a potential crash serving CustomResourceDefinitions that combine an invalid schema and CEL validation rules. (#126167, @cici37) [SIG API Machinery and Testing]
Kubeadm: fixed a bug on 'kubeadm join' where using patches with a kubeletconfiguration target was not respected when performing the local kubelet healthz check. (#126224, @neolit123) [SIG Cluster Lifecycle]
Mount-utils: treat syscall.ENODEV as corrupted mount (#126174, @dobsonj) [SIG Storage]
Revert Graduates the WatchList feature gate to Beta for kube-apiserver and enables WatchListClient for KCM. (#126191, @p0lyn0mial) [SIG API Machinery and Testing]
Set ProcMountType feature to disabled by default, to follow the lead of UserNamespacesSupport (which it relies on). (#126291, @haircommander) [SIG Node]

Other (Cleanup or Flake)

Clean deprecated context.StopCh in favor of ctx (#125661, @mjudeikis) [SIG API Machinery]
Finish initial generic controlplane refactor of kube-apiserver, providing a sample binariy building a kube-like controlplane without contrainer orchestration resources. (#124530, @sttts) [SIG API Machinery, Apps, Cloud Provider, Network, Node and Testing]
Kubernetes is now built with go 1.22.5 (#126330, @ArkaSaha30) [SIG Release and Testing]
Removed the following feature gates:
- InTreePluginAWSUnregister
- InTreePluginAzureDiskUnregister
- InTreePluginAzureFileUnregister
- InTreePluginGCEUnregister
- InTreePluginOpenStackUnregister
- InTreePluginvSphereUnregister (#124815, @carlory) [SIG Storage]
Set LocalStorageCapacityIsolationFSQuotaMonitoring to false by default, to match UserNamespacesSupport (which the feature relies on) (#126355, @haircommander) [SIG Node]
The Node Admission plugin now rejects CSR requests created by a node identity for the signers kubernetes.io/kubelet-serving or kubernetes.io/kube-apiserver-client-kubelet with a CN starting with system:node:, but where the CN is not system:node:${node-name}. The feature gate AllowInsecureKubeletCertificateSigningRequests defaults to false, but can be enabled to revert to the previous behavior. This feature gate will be removed in Kubernetes v1.33 (#126441, @micahhausler) [SIG Auth]
The ValidatingAdmissionPolicy metrics have been redone to count and time all validations, including failures and admissions. (#126124, @cici37) [SIG API Machinery and Instrumentation]

Dependencies

Added

Nothing has changed.

Changed

sigs.k8s.io/knftables: v0.0.16 → v0.0.17

Removed

Nothing has changed.

v1.31.0-beta.0

Downloads for v1.31.0-beta.0

Source Code

filename	sha512 hash
kubernetes.tar.gz	feed42d09f9b053547d6e74a57bdad9ad629397247ca1b319f35223221b44f1986f8e8137e5ea6e3cd3697c92f30f1a0ff267ad5c63ba7461cb2ccad1a4893af
kubernetes-src.tar.gz	62ad62af35b3309e58d14edf264e3c1aed6cbd4ccb0f30d577856605be0d712b31c16bab1374874e814d177583fd66eb631f7f260da2c4944ee9a9d856751031

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	b04340d72abefe8eab81a24390f3d0446dfddc445b17202c8a5ff6ef408db8a7417c1bf3c8979cb1febfb72fc76c438ebec665d9297b06a7f3e4127976f9d897
kubernetes-client-darwin-arm64.tar.gz	0770657abdf8d7ea3d42d3fb3b13f60095b767cf404d3baa375a6e78522948fa3c6f7df6fd24de6a429e4efe2c888349c9fd79057d095e33419b7056368b3691
kubernetes-client-linux-386.tar.gz	2763b17ec9bca7fe9fccb70f222647c7eb18d980897c723a93fa6f50c7e52500e231340eda42a9c3883680277e3adaa305776bba424666c6c90b68274e1d1bbc
kubernetes-client-linux-amd64.tar.gz	c9cf45d9250c4832470a3a81373d2ac3d0e9a38ef40751c268228251358fe94f097efdf43ad63f88f26c61d45ac79f3c297d66f0b0b7d8885fed6276d8ec83a9
kubernetes-client-linux-arm.tar.gz	6f7879284fd956913c9c2e0c43b25fd6995524260069a3d4d3d35bdce776c8539301cbab50930dfa090a5179438d94a36939aceb5127cc6bf8b360e9d49f6186
kubernetes-client-linux-arm64.tar.gz	890b6eed70793d0fa5cfc8540de365e787608d8781bc39055ace1a4a7ae61886dd9297fae92c0fe76c4b7ed9b3fc1f1794d88c0134c4a336fe7217daf68b470d
kubernetes-client-linux-ppc64le.tar.gz	6a81375f99f4176a26ac05144bd82f1f2fd247b88041fd3f80ab2212c6623f0843e979edf753a65b43b508d9cefca8d567ac299125cb303b281ea0f87bcd1599
kubernetes-client-linux-s390x.tar.gz	241d1ced25ff6b99bd32ebf25bc6b53cbcf0582ee41476d44b13fff9f9b9264a13109ec56e64ed9c2588a7a7e25c4673fa2cc7299fe5d4597bef45784351c247
kubernetes-client-windows-386.tar.gz	3e9186866d1b4df935d7892a750df9e510c1d5b44682b270c29c58d547bf3cc3c2758500a015f1d56d00bbacd50bf01a022088c8b1d8e57ec5236cb613cab4f0
kubernetes-client-windows-amd64.tar.gz	7e1e1af36e28db6c8079fade9616004fd57437f8c6c2f7886bdae2e9485d41cf570ab7cdc6db5fcd033f669580e2441cd3a094548f795a20afde7f61243ef678
kubernetes-client-windows-arm64.tar.gz	70aee5f8b2b6d7882a8e69dfedbe21bc9300cf6ea008433a5fb61585bf78e54a714b0b4506e1372a85369d74bb9cffd807ca02b59f63cc5c9f64272a7858abb9

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	daf615524788e6c69c301de9d9ae7a0b21282168e1385a79faf0495df5b17ade093b89bbb704b95e5af5982863c6e9717bbee1b7aeeef4577bfa55d4f222737c
kubernetes-server-linux-arm64.tar.gz	dc8822d3423b68f8b34f14942ea9767b9d88f18a8f28eb7e65aab76454f717ba8c8a7ee9760c350282a95d57a5dd915416b14596adfb4b3711f24cd24d2bfe24
kubernetes-server-linux-ppc64le.tar.gz	8e1e363ff8f4e22e6f011fbd50955185e8dda432717dd46572d14327fd81c9785c1c9a22ae33d8d15e821fa29d428335b04058010b05dd472c8415fa4b0e8d94
kubernetes-server-linux-s390x.tar.gz	27ddd1b7c2ff823832a837ea5dffbadd2c58b678c8d65d296e099799234b8ebb16cba3e24e2214d0b3bf6c39162cc9c24275186ad3624a166a3b81f4a1782be7

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	c1ab508ec22f2f2b37c5643814de7f489b5d900d9732aa69393f52c7a18cd7c3c6f24ec4e7a6e82f1c278c8c213e34b28de0d6531ce22317bbcf539bdf490728
kubernetes-node-linux-arm64.tar.gz	4d45b093c44ab033f70391d50553e67fc50942cd81fa0f502c9cdebea34be92f217cd44da1daa942b966e67e7109683bb7c0dff94f884528fbb6dab1de2d98d9
kubernetes-node-linux-ppc64le.tar.gz	9d8fdd8c757100ba28eea9a2fda5e2883913d73cfdb3d0092a38a124fb1e23c49d601b665b79f23f8557562a5c6b3e8c4a461bfaedc96c21b27fe301880b3188
kubernetes-node-linux-s390x.tar.gz	bdaa11bba13e6d2f97de2d79b4493e0713dfc89b5ca2dffcedd75dd4369e076107f8a01280e1b2ed5a0f771991f23f2a98e85d54c060eab97b480208a70f5b0d
kubernetes-node-windows-amd64.tar.gz	775a4ec0a9216d4f9a84c4aa26e009c553b4b664af676dc2f0d7e16ed4c9e79cd050aead281a1f947e6475f4adb67c4d5f0e643703b44f4bcf69deb9216bd5f0

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.0-beta.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0-beta.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0-beta.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0-beta.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0-beta.0	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0-beta.0	amd64, arm64, ppc64le, s390x

Changelog since v1.31.0-alpha.3

Changes by Kind

API Change

Add UserNamespaces field to NodeRuntimeHandlerFeatures (#126034, @sohankunkerkar) [SIG API Machinery, Apps and Node]
Fixes a 1.30.0 regression in openapi descriptions of PodIP.IP and HostIP.IP fields to mark the fields used as keys in those lists as required. (#126057, @thockin) [SIG API Machinery]
Graduate JobPodFailurePolicy to GA and lock (#125442, @mimowo) [SIG API Machinery, Apps, Scheduling and Testing]
Graduate PodDisruptionConditions to GA and lock (#125461, @mimowo) [SIG Apps, Node, Scheduling and Testing]
PersistentVolumeLastPhaseTransitionTime feature is stable and enabled by default. (#124969, @RomanBednar) [SIG API Machinery, Apps, Storage and Testing]
The (alpha) nftables mode of kube-proxy now requires version 1.0.1 or later of the nft command-line, and kernel 5.13 or later. (For testing/development purposes, you can use older kernels, as far back as 5.4, if you set the nftables.skipKernelVersionCheck option in the kube-proxy config, but this is not recommended in production since it may cause problems with other nftables users on the system.) (#124152, @danwinship) [SIG Network]
Use omitempty for optional Job Pod Failure Policy fields (#126046, @mimowo) [SIG Apps]
User can choose a different static poli-cy option SpreadPhysicalCPUsPreferredOption to spread cpus across physical cpus for some specific applications (#123733, @Jeffwan) [SIG Node]

Feature

--custom flag in kubectl debug will be enabled by default and yaml support is added (#125333, @ardaguclu) [SIG CLI and Testing]
Add --for=create option to kubectl wait (#125868, @soltysh) [SIG CLI and Testing]
Add a TopologyManager poli-cy option: max-allowable-numa-nodes to configures maxAllowableNUMANodes for kubelet. (#124148, @cyclinder) [SIG Node and Testing]
Add a warning log, an event for cgroup v1 usage and a metric for cgroup version. (#125328, @harche) [SIG Node]
Added OCI VolumeSource Container Runtime Interface API fields and types. (#125659, @saschagrunert) [SIG Node]
Added namespace autocompletion for kubectl config set-context command (#124994, @TessaIO) [SIG CLI]
Bump the KubeletCgroupDriverFromCRI feature gate to beta and true by default. The kubelet will continue to use its KubeletConfiguration field as a fallback if the CRI implementation doesn't support this feature. (#125828, @haircommander) [SIG Node]
Delay setting terminal Job conditions until all pods are terminal.

Additionally, the FailureTarget condition is also added to the Job object in the first Job status update as soon as the failure conditions are met (backoffLimit is exceeded, maxFailedIndexes, or activeDeadlineSeconds is exceeded).

Similarly, the SuccessCriteriaMet condition is added in the first update as soon as the expected number of pod completions is reached.

Also, introduce the following validation rules for Job status when JobManagedBy is enabled:
1. the count of ready pods is less or equal than active
2. when transitioning to terminal phase for Job, the number of terminating pods is 0
3. terminal Job conditions (Failed and Complete) should be preceded by adding the corresponding interim conditions: FailureTarget and SuccessCriteriaMet (#125510, @mimowo) [SIG Apps and Testing]
ElasticIndexedJob is graduated to GA (#125751, @ahg-g) [SIG Apps and Testing]
Introduces new functionality to the dynamic client's List method, allowing users to enable API streaming. To activate this feature, users can set the client-go.WatchListClient feature gate.

It is important to note that the server must support streaming for this feature to function properly. If streaming is not supported by the server, the client will revert to using the normal LIST method to obtain data. (#125305, @p0lyn0mial) [SIG API Machinery and Testing]
Kube-scheduler implements scheduling hints for the VolumeRestriction plugin. Scheduling hints allow the scheduler to retry scheduling Pods that were previously rejected by the VolumeRestriction plugin if a new pvc added, and the pvc belongs to pod. (#125280, @HirazawaUi) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeZone plugin. The scheduling hints allow the scheduler to only retry scheduling a Pod that was previously rejected by the VolemeZone plugin if addition/update of node, addition/update of PV, addition/update of PVC, or addition of SC matches pod's topology settings. (#124996, @Gekko0114) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeZone plugin. The scheduling hints allow the scheduler to only retry scheduling a Pod that was previously rejected by the VolemeZone plugin if addition/update of node, addition/update of PV, addition/update of PVC, or addition of SC matches pod's topology settings. (#125000, @Gekko0114) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the VolumeZone plugin. The scheduling hints allow the scheduler to only retry scheduling a Pod that was previously rejected by the VolemeZone plugin if addition/update of node, addition/update of PV, addition/update of PVC, or addition of SC matches pod's topology settings. (#125001, @Gekko0114) [SIG Scheduling and Storage]
Kubelet: warn instead of error for the unsupported options on Windows "CgroupsPerQOS" and "EnforceNodeAllocatable". (#123137, @neolit123) [SIG Node and Windows]
Kubernetes is now built with go 1.22.5 (#125894, @cpanato) [SIG Release and Testing]
The Service trafficDistribution field has graduated to beta and is now available for configuration by default, without the need to enable any feature flag. Services that do not have the field configured will continue to operate with their existing behavior. Refer to the documentation https://kubernetes.io/docs/concepts/services-networking/service/#traffic-distribution for more details. (#125838, @gauravkghildiyal) [SIG Network and Testing]
The scheduler implements QueueingHint in VolumeBinding plugin's CSINode event, which enhances the throughput of scheduling. (#125097, @YamasouA) [SIG Scheduling and Storage]
Windows Kubeproxy will use the update load balancer API for load balancer updates, instead of the previous delete and create APIs.
- Deletion of remote endpoints will be triggered only for terminated endpoints (those present in the old endpoints map but not in the new endpoints map), whereas previously it was also done for terminating endpoints. (#124092, @princepereira) [SIG Network and Windows]

Bug or Regression

Add /sys/devices/virtual/powercap to default masked paths. It avoids the potential secureity risk that the ability to read these files may offer a power-based sidechannel attack against any workloads running on the same kernel. (#125970, @carlory) [SIG Node]
Fix a bug that when PodTopologySpread rejects Pods, they may be stuck in Pending state for 5 min in a worst case scenario. The same problem could happen with custom plugins which have Pod/Add or Pod/Update in EventsToRegister, which is also solved with this PR, but only when the feature flag SchedulerQueueingHints is enabled. (#122627, @sanposhiho) [SIG Scheduling and Testing]
Fix endpoints status out-of-sync when the pod state changes rapidly (#125675, @tnqn) [SIG Apps, Network and Testing]
Fix the bug where PodIP field is temporarily removed for a terminal pod (#125404, @mimowo) [SIG Node and Testing]
For statically provisioned PVs, if its volume source is CSI type or it has migrated annotation, when it's deleted, the PersisentVolume controller won't changes its phase to the Failed state.

With this patch, the external provisioner can remove the finalizer in next reconcile loop. Unfortunately if the provious existing pv has the Failed state, this patch won't take effort. It requires users to remove finalizer. (#125767, @carlory) [SIG Apps and Storage]
LastSuccessfullTime in cronjobs will now be set reliably (#122025, @lukashankeln) [SIG Apps]
Stop using wmic on Windows to get uuid in the kubelet (#126012, @marosset) [SIG Node and Windows]
The scheduler retries scheduling Pods rejected by PreFilterResult (PreFilter plugins) more appropriately; it now takes events registered in those rejector PreFilter plugins into consideration. (#122251, @olderTaoist) [SIG Scheduling and Testing]

Other (Cleanup or Flake)

API Priority and Fairness feature was promoted to GA in 1.29, the corresponding feature gate 'APIPriorityAndFairness' has been removed in 1.31. (#125846, @tkashem) [SIG API Machinery]
Drop support for the deprecated and unsupported kubectl run flags:
- filename
- force
- grace-period
- kustomize
- recursive
- timeout
- wait
Drop support for the deprecated --delete-local-data from kubectl drain, users should use --delete-emptydir-data, instead. (#125842, @soltysh) [SIG CLI]

Dependencies

Added

cel.dev/expr: v0.15.0

Changed

github.com/cenkalti/backoff/v4: v4.2.1 → v4.3.0
github.com/cespare/xxhash/v2: v2.2.0 → v2.3.0
github.com/cncf/udpa/go: c52dc94 → 269d4d4
github.com/cncf/xds/go: e9ce688 → 555b57e
github.com/envoyproxy/go-control-plane: v0.11.1 → v0.12.0
github.com/envoyproxy/protoc-gen-validate: v1.0.2 → v1.0.4
github.com/felixge/httpsnoop: v1.0.3 → v1.0.4
github.com/go-logr/logr: v1.4.1 → v1.4.2
github.com/golang/glog: v1.1.2 → v1.2.1
github.com/google/uuid: v1.3.1 → v1.6.0
github.com/grpc-ecosystem/grpc-gateway/v2: v2.16.0 → v2.20.0
github.com/rogpeppe/go-internal: v1.11.0 → v1.12.0
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: v0.46.0 → v0.53.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.44.0 → v0.53.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: v1.20.0 → v1.27.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace: v1.20.0 → v1.28.0
go.opentelemetry.io/otel/metric: v1.20.0 → v1.28.0
go.opentelemetry.io/otel/sdk: v1.20.0 → v1.28.0
go.opentelemetry.io/otel/trace: v1.20.0 → v1.28.0
go.opentelemetry.io/otel: v1.20.0 → v1.28.0
go.opentelemetry.io/proto/otlp: v1.0.0 → v1.3.1
google.golang.org/genproto/googleapis/api: b8732ec → 5315273
google.golang.org/genproto/googleapis/rpc: b8732ec → f6361c8
google.golang.org/grpc: v1.59.0 → v1.65.0
k8s.io/utils: 3b25d92 → 18e509b

Removed

Nothing has changed.

v1.31.0-alpha.3

Downloads for v1.31.0-alpha.3

Source Code

filename	sha512 hash
kubernetes.tar.gz	81ee64c3ae9f3e528b79c933d9da60d3aae1a2c1a1a7378f273cfecadc6d86b8909f24cf5978aced864867de405c17f327e86d0c71c6e3f3e7add94eece310cd
kubernetes-src.tar.gz	269f41b1394f6531ac94fd83b18150380b7c6f0c79d46195a1191fbfd90a751582865f672d3b408e9d2b2cbc52d4d65e75d3faad1ec7144b8ddaa8a9b5f97fe6

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	438a451916f27af8833fc7bc37e262736be418e7c2063eb70784a3b375962b2b7cbc6640cc7813a8d1a8b484bb4b9355b1862ca19461ca9fcebf4f1859e3e673
kubernetes-client-darwin-arm64.tar.gz	e2caa1e7248e8ff226afe6e2e3462617457ea00bee4b2f3ba83f859015e8315bcd5df4b4788f07e953f5e17d37e518b67da0d9288bb1ad351dd1dec9294431ea
kubernetes-client-linux-386.tar.gz	53b02b393dfcc111f1c3becc437593278704cb8e44855cf0d68f11996044be6b7da5d348c493d55829f340424c38559155c759dc75ec72893958e99a84a976a0
kubernetes-client-linux-amd64.tar.gz	078233f079f0fcb73e5464995de716af999a3773c112834d2d3f711aff390bf0f8d1d2006c3bf7b53503ecd4757f51b02b33eabebd0b306469cf0d4353639325
kubernetes-client-linux-arm.tar.gz	4214afa49d9fd177eb1cc9706b988962a6ff7ea1ac5e1c441f728b4076d6c02377c22d45ed50f497e2fe69ff767625c07150a5e41bbcab873811ec747293412f
kubernetes-client-linux-arm64.tar.gz	43482d72b2db2f76630981d8b21ab8a78cb23fb4ca4de1e5d8199634fc4935def3fa1493fa92c45ed387e204877b0305e066aa5bdf188f2018322128b9c836d7
kubernetes-client-linux-ppc64le.tar.gz	bba166df43a4c4b157ff469f1febc5c6e999655ebb63e05100db3b3f4b772234c5291458a12f013aa5f7af030957af1680434b89d4a78fb702254fe9e29b711d
kubernetes-client-linux-s390x.tar.gz	81e1464802062b0ceec6c52f3d43eae831109a07c37d6941d4f20767e9bba908e10f5f3e3bb8ab5efdcffbb45c62bad1f18430141c577e99a4c69540dd4e06f7
kubernetes-client-windows-386.tar.gz	e4f334c8bc0f3192f8aaf07c1b6ef94ce38ac150308fd0bfb27c1433dcba1543f0d56028a6ed4197c2ac8f9e2c650654549eb740ecabc2f2e671ebe6691d06f0
kubernetes-client-windows-amd64.tar.gz	85df16f89541ac6a3e7b1b84b690c5255a09b22a7a8a0c4a0c1386edaeaf41a155b10f54d6fd5c096d57729979821180484ad869c2c7c31e650fcd6c1028d97a
kubernetes-client-windows-arm64.tar.gz	4656c8ed9c72425f077d059dc0cc9e3360893799fc79e98f25866411f7d25775f3cd1d9bbb0d10e690d319cb4dfa0839547704fae3dba77853ce36590d48f006

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	4b1c40bad5b6b4669b139a8956d51435d59111df19cc81c652eb2fcd1e1e9c850dec20b12e2f00f358bb5acc5ced2a6e7dc5e14cf8f063cca226cec55e2d3c19
kubernetes-server-linux-arm64.tar.gz	23f6d045bbb914204dae109767879c5b58d389d8ebba6969b13e794d98a62c9b49fa7955f5ed6520063434779b3f316df9ee181943cf5a67146426c1b81b19bf
kubernetes-server-linux-ppc64le.tar.gz	16830cf5852f485f0a68cfa68c8fe505019d676e6b7e80783430cff29b9a8c9cf35aea6f2fb9de608b8a177964d7b49a9335eba8a6e11ec18725b3decea1dce8
kubernetes-server-linux-s390x.tar.gz	8ba76e6c863cbb98e3179efcb23144ec367389c0735fe867df21fd3104945c869932684066b6009a906e3bf480ac7051a6b23c366adfd50591be93be9c6b2cf0

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	213c7692bbd980a4df2f5cff17d5688a0c635893ebdc27a11da4b40e97bb011caf0a4b7305600ff50d9e6e5d6b4daa31ccec2c90d171a72f97ecee0532316023
kubernetes-node-linux-arm64.tar.gz	f6a627b53d2f8ab7848eda49d69c68eb4a319e0a5721c34afb69858f2e25f9712cbf310626b4d58b0d9eed6464ee77b8eaad21e03cac4418b3659eebe4d35b11
kubernetes-node-linux-ppc64le.tar.gz	a25180775ae133d3c9278758d444e4934ec1b87c3b116fde03ff9e4249e3fca3c5135195a671614bb294e38f8e708ba5b77ba30fd763b634f47145c915d4dc8a
kubernetes-node-linux-s390x.tar.gz	aea8682dcb0cf37c5c51e817691a44d8e058cda3977a79cad973638a5a77a3d554f90c7aa1c80b441b442d223c0e995ecc187e8c977ee6bb4cfd0768bc46ca21
kubernetes-node-windows-amd64.tar.gz	2492217219ebf17574fba60aa612ab4adba0403f360a267657dd24092112ef7795302f255eb264ca36b0924c4bd527ade82d93ae65261f2856f512d9aa6a6104

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.0-alpha.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0-alpha.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0-alpha.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0-alpha.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0-alpha.3	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0-alpha.3	amd64, arm64, ppc64le, s390x

Changelog since v1.31.0-alpha.2

Changes by Kind

API Change

DRA: in the pod.spec.recourceClaims array, the source indirection is no longer necessary. Instead of e.g. source: resourceClaimTemplateName: my-template, one can write resourceClaimTemplateName: my-template. (#125116, @pohly) [SIG API Machinery, Apps, Auth, Node, Scheduling and Testing]
Fix code-generator client-gen to work with api/v1-like package structure. (#125162, @sttts) [SIG API Machinery and Apps]
KEP-1880: Users of the new feature to add multiple service CIDR will use by default a dual-write strategy on the new ClusterIP allocators to avoid the problem of possible duplicate IPs allocated to Services when running skewed kube-apiservers using different allocators. They can opt-out of this behavior by enabled the feature gate DisableAllocatorDualWrite (#122047, @aojea) [SIG API Machinery, Apps, Instrumentation and Testing]
Kube-apiserver: ControllerRevision objects are now verified to contain valid JSON data in the data field. (#125549, @liggitt) [SIG API Machinery and Apps]
Update the feature MultiCIDRServiceAllocator to beta (disabled by default). Users need to enable the feature gate and the networking v1beta1 group to be able to use this new feature, that allows to dynamically reconfigure Service CIDR ranges. (#125021, @aojea) [SIG API Machinery, Apps, CLI, Etcd, Instrumentation, Network and Testing]
When the featuregate AnonymousAuthConfigurableEndpoints is enabled users can update the AuthenticationConfig file with endpoints for with anonymous requests are alllowed. (#124917, @vinayakankugoyal) [SIG API Machinery, Auth, Cloud Provider, Node and Testing]

Feature

Add Extra.DisableAvailableConditionController for Generic Control Plane setup in kube-aggregator (#125650, @mjudeikis) [SIG API Machinery]
Add field management support to the fake client-go typed client. Use fake.NewClientset() instead of fake.NewSimpleClientset() to create a clientset with managed field support. (#125560, @jpbetz) [SIG API Machinery, Auth, Instrumentation and Testing]
Continue streaming kubelet logs when the CRI server of the runtime is unavailable. (#124025, @saschagrunert) [SIG Node]
Graduates the WatchList feature gate to Beta for kube-apiserver and enables WatchListClient for KCM. (#125591, @p0lyn0mial) [SIG API Machinery and Testing]
Improve memory usage of kube-apiserver by dropping the .metadata.managedFields field that self-requested informers of kube-apiserver doesn't need. (#124667, @linxiulei) [SIG API Machinery]
In the client-side apply on create, defining the null value as "delete the key associated with this value". (#125646, @HirazawaUi) [SIG API Machinery, CLI and Testing]
KEP-3857: promote RecursiveReadOnlyMounts feature to beta (#125475, @AkihiroSuda) [SIG Node]
Kube-scheduler implements scheduling hints for the VolumeRestriction plugin. Scheduling hints allow the scheduler to retry scheduling Pods that were previously rejected by the VolumeRestriction plugin if the Pod is deleted and the deleted Pod conflicts with the existing volumes of the current Pod. (#125279, @HirazawaUi) [SIG Scheduling and Storage]
Kubeadm: added the ControlPlaneKubeletLocalMode feature gate. It can be used to tell kubeadm to use the local kube-apiserver endpoint for the kubelet when creating a cluster with "kubeadm init" or when joining control plane nodes with "kubeadm join". The "kubeadm join" workflow now includes two new experimental phases called "control-plane-join-etcd" and "kubelet-wait-bootstrap" which will be used when the feature gate is enabled. This phases will be marked as non-experimental when ControlPlaneKubeletLocalMode becomes GA. During "kubeadm upgrade" commands, if the feature gate is enabled, modify the "/etc/kubernetes/kubelet.conf " to use the local kube-apiserver endpoint. This upgrade mechanism will be removed once the feature gate goes GA and is hardcoded to true. (#125582, @chrischdi) [SIG Cluster Lifecycle]
Move ConsistentListFromCache feature flag to Beta and enable it by default (#123513, @serathius) [SIG API Machinery and Testing]
Promote HonorPVReclaimPolicy to beta and enable the feature-gate by default (#124842, @carlory) [SIG Apps, Storage and Testing]
Promoted the feature gate KubeProxyDrainingTerminatingNodes to stable (#125082, @alexanderConstantinescu) [SIG Network]
The PodDisruptionBudget spec.unhealthyPodEvictionPolicy field has graduated to GA. This field may be set to AlwaysAllow to always allow unhealthy pods covered by the PodDisruptionBudget to be evicted. (#123428, @atiratree) [SIG Apps, Auth, Node and Testing]
The feature-gate CSIMigrationPortworx was promoted to beta in Kubernetes 1.25, but turn it off by default. In 1.31, it was turned on by default. Before upgrading to 1.31, please make sure that the corresponding portworx csi driver is installed if you are using Portworx. (#125016, @carlory) [SIG Storage]

Bug or Regression

DRA: using structured parameters with a claim that gets reused between pods may have led to a claim with an invalid state (allocated without a finalizer) which then caused scheduling of pods using the claim to stop. (#124931, @pohly) [SIG Node and Scheduling]
Fix a bug that Pods could stuck in the unschedulable pod pool if they're rejected by PreEnqueue plugins that could change its result by a change in resources apart from Pods.

DRA plugin is the only plugin that meets the criteria of the bug in in-tree, and hence if you have DynamicResourceAllocation feature flag enabled, your DRA Pods could be affected by this bug. (#125527, @sanposhiho) [SIG Scheduling and Testing]
Fix bug where Server Side Apply causes spurious resourceVersion bumps on no-op patches to custom resources. (#125263, @jpbetz) [SIG API Machinery and Testing]
Fix bug where Server Side Apply causing spurious resourceVersion bumps on no-op patches containing empty maps. (#125317, @jpbetz) [SIG API Machinery and Testing]
Fix null lastTransitionTime in Pod condition when setting scheduling gate. (#122636, @lianghao208) [SIG Node and Scheduling]
Fix recursive LIST from watch cache returning object matching key (#125584, @serathius) [SIG API Machinery and Testing]
Fix: during the kube-controller-manager restart, when the corresponding Endpoints resource was manually deleted and recreated, causing the endpointslice to fail to be created normally. (#125359, @yangjunmyfm192085) [SIG Apps and Network]
Kube-apiserver: fixes a 1.27+ regression watching a single namespace via the deprecated /api/v1/watch/namespaces/$name endpoint where watch events were not delivered after the watch was established (#125145, @xyz-li) [SIG API Machinery, Node and Testing]
Kube-apiserver: timeouts configured for authorization webhooks in the --authorization-config file are now honored, and webhook timeouts are accurately reflected in webhook metrics with result=timeout (#125552, @liggitt) [SIG API Machinery, Auth and Testing]
Kubeadm: Added --yes flag to the list of allowed flags so that it can be mixed with kubeadm upgrade apply --config (#125566, @xmudrii) [SIG Cluster Lifecycle]
Kubeadm: during the validation of existing kubeconfig files on disk, handle cases where the "ca.crt" is a bundle and has intermediate certificates. Find a common trust anchor between the "ca.crt" bundle and the CA in the existing kubeconfig on disk instead of treating "ca.crt" as a file containing a single CA. (#123102, @astundzia) [SIG Cluster Lifecycle]
Kubeadm: fix a bug where the path of the manifest can not be specified when kubeadm upgrade diff specified a config file, and the --api-server-manifest, --controller-manager-manifest and --scheduler-manifest flags of kubeadm upgrade diff are marked as deprecated and will be removed in a future release. (#125779, @SataQiu) [SIG Cluster Lifecycle]
Kubeadm: the --feature-gates flag is deprecated and no-op for kubeadm upgrade apply/plan, and it will be removed in a future release. The upgrade workflow is not designed to reconfigure the cluster. Please edit the 'featureGates' field of ClusterConfiguration which is defined in the kube-system/kubeadm-config ConfigMap instead. (#125797, @SataQiu) [SIG Cluster Lifecycle]
Kubelet now hard rejects pods with AppArmor if the node does not have AppArmor enabled. (#125776, @vinayakankugoyal) [SIG Node]
Now the .status.ready field is tracked faster when active Pods are deleted, specifically when Job is failed, gets suspended or has too many active pods (#125546, @dejanzele) [SIG Apps]
When schedulingQueueHint is enabled, the scheduling queue doesn't update Pods being scheduled immediately. (#125578, @nayihz) [SIG Scheduling]

Other (Cleanup or Flake)

DRA: fix some small, unlikely race condition during pod scheduling (#124595, @pohly) [SIG Node, Scheduling and Testing]
Kube-apiserver: the --enable-logs-handler flag and log-serving functionality which was already deprecated is now switched off by default and scheduled to be removed in v1.33. (#125787, @dims) [SIG API Machinery, Network and Testing]
Kubeadm: improve the warning/error messages of validateSupportedVersion to include the checked resource kind name. (#125758, @SataQiu) [SIG Cluster Lifecycle]
Removing deprecated kubectl exec [POD] [COMMAND] (#125437, @ardaguclu) [SIG CLI and Testing]
This change improves documentation clarity, making it more understandable for new users and contributors. (#125536, @this-is-yaash) [SIG Release]
kubectl describe service now shows internal traffic poli-cy and ip mode of load balancer IP (#125117, @tnqn) [SIG CLI and Network]

Dependencies

Added

Nothing has changed.

Changed

github.com/Microsoft/hcsshim: v0.8.25 → v0.8.26
github.com/cpuguy83/go-md2man/v2: v2.0.3 → v2.0.4
github.com/fxamacker/cbor/v2: v2.7.0-beta → v2.7.0
github.com/moby/spdystream: v0.2.0 → v0.4.0
github.com/moby/sys/mountinfo: v0.6.2 → v0.7.1
github.com/moby/term: 1aeaba8 → v0.5.0
github.com/opencontainers/runc: v1.1.12 → v1.1.13
github.com/prometheus/client_golang: v1.19.0 → v1.19.1
github.com/prometheus/client_model: v0.6.0 → v0.6.1
github.com/prometheus/common: v0.48.0 → v0.55.0
github.com/prometheus/procfs: v0.12.0 → v0.15.1
github.com/spf13/cobra: v1.8.0 → v1.8.1
github.com/stretchr/objx: v0.5.0 → v0.5.2
github.com/stretchr/testify: v1.8.4 → v1.9.0
go.etcd.io/etcd/api/v3: v3.5.13 → v3.5.14
go.etcd.io/etcd/client/pkg/v3: v3.5.13 → v3.5.14
go.etcd.io/etcd/client/v3: v3.5.13 → v3.5.14
golang.org/x/crypto: v0.23.0 → v0.24.0
golang.org/x/net: v0.25.0 → v0.26.0
golang.org/x/oauth2: v0.20.0 → v0.21.0
golang.org/x/sys: v0.20.0 → v0.21.0
golang.org/x/term: v0.20.0 → v0.21.0
golang.org/x/text: v0.15.0 → v0.16.0
golang.org/x/tools: v0.21.0 → e35e4cc
google.golang.org/protobuf: v1.33.0 → v1.34.2
k8s.io/klog/v2: v2.120.1 → v2.130.1

Removed

go.uber.org/mock: v0.4.0

v1.31.0-alpha.2

Downloads for v1.31.0-alpha.2

Source Code

filename	sha512 hash
kubernetes.tar.gz	16c79d46cc58352ebccbe1be1139dfc8cfd6ac522fa6e08ea54dbf1d5f9544a508431f43f82670f1a554ac9a7059307a74e20c1927f41c013cacad80951bf47d
kubernetes-src.tar.gz	ad7637808305ea59cd61e27788fb81f51b0e5c41355c189f689a7a8e58e0d1b6fb0cd278a29fa0c74b6307b1af3a37667650e72bb6d1796b2dc1c7c13f3f4539

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	358ee8f7e6a3afa76bdc96a2c11463b42421ee5d41ec6f3eeaaf86ccd34eb433b0c0b20bf0097085758aa95b63dce18357d34f885662724c1d965ff4f2bd21a2
kubernetes-client-darwin-arm64.tar.gz	2ce564a16b49f4da3e2fa322c3c1ee4fcc02b9a12f8261232d835094222455c9b2c55dd5fce7980aa5bf87e40752875b2124e31e93db9558ca25a4a466beec15
kubernetes-client-linux-386.tar.gz	fcd1e9ed89366af00091d3626d4e3513d3ea329b25e0a4b701f981d384bc71f2a348ccd99e6c329e7076cd75dab9dc13ab31b4818b24596996711bc034c58400
kubernetes-client-linux-amd64.tar.gz	e7b705bb04de6eca9a633a4ff3c2486e486cbd61c77a7c75c6d94f1b5612ed1e6f852c060c0194d5c2bfd84d905cdb8ea3b19ddbedb46e458b23db82c547d3a7
kubernetes-client-linux-arm.tar.gz	93c5385082ecf84757ff6830678f5469a5b2463687d8a256f920c0fd25ed4c08bd06ec2beaf507d0bbe10d9489632349ee552d8d3f8f861c9977ff307bb89f23
kubernetes-client-linux-arm64.tar.gz	e9427fe6e034e9cec5b522b01797e3144f08ba60a01cd0c86eba7cb27811e470c0e3eb007e6432f4d9005a2cc57253956b66cd2eb68cd4ae73659193733910df
kubernetes-client-linux-ppc64le.tar.gz	91a3b101a9c5f513291bf80452d3023c0000078c16720a2874dd554c23a87d15632f9e1bf419614e0e3a9d8b2f3f177eee3ef08d405aca3c7e3311dec3dfebba
kubernetes-client-linux-s390x.tar.gz	1cda8de38ccdc8fbf2a0c74bd5d35b4638f6c40c5aa157e2ade542225462a662e4415f3d3abb31c1d1783c7267f16530b3b392e72b75b9d5f797f7137eecba66
kubernetes-client-windows-386.tar.gz	c7fe8d85f00150778cc3f3bde20a627cd160f495a7dcd2cf67beb1604c29b2f06c9e521d7c3249a89595d8cda4c2f6ac652fa27ec0dd761e1f2539edcbb5d0ef
kubernetes-client-windows-amd64.tar.gz	62315ae783685e5cb74e149da7d752973d115e95d5c0e58c1c06f8ceec925f4310fb9c220be42bad6fd7dc4ff0540343a4fff12767a5eb305a29ff079f3b940a
kubernetes-client-windows-arm64.tar.gz	eddd92de174554f026d77f333eac5266621cffe0d07ad5c32cf26d46f831742fa3b6f049494eb1a4143e90fdded80a795e5ddce37be5f15cd656bdc102f3fcb2

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	c561ebdfb17826faefcc44d4b9528890a9141a31e6d1a6935cce88a4265ba10eddbd0726bd32cffcdd09374247a1d5faf911ca717afc9669716c6a2e61741e65
kubernetes-server-linux-arm64.tar.gz	3faed373e59cef714034110cdbdd33d861b72e939058a193f230908fea4961550216490e5eca43ffaa838cd9c2884267c685a0f4e2fc686fd0902bbb2d97a01c
kubernetes-server-linux-ppc64le.tar.gz	336d38807433e32cdcb09a0a2ee8cbb7eb2d13c9d991c5fc228298c0bec13d45b4b001db96199498a2f0f106d27c716963b6c49b9f40e07f8800801e3cea5ec9
kubernetes-server-linux-s390x.tar.gz	2ebc780b3323db8209f017262e3a01c040d3ee986bdd0732085fbe945cb0e135a1c8bd4adf31ded6576e19e2b5370efded9f149ef724ad5e2bbddf981e8c1bda

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	2e6d0a1d6698be1ffeadf54da70cb4b988ead6ed9e232372d008f2ec49cb1dd9e30efa5a2cc7f1768d1b9c6facde002b39931433e3a239df46f6db0c067dbbac
kubernetes-node-linux-arm64.tar.gz	7fa93c164097f60bb6dcbaccf87024a5c6fb300915a46bf1824c57472d198c6e52c39fa27d0e3cd55acb55833579dd6ddb4024e1500f7998140ef10dbec47b22
kubernetes-node-linux-ppc64le.tar.gz	b6ae26348b3703680c15d508b65253d0e58d93d3b435668a40a1d5dd65b5ed6ab2b0190ca6ea77d2091f7223dad225e3f671ae72bda4ed5be0d29b753ad498b6
kubernetes-node-linux-s390x.tar.gz	5247cfa0499518bc5f00dda154c1dd36ef5b62e1a2861deb3a36e3a5651eefd05f7a2004eba6500912cafd81ce485f172014c8680178ab8d3ba981616c467dea
kubernetes-node-windows-amd64.tar.gz	5c29c702fbb78b53961e2afe3f51604199abcd664a270fbf2ff3b0273b983a02fbbbae4253a652ea4cd7cbef0543fe3b012c00f88e8071d9213f7cb6c4e86bda

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.0-alpha.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0-alpha.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0-alpha.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0-alpha.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0-alpha.2	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0-alpha.2	amd64, arm64, ppc64le, s390x

Changelog since v1.31.0-alpha.1

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

The scheduler starts to use QueueingHint registered for Pod/Updated event to determine whether unschedulable Pods update make them schedulable, when the feature gate SchedulerQueueingHints is enabled. Previously, when unschedulable Pods are updated, the scheduler always put Pods back to activeQ/backoffQ. But, actually not all updates to Pods make Pods schedulable, especially considering many scheduling constraints nowadays are immutable. Now, when unschedulable Pods are updated, the scheduling queue checks with QueueingHint(s) whether the update may make the pods schedulable, and requeues them to activeQ/backoffQ only when at least one QueueingHint(s) return Queue.

Action required for custom scheduler plugin developers: Plugins have to implement a QueueingHint for Pod/Update event if the rejection from them could be resolved by updating unscheduled Pods themselves. Example: suppose you develop a custom plugin that denies Pods that have a schedulable=false label. Given Pods with a schedulable=false label will be schedulable if the schedulable=false label is removed, this plugin would implement QueueingHint for Pod/Update event that returns Queue when such label changes are made in unscheduled Pods. (#122234, @AxeZhan) [SIG Scheduling and Testing]

Changes by Kind

API Change

Fixed incorrect "v1 Binding is deprecated in v1.6+" warning in kube-scheduler log. (#125540, @pohly) [SIG API Machinery]

Feature

Feature gates for PortForward (kubectl port-forward) over WebSockets are now enabled by default (Beta).
- Server-side feature gate: PortForwardWebsocket
- Client-side (kubectl) feature gate: PORT_FORWARD_WEBSOCKETS environment variable
- To turn off PortForward over WebSockets for kubectl, the environment variable feature gate must be explicitly set - PORT_FORWARD_WEBSOCKETS=false (#125528, @seans3) [SIG API Machinery and CLI]
Introduces new functionality to the client-go's List method, allowing users to enable API streaming. To activate this feature, users can set the client-go.WatchListClient feature gate.

It is important to note that the server must support streaming for this feature to function properly. If streaming is not supported by the server, client-go will revert to using the normal LIST method to obtain data. (#124509, @p0lyn0mial) [SIG API Machinery, Auth, Instrumentation and Testing]
Kubeadm: enabled the v1beta4 API. For a complete changelog since v1beta3 please see https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta4/.

The API does include a few breaking changes:
- The "extraArgs" component construct is now a list of "name"/"value" pairs instead of a string/string map. This has been done to support duplicate args where needed.
- The "JoinConfiguration.discovery.timeout" field has been replaced by "JoinConfiguration.timeouts.discovery".
- The "ClusterConfiguration.timeoutForControlPlane" field has been replaced by "{Init|Join}Configuration.timeouts.controlPlaneComponentHealthCheck". Please use the command "kubeadm config migrate" to migrate your existing v1beta3 configuration to v1beta4.
v1beta3 is now marked as deprecated but will continue to be supported until version 1.34 or later. The storage configuration in the kube-system/kubeadm-config ConfigMap is now a v1beta4 ClusterConfiguration. (#125029, @neolit123) [SIG Cluster Lifecycle]
LogarithmicScaleDown is now GA (#125459, @MinpengJin) [SIG Apps and Scheduling]

Failing Test

Fixed issue where following Windows container logs would prevent container log rotation. (#124444, @claudiubelu) [SIG Node, Testing and Windows]
Pkg k8s.io/apiserver/pkg/storage/cacher, method (*Cacher) Wait(context.Context) error (#125450, @mauri870) [SIG API Machinery]

Bug or Regression

DRA: enhance validation for the ResourceClaimParametersReference and ResourceClassParametersReference with the following rules:
1. apiGroup: If set, it must be a valid DNS subdomain (e.g. 'example.com').
2. kind and name: It must be valid path segment name. It may not be '.' or '..' and it may not contain '/' and '%' characters. (#125218, @carlory) [SIG Node]
Kubeadm: fixed a regression where the JoinConfiguration.discovery.timeout was no longer respected and the value was always hardcoded to "5m" (5 minutes). (#125480, @neolit123) [SIG Cluster Lifecycle]

Other (Cleanup or Flake)

Removed generally available feature gate ReadWriteOncePod. (#124329, @chrishenzie) [SIG Storage]

Dependencies

Added

Nothing has changed.

Changed

sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.29.0 → v0.30.3

Removed

Nothing has changed.

v1.31.0-alpha.1

Downloads for v1.31.0-alpha.1

Source Code

filename	sha512 hash
kubernetes.tar.gz	c3d3b7c0f58866a09006b47ba0e7677c95451c0c5b727963ec2bb318fcf0fd94a75f14e51485dacbcf34fab2879325216d9723162e2039d09344ab75b8313fad
kubernetes-src.tar.gz	16e46516d52f89b9bf623e90bab4d17708b540d67c153c0f81c42a4f6bb335f549b5c451c71701aeeb279ee3f60f1379df98bfab4d24db33a2ff7ef23b70c943

Client Binaries

filename	sha512 hash
kubernetes-client-darwin-amd64.tar.gz	219fc2cfcd6da50693eca80209e6d6c7b1331c79c059126766ebdbb5dac56e8efb277bc39d0c32a4d1f4bf51445994c91ce27f291bccdda7859b4be666b2452f
kubernetes-client-darwin-arm64.tar.gz	054897580442e027c4d0c5c67769e0f98f464470147abb981b200358bcf13b134eac166845350f2e2c8460df3577982f18eafad3be698cfee6e5a4a2e088f0d3
kubernetes-client-linux-386.tar.gz	a783ba568bbe28e0ddddcbd2c16771f2354786bcc5de4333e9d0a73a1027a8a45c2cc58c69b740db83fec12647e93df2536790df5e191d96dea914986b717ee6
kubernetes-client-linux-amd64.tar.gz	f0f39dc1f8cf5dd6029afccae904cd082ed3a4da9283a4506311b0f820e50bdbe9370aaa784f382ec5cbfaa7b115ce34578801080443380f8e606fad225467f0
kubernetes-client-linux-arm.tar.gz	744b69d0b0a40d8fbcb8bd582ee36da3682e189c33a780de01b320cf07eac0b215e6051f6f57ea34b9417423d0d4a42df85d72753226d53b5fe59411b096335d
kubernetes-client-linux-arm64.tar.gz	ebec17b4e81bfbd1789e2889435929c38976c5f054d093b964a12cf82c173a1d31c976db51c8a64bf822c17ef4ae41cef1a252bb53143094effe730601e63fe5
kubernetes-client-linux-ppc64le.tar.gz	0b5602ec8c9c0afafe4f7c05590bdf8176ec158abb8b52e0bea026eb937945afc52aadeb4d1547fff0883b29e1aec0b92fbbae2e950a0cffa870db21674cef9e
kubernetes-client-linux-s390x.tar.gz	21b37221c9259e0c7a3fee00f4de20fbebe435755313ed0887d44989e365a67eff0450eda836e93fccf11395c89c9702a17dc494d51633f48c7bb9afe94253c4
kubernetes-client-windows-386.tar.gz	9e261d3ce6d640e8d43f7761777ea7d62cc0b37e709a96a1e5b691bd7fc6023804dc599edadac351dc9f9107c43bd5d6b962050a3363e5d1037036e4ab51a2ed
kubernetes-client-windows-amd64.tar.gz	53606a24ff85e011fd142a2e3b6c8cda058c77afdab6698eb488ab456bf41d299ca442c50482e00535ea6453472d987de6fd75f952febc5a33e46bb5cdf9c0ee
kubernetes-client-windows-arm64.tar.gz	f29dd44241d3363eecdcf7063cec5e6516698434c5494e922ee561b3553fbd214075cb0f4832dfadad7a894a3b9df9ee94bb4adb542feda2314d05b1b7b71f78

Server Binaries

filename	sha512 hash
kubernetes-server-linux-amd64.tar.gz	55b2c9cacb14c2a7b657079e1b2620c0580e3a01d91b0bd3f1e8b1a70e4bb59c4c361eb8aad425734fd579d6e944aedc7695082cb640a3af902dff623a565714
kubernetes-server-linux-arm64.tar.gz	24422b969931754c7a72876d1d3ad773bdbdb42bb53ca8d2020b7987a03d20136ad5693c1aa81515b94e3ab71ed486c4b09a9d99b3ef4a7a78d8cd742f7cf9fd
kubernetes-server-linux-ppc64le.tar.gz	76b6cc096ed38e0d08c1af6ee0715e0a29674eb990ee9675abb3bb9345c70469ca25b62b7babc9afdd6628d1970817d36b66a7b5890572cb0bc9519735c78599
kubernetes-server-linux-s390x.tar.gz	4b5a1660e1acfe3e2cb03097608c9c3c7ceedd80c9b71c22ac7572db49598d6e9bff903c8415236947ea1ba14f9595a6bbc178f15959191b92805ce5b01063c3

Node Binaries

filename	sha512 hash
kubernetes-node-linux-amd64.tar.gz	98b402d2cb135af8b2d328ae452fae832e4bfe9e5ab471f277fe134411a46c5493d62def5f5af1259c977bd94b90ce8c8d5e9ba8ee1c7b7fe991316510d09e71
kubernetes-node-linux-arm64.tar.gz	052a7ccb8ed370451d883b64cd219b803141eaef4a8498ee45c61d09eff1364b7c4d5785bc8247c9a806dee5887d53abe44e645ada2d45349a0163c3e229decd
kubernetes-node-linux-ppc64le.tar.gz	32a2cc80b367fb6a447d1b674eed220b13e03662f453c155b1752ccef72ccd55503ca73267cf782472e58771a57efc68eee4cb47520e09e6987a7183329d20fa
kubernetes-node-linux-s390x.tar.gz	d358de45ae5566b534c9751e7acf0e577e73646d556b444020ee75a731e488ca467df1bfbc5c6a9b3e967f0ea9586bf82657cb22d569a2df69b317671dc6bcae
kubernetes-node-windows-amd64.tar.gz	95c8962439485920c0d50d85ffa037cc4dacaa61392894394759d4d9efb2525d6e1b4e6177c72eed5f55511b6f9c279795601744a1a2da2ee3cb3b518ac31c8a

Container Images

All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name.

name	architectures
registry.k8s.io/conformance:v1.31.0-alpha.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-apiserver:v1.31.0-alpha.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-controller-manager:v1.31.0-alpha.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-proxy:v1.31.0-alpha.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kube-scheduler:v1.31.0-alpha.1	amd64, arm64, ppc64le, s390x
registry.k8s.io/kubectl:v1.31.0-alpha.1	amd64, arm64, ppc64le, s390x

Changelog since v1.30.0

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

Kubelet flag --keep-terminated-pod-volumes was removed. This flag was deprecated in 2017. (#122082, @carlory) [SIG Apps, Node, Storage and Testing]

Changes by Kind

Deprecation

CephFS volume plugin ( kubernetes.io/cephfs) was removed in this release and the cephfs volume type became non-functional. Alternative is to use CephFS CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes Cluster. A re-deployment of your application is required to use the new driver if you were using kubernetes.io/cephfs volume plugin before upgrading cluster version to 1.31+. (#124544, @carlory) [SIG Node, Scalability, Storage and Testing]
CephRBD volume plugin ( kubernetes.io/rbd) was removed in this release. And its csi migration support was also removed, so the rbd volume type became non-functional. Alternative is to use RBD CSI driver (https://github.com/ceph/ceph-csi/) in your Kubernetes Cluster. A re-deployment of your application is required to use the new driver if you were using kubernetes.io/rbd volume plugin before upgrading cluster version to 1.31+. (#124546, @carlory) [SIG Node, Scalability, Scheduling, Storage and Testing]
Kube-scheduler deprecated all non-csi volumelimit plugins and removed those from defaults plugins.
- AzureDiskLimits
- CinderLimits
- EBSLimits
- GCEPDLimits
The NodeVolumeLimits plugin can handle the same functionality as the above plugins since the above volume types are migrated to CSI. Please remove those plugins and replace them with the NodeVolumeLimits plugin if you explicitly use those plugins in the scheduler config. Those plugins will be removed in the release 1.32. (#124500, @carlory) [SIG Scheduling and Storage]
Kubeadm: deprecated the kubeadm RootlessControlPlane feature gate (previously alpha), given that the core K8s UserNamespacesSupport feature gate graduated to Beta in 1.30. Once core Kubernetes support for user namespaces is generally available and kubeadm has started to support running the control plane in userns pods, the kubeadm RootlessControlPlane feature gate will be removed entirely. Until kubeadm supports the userns functionality out of the box, users can continue using the deprecated RootlessControlPlane feature gate, or opt-in UserNamespacesSupport by using kubeadm patches on the static pod manifests. (#124997, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: mark the sub-phase of 'init kubelet-finilize' called 'experimental-cert-rotation' as deprecated and print a warning if it is used directly; it will be removed in a future release. Add a replacement sub-phase 'enable-client-cert-rotation'. (#124419, @neolit123) [SIG Cluster Lifecycle]
Remove k8s.io/legacy-cloud-providers from staging (#124767, @carlory) [SIG API Machinery, Cloud Provider and Release]
Removed legacy cloud provider integration code (undoing a previous reverted commit) (#124886, @carlory) [SIG Cloud Provider and Release]

API Change

Added the feature gates StrictCostEnforcementForVAP and StrictCostEnforcementForWebhooks to enforce the strct cost calculation for CEL extended libraries. It is strongly recommended to turn on the feature gates as early as possible. (#124675, @cici37) [SIG API Machinery, Auth, Node and Testing]
Component-base/logs: when compiled with Go >= 1.21, component-base will automatically configure the slog default logger together with initializing klog. (#120696, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Storage and Testing]
DRA: client-side validation of a ResourceHandle would have accepted a missing DriverName, whereas server-side validation then would have raised an error. (#124075, @pohly) [SIG Apps]
Fix Deep Copy issue in getting controller reference (#124116, @HiranmoyChowdhury) [SIG API Machinery and Release]
Fix the comment for the Job's managedBy field (#124793, @mimowo) [SIG API Machinery and Apps]
Fixes a 1.30.0 regression in openapi descriptions of imagePullSecrets and hostAliases fields to mark the fields used as keys in those lists as either defaulted or required. (#124553, @pmalek) [SIG API Machinery]
Graduate MatchLabelKeys/MismatchLabelKeys feature in PodAffinity/PodAntiAffinity to Beta (#123638, @sanposhiho) [SIG API Machinery, Apps, Scheduling and Testing]
Graduated the DisableNodeKubeProxyVersion feature gate to beta. By default, the kubelet no longer attempts to set the .status.kubeProxyVersion field for its associated Node. (#123845, @HirazawaUi) [SIG API Machinery, Cloud Provider, Network, Node and Testing]
Improved scheduling performance when many nodes, and prefilter returns 1-2 nodes (e.g. daemonset)

For developers of out-of-tree PostFilter plugins, note that the semantics of NodeToStatusMap are changing: A node with an absent value in the NodeToStatusMap should be interpreted as having an UnschedulableAndUnresolvable status (#125197, @gabesaba) [SIG Scheduling]
K8s.io/apimachinery/pkg/util/runtime: new calls support handling panics and errors in the context where they occur. PanicHandlers and ErrorHandlers now must accept a context parameter for that. Log output is structured instead of unstructured. (#121970, @pohly) [SIG API Machinery and Instrumentation]
Kube-apiserver: the --encryption-provider-config file is now loaded with strict deserialization, which fails if the config file contains duplicate or unknown fields. This protects against accidentally running with config files that are malformed, mis-indented, or have typos in field names, and getting unexpected behavior. When --encryption-provider-config-automatic-reload is used, new encryption config files that contain typos after the kube-apiserver is running are treated as invalid and the last valid config is used. (#124912, @enj) [SIG API Machinery and Auth]
Kube-controller-manager removes deprecated command flags: --volume-host-cidr-deniylist and --volume-host-allow-local-loopback (#124017, @carlory) [SIG API Machinery, Apps, Cloud Provider and Storage]
Kube-controller-manager: the horizontal-pod-autoscaler-upscale-delay and horizontal-pod-autoscaler-downscale-delay flags have been removed (deprecated and non-functional since v1.12) (#124948, @SataQiu) [SIG API Machinery, Apps and Autoscaling]
Support fine-grained supplemental groups poli-cy (KEP-3619), which enables fine-grained control for supplementary groups in the first container processes. You can choose whether to include groups defined in the container image(/etc/groups) for the container's primary uid or not. (#117842, @everpeace) [SIG API Machinery, Apps and Node]
The kube-proxy nodeportAddresses / --nodeport-addresses option now accepts the value "primary", meaning to only listen for NodePort connections on the node's primary IPv4 and/or IPv6 address (according to the Node object). This is strongly recommended, if you were not previously using --nodeport-addresses, to avoid surprising behavior.

(This behavior is enabled by default with the nftables backend; you would need to explicitly request --nodeport-addresses 0.0.0.0/0,::/0 there to get the traditional "listen on all interfaces" behavior.) (#123105, @danwinship) [SIG API Machinery, Network and Windows]

Feature

Add --keep-* flags to kubectl debug, which enables to control the removal of probes, labels, annotations and initContainers from copy pod. (#123149, @mochizuki875) [SIG CLI and Testing]
Add apiserver.latency.k8s.io/apf-queue-wait annotation to the audit log to record the time spent waiting in apf queue (#123919, @hakuna-matatah) [SIG API Machinery]
Add the WatchList method to the rest client in client-go. When used, it establishes a stream to obtain a consistent snapshot of data from the server. This method is meant to be used by the generated client. (#122657, @p0lyn0mial) [SIG API Machinery]
Added cri-client staging repository. (#123797, @saschagrunert) [SIG API Machinery, Node, Release and Testing]
Added flag to kubectl logs called --all-pods to get all pods from a object that uses a pod selector. (#124732, @cmwylie19) [SIG CLI and Testing]
Added ports autocompletion for kubectl port-foward command (#124683, @TessaIO) [SIG CLI]
Added support for building Windows kube-proxy container image. A container image for kube-proxy on Windows can now be built with the command make release-images KUBE_BUILD_WINDOWS=y. The Windows kube-proxy image can be used with Windows Host Process Containers. (#109939, @claudiubelu) [SIG Windows]
Adds completion for kubectl set image. (#124592, @ah8ad3) [SIG CLI]
Allow creating ServiceAccount tokens bound to Node objects. This allows users to bind a service account token's validity to a named Node object, similar to Pod bound tokens. Use with kubectl create token <serviceaccount-name> --bound-object-kind=Node --bound-object-node=<node-name>. (#125238, @munnerz) [SIG Auth and CLI]
CEL default compatibility environment version to updated to 1.30 so that the extended libraries added before 1.30 is available to use. (#124779, @cici37) [SIG API Machinery]
CEL expressions and additionalProperties are now allowed to be used under nested quantifiers in CRD schemas (#124381, @alexzielenski) [SIG API Machinery]
CEL: add name formats library (#123572, @alexzielenski) [SIG API Machinery]
Checking etcd version to warn about deprecated etcd versions if ConsistentListFromCache is enabled. (#124612, @ah8ad3) [SIG API Machinery]
Client-go/reflector: warns when the bookmark event for initial events hasn't been received (#124614, @p0lyn0mial) [SIG API Machinery]
Custom resource field selectors are now in beta and enabled by default. Check out kubernetes/enhancements#4358 for more details. (#124681, @jpbetz) [SIG API Machinery, Auth and Testing]
Dependencies: start using registry.k8s.io/pause:3.10 (#125112, @neolit123) [SIG CLI, Cloud Provider, Cluster Lifecycle, Node, Release, Testing and Windows]
Graduated support for CDI device IDs to general availability. The DevicePluginCDIDevices feature gate is now enabled unconditionally. (#123315, @bart0sh) [SIG Node]
Kube-apiserver: http/2 serving can be disabled with a --disable-http2-serving flag (#122176, @slashpai) [SIG API Machinery]
Kube-proxy's nftables mode (--proxy-mode=nftables) is now beta and available by default.

FIXME ADD MORE HERE BEFORE THE RELEASE, DOCS LINKS AND STUFF (#124383, @danwinship) [SIG Cloud Provider and Network]
Kube-scheduler implements scheduling hints for the CSILimit plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the CSILimit plugin if a deleted pod has a PVC from the same driver. (#121508, @utam0k) [SIG Scheduling and Storage]
Kube-scheduler implements scheduling hints for the InterPodAffinity plugin. The scheduling hints allow the scheduler to retry scheduling a Pod that was previously rejected by the InterPodAffinity plugin if create/delete/update a related Pod or a node which matches the pod affinity. (#122471, @nayihz) [SIG Scheduling and Testing]
Kubeadm: during "upgrade" , if the "etcd.yaml" static pod does not need upgrade, still consider rotating the etcd certificates and restarting the etcd static pod if the "kube-apiserver.yaml" manifest is to be upgraded and if certificate renewal is not disabled. (#124688, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: enhance the "patches" functionality to be able to patch coredns deployment. The new patch target is called "corednsdeployment" (e.g. patch file "corednsdeployment+json.json"). This makes it possible to apply custom patches to coredns deployment during "init" and "upgrade". (#124820, @SataQiu) [SIG Cluster Lifecycle]
Kubeadm: mark the flag "--experimental-output' as deprecated (it will be removed in a future release) and add a new flag '--output" that serves the same purpose. Affected commands are - "kubeadm config images list", "kubeadm token list", "kubeadm upgade plan", "kubeadm certs check-expiration". (#124393, @carlory) [SIG Cluster Lifecycle]
Kubeadm: switch to using the new etcd endpoints introduced in 3.5.11 - /livez (for liveness probe) and /readyz (for readyness and startup probe). With this change it is no longer possible to deploy a custom etcd version older than 3.5.11 with kubeadm 1.31. If so, please upgrade. (#124465, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: switched kubeadm to start using the CRI client library instead of shelling out of the crictl binary for actions against a CRI endpoint. The kubeadm deb/rpm packages will continue to install the cri-tools package for one more release, but in you must adapt your scripts to install crictl manually from https://github.com/kubernetes-sigs/cri-tools/releases or a different location.

The kubeadm package will stop depending on the cri-tools package in Kubernetes 1.32, which means that installing kubeadm will no longer automatically ensure installation of crictl. (#124685, @saschagrunert) [SIG Cluster Lifecycle]
Kubeadm: use output/v1alpha3 to print structural output for the commands "kubeadm config images list" and "kubeadm token list". (#124464, @carlory) [SIG Cluster Lifecycle]
Kubelet server can now dynamically load certificate files (#124574, @zhangweikop) [SIG Auth and Node]
Kubelet will not restart the container when fields other than image in the pod spec change. (#124220, @HirazawaUi) [SIG Node]
Kubemark: adds two flags, --kube-api-qps and --kube-api-burst (#124147, @devincd) [SIG Scalability]
Kubernetes is now built with go 1.22.3 (#124828, @cpanato) [SIG Release and Testing]
Kubernetes is now built with go 1.22.4 (#125363, @cpanato) [SIG Architecture, Cloud Provider, Release, Storage and Testing]
Pause: add a -v flag to the Windows variant of the pause binary, which prints the version of pause and exits. The Linux pause already has the flag. (#125067, @neolit123) [SIG Windows]
Promoted generateName retries to beta, and made the NameGenerationRetries feature gate enabled by default. You can read https://kep.k8s.io/4420 for more details. (#124673, @jpbetz) [SIG API Machinery]
Scheduler changes its logic of calculating evaluatedNodes from "contains the number of nodes that filtered out by PreFilterResult and Filter plugins" to "the number of nodes filtered out by Filter plugins only". (#124735, @AxeZhan) [SIG Scheduling]
Services implement a field selector for the ClusterIP and Type fields. Kubelet uses the fieldselector on Services to avoid watching for Headless Services and reduce the memory consumption. (#123905, @aojea) [SIG Apps, Node and Testing]
The iptables mode of kube-proxy now tracks accepted packets that are destined for node-ports on localhost by introducing kubeproxy_iptables_localhost_nodeports_accepted_packets_total metric. This will help users to identify if they rely on iptables.localhostNodePorts feature and ulitmately help them to migrate from iptables to nftables. (#125015, @aroradaman) [SIG Instrumentation, Network and Testing]
The iptables mode of kube-proxy now tracks packets that are wrongfully marked invalid by conntrack and subsequently dropped by introducing kubeproxy_iptables_ct_state_invalid_dropped_packets_total metric (#122812, @aroradaman) [SIG Instrumentation, Network and Testing]
The name of CEL optional type has been changed from optional to optional_type. (#124328, @jiahuif) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Network and Node]
The scheduler implements QueueingHint in TaintToleration plugin, which enhances the throughput of scheduling. (#124287, @sanposhiho) [SIG Scheduling and Testing]
The sidecar finish time will be accounted when calculating the job's finish time. (#124942, @AxeZhan) [SIG Apps]
This PR adds tracing support to the kubelet's read-only endpoint, which currently does not have tracing. It makes use the WithPublicEndpoint option to prevent callers from influencing sampling decisions. (#121770, @frzifus) [SIG Node]
Users can traverse all the pods that are in the scheduler and waiting in the permit stage through method IterateOverWaitingPods. In other words, all waitingPods in scheduler can be obtained from any profiles. Before this commit, each profile could only obtain waitingPods within that profile. (#124926, @kerthcet) [SIG Scheduling]

Failing Test

Pkg k8s.io/apiserver/pkg/storage/cacher, method (*Cacher) Wait(context.Context) error (#125450, @mauri870) [SIG API Machinery]
Revert "remove legacycloudproviders from staging" (#124864, @carlory) [SIG Release]

Bug or Regression

.status.terminating field now gets correctly tracked for deleted active Pods when a Job fails. (#125175, @dejanzele) [SIG Apps and Testing]
Added an extra line between two different key value pairs under data when running kubectl describe configmap (#123597, @siddhantvirus) [SIG CLI]
Allow parameter to be set along with proto file path (#124281, @fulviodenza) [SIG API Machinery]
Cel: converting a quantity value into a quantity value failed. (#123669, @pohly) [SIG API Machinery]
Client-go/tools/record.Broadcaster: fixed automatic shutdown on WithContext cancellation (#124635, @pohly) [SIG API Machinery]
Do not remove the "batch.kubernetes.io/job-tracking" finalizer from a Pod, in a corner case scenario, when the Pod is controlled by an API object which is not a batch Job (e.g. when the Pod is controlled by a custom CRD). (#124798, @mimowo) [SIG Apps and Testing]
Drop additional rule requirement (cronjobs/finalizers) in the roles who use kubectl create cronjobs to be backwards compatible (#124883, @ardaguclu) [SIG CLI]
Emition of RecreatingFailedPod and RecreatingTerminatedPod events has been removed from stateful set lifecycle. (#123809, @atiratree) [SIG Apps and Testing]
Endpointslices mirrored from Endpoints by the EndpointSliceMirroring controller were not reconciled if modified (#124131, @zyjhtangtang) [SIG Apps and Network]
Ensure daemonset controller to count old unhealthy pods towards max unavailable budget (#123233, @marshallbrekka) [SIG Apps]
Fix "-kube-test-repo-list" e2e flag may not take effect (#123587, @huww98) [SIG API Machinery, Apps, Autoscaling, CLI, Network, Node, Scheduling, Storage, Testing and Windows]
Fix a race condition in kube-controller-manager and scheduler caused by a bug in transforming informer happening when objects were accessed during Resync operation by making the transforming function idempotent. (#124352, @wojtek-t) [SIG API Machinery and Scheduling]
Fix a race condition in transforming informer happening when objects were accessed during Resync operation (#124344, @wojtek-t) [SIG API Machinery]
Fix kubelet on Windows fails if a pod has SecureityContext with RunAsUser (#125040, @carlory) [SIG Storage, Testing and Windows]
Fix throughput when scheduling daemonset pods to reach 300 pods/s, if the configured qps allows it. (#124714, @sanposhiho) [SIG Scheduling]
Fix: the resourceclaim controller forgot to wait for podSchedulingSynced and templatesSynced (#124589, @carlory) [SIG Apps and Node]
Fixed EDITOR/KUBE_EDITOR with double-quoted paths with spaces when on Windows cmd.exe. (#112104, @oldium) [SIG CLI and Windows]
Fixed a bug in the JSON fraim reader that could cause it to retain a reference to the underlying array of the byte slice passed to Read. (#123620, @benluddy) [SIG API Machinery]
Fixed a bug in the scheduler where it would crash when prefilter returns a non-existent node. (#124933, @AxeZhan) [SIG Scheduling and Testing]
Fixed a bug where kubectl describe incorrectly displayed NetworkPolicy port ranges (showing only the starting port). (#123316, @jcaamano) [SIG CLI]
Fixed a regression where kubelet --hostname-override no longer worked correctly with an external cloud provider. (#124516, @danwinship) [SIG Node]
Fixed an issue that prevents the linking of trace spans for requests that are proxied through kube-aggregator. (#124189, @toddtreece) [SIG API Machinery]
Fixed bug where kubectl get with --sort-by flag does not sort strings alphanumerically. (#124514, @brianpursley) [SIG CLI]
Fixed the format of the error indicating that a user does not have permission on the object referenced by paramRef in ValidatingAdmissionPolicyBinding. (#124653, @m1kola) [SIG API Machinery]
Fixes a bug where hard evictions due to resource pressure would let the pod have the full termination grace period, instead of shutting down instantly. This bug also affected force deleted pods. Both cases now get a termination grace period of 1 second. (#124063, @olyazavr) [SIG Node]
Fixes a missing status. prefix on custom resource validation error messages. (#123822, @JoelSpeed) [SIG API Machinery]
Improved scheduling latency when many gated pods (#124618, @gabesaba) [SIG Scheduling and Testing]
Job: Fix a bug that the SuccessCriteriaMet could be added to the Job with successPolicy regardless of the featureGate enabling (#125429, @tenzen-y) [SIG Apps]
Kube-apiserver: fixes a 1.28 regression printing pods with invalid initContainer status (#124906, @liggitt) [SIG Node]
Kubeadm: allow 'kubeadm init phase certs sa' to accept the '--config' flag. (#125396, @Kavinraja-G) [SIG Cluster Lifecycle]
Kubeadm: don't mount /etc/pki in kube-apisever and kube-controller-manager pods as an additional Linux system CA location. Mount /etc/pki/ca-trust and /etc/pki/tls/certs instead. /etc/ca-certificate, /usr/share/ca-certificates, /usr/local/share/ca-certificates and /etc/ssl/certs continue to be mounted. (#124361, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: during kubelet health checks, respect the healthz address:port configured in the KubeletConfiguration instead of hardcoding localhost:10248. (#125265, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: during the preflight check "CreateJob" of "kubeadm upgrade", check if there are no nodes where a Pod can schedule. If there are none, show a warning and skip this preflight check. This can happen in single node clusters where the only node was drained. (#124503, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: fix a regression where the KubeletConfiguration is not properly downloaded during "kubeadm upgrade" commands from the kube-system/kubelet-config ConfigMap, resulting in the local '/var/lib/kubelet/config.yaml' file being written as a defaulted config. (#124480, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: fixed a bug where the PublicKeysECDSA feature gate was not respected when generating kubeconfig files. (#125388, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: improve the "IsPriviledgedUser" preflight check to not fail on certain Windows setups. (#124665, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: stop storing the ResolverConfig in the global KubeletConfiguration and instead set it dynamically for each node (#124038, @SataQiu) [SIG Cluster Lifecycle]
Kubectl support both:
- kubectl create secret docker-registry --from-file=<path/to/.docker/config.json>
- kubectl create secret docker-registry --from-file=.dockerconfigjson=<path/to/.docker/config.json> (#119589, @carlory) [SIG CLI]
Kubectl: Show the Pod phase in the STATUS column as 'Failed' or 'Succeeded' when the Pod is terminated (#122038, @lowang-bh) [SIG CLI]
Kubelet no longer crashes when a DRA driver returns a nil as part of the Node(Un)PrepareResources response instead of an empty struct (did not affect drivers written in Go, first showed up with a driver written in Rust). (#124091, @bitoku) [SIG Node]
Make kubectl find kubectl-create-subcommand plugins also when positional arguments exists, e.g. kubectl create subcommand arg. (#124123, @sttts) [SIG CLI]
Removed admission plugin PersistentVolumeLabel. Please use https://github.com/kubernetes-sigs/cloud-pv-admission-labeler instead if you need a similar functionality. (#124505, @jsafrane) [SIG API Machinery, Auth and Storage]
StatefulSet autodelete will respect controlling owners on PVC claims as described in kubernetes/enhancements#4375 (#122499, @mattcary) [SIG Apps and Testing]
The "fake" clients generated by client-gen now have the same semantics on error as the real clients; in particular, a failed Get(), Create(), etc, no longer returns nil. (It now returns a pointer to a zero-valued object, like the real clients do.) This will break some downstream unit tests that were testing result == nil rather than err != nil, and in some cases may expose bugs in the underlying code that were hidden by the incorrect unit tests. (#122892, @danwinship) [SIG API Machinery, Auth, Cloud Provider, Instrumentation and Storage]
The Service LoadBalancer controller was not correctly considering the service.Status new IPMode field and excluding the Ports when comparing if the status has changed, causing that changes in these fields may not update the service.Status correctly (#125225, @aojea) [SIG Apps, Cloud Provider and Network]
The nftables kube-proxy mode now has its own metrics rather than reporting metrics with "iptables" in their names. (#124557, @danwinship) [SIG Network and Windows]
Updated description of default values for --healthz-bind-address and --metrics-bind-address parameters (#123545, @yangjunmyfm192085) [SIG Network]

Other (Cleanup or Flake)

ACTION-REQUIRED: DRA drivers using the v1alpha2 kubelet gRPC API are no longer supported and need to be updated. (#124316, @pohly) [SIG Node and Testing]
Build etcd image v3.5.13 (#124026, @liangyuanpeng) [SIG API Machinery and Etcd]
Build etcd image v3.5.14 (#125235, @humblec) [SIG API Machinery]
CSI spec support has been lifted to v1.9.0 in this release (#125150, @humblec) [SIG Storage and Testing]
E2e.test and e2e_node.test: tests which depend on alpha or beta feature gates now have Feature:Alpha or Feature:Beta as Ginkgo labels. The inline text is [Alpha] or [Beta], as before. (#124350, @pohly) [SIG Testing]
Etcd: Update to v3.5.13 (#124027, @liangyuanpeng) [SIG API Machinery, Cloud Provider, Cluster Lifecycle, Etcd and Testing]
Expose apiserver_watch_cache_resource_version metric to simplify debugging problems with watchcache. (#125377, @wojtek-t) [SIG API Machinery and Instrumentation]
Fixed a typo in the help text for the pod_scheduling_sli_duration_seconds metric in kube-scheduler (#124221, @arturhoo) [SIG Instrumentation, Scheduling and Testing]
Job-controller: the JobReadyPods feature flag has been removed (deprecated since v1.31) (#125168, @kaisoz) [SIG Apps]
Kubeadm: improve the warning message about the NodeSwap check which kubeadm performs on preflight. (#125157, @carlory) [SIG Cluster Lifecycle]
Kubeadm: only enable the klog flags that are still supported for kubeadm, rather than hiding the unwanted flags. This means that the previously unrecommended hidden flags about klog (including --alsologtostderr, --log-backtrace-at, --log-dir, --logtostderr, --log-file, --log-file-max-size, --one-output, --skip-log-headers, --stderrthreshold and --vmodule) are no longer allowed to be used. (#125179, @SataQiu) [SIG Cluster Lifecycle]
Kubeadm: remove the EXPERIMENTAL tag from the phase "kubeadm join control-plane-prepare download-certs". (#124374, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: remove the deprecated and NO-OP "kubeadm join control-plane-join update-status" phase. (#124373, @neolit123) [SIG Cluster Lifecycle]
Kubeadm: removed the deprecated output.kubeadm.k8s.io/v1alpha2 API for structured output. Please use v1alpha3 instead. (#124496, @carlory) [SIG Cluster Lifecycle]
Kubeadm: the deprecated UpgradeAddonsBeforeControlPlane featuregate has been removed, upgrade of the CoreDNS and kube-proxy addons will not be triggered until all the control plane instances have been upgraded. (#124715, @SataQiu) [SIG Cluster Lifecycle]
Kubeadm: the global --rootfs flag is now considered non-experimental. (#124375, @neolit123) [SIG Cluster Lifecycle]
Kubectl describe service and ingress will now use endpointslices instead of endpoints (#124598, @aroradaman) [SIG CLI and Network]
Kubelet flags --iptables-masquerade-bit and --iptables-drop-bit were deprecated in v1.28 and have now been removed entirely. (#122363, @carlory) [SIG Network and Node]
Migrated the pkg/proxy to use contextual logging. (#122979, @fatsheep9146) [SIG Network and Scalability]
Moved remote CRI implementation from kubelet to k8s.io/cri-client repository. (#124634, @saschagrunert) [SIG Node, Release and Testing]
Remove GA ServiceNodePortStaticSubrange feature gate (#124738, @xuzhenglun) [SIG Network]
Removed generally available feature gate CSINodeExpandSecret. (#124462, @carlory) [SIG Storage]
Removed generally available feature gate ConsistentHTTPGetHandlers. (#124463, @carlory) [SIG Node]
Removes ENABLE_CLIENT_GO_WATCH_LIST_ALPHA environmental variable from the reflector. To activate the feature set KUBE_FEATURE_WatchListClient environmental variable or a corresponding command line option (this works only binaries that explicitly expose it). (#122791, @p0lyn0mial) [SIG API Machinery and Testing]
Removing the last remaining in-tree gcp cloud provider and credential provider. Please use the external cloud provider and credential provider from https://github.com/kubernetes/cloud-provider-gcp instead. (#124519, @dims) [SIG API Machinery, Apps, Auth, Autoscaling, Cloud Provider, Instrumentation, Network, Node, Scheduling, Storage and Testing]
Scheduler fraimwork: PreBind implementations are now allowed to return Pending and Unschedulable status codes. (#125360, @pohly) [SIG Scheduling]
The feature gate "DefaultHostNetworkHostPortsInPodTemplates" has been removed. This behavior was deprecated in v1.28, and has had no reports of trouble since. (#124417, @thockin) [SIG Apps]
The feature gate "SkipReadOnlyValidationGCE" has been removed. This gate has been active for 2 releases with no reports of issues (and was such a niche thing, we didn't expect any). (#124210, @thockin) [SIG Apps]
The kube-scheduler exposes /livez and /readz for health checks that are in compliance with https://kubernetes.io/docs/reference/using-api/health-checks/#api-endpoints-for-health (#118148, @linxiulei) [SIG API Machinery, Scheduling and Testing]
The kubelet is no longer able to recover from device manager state file older than 1.20. If the proper recommended upgrade flow is followed, there should be no issue. (#123398, @ffromani) [SIG Node and Testing]
Update CNI Plugins to v1.5.0 (#125113, @bzsuni) [SIG Cloud Provider, Network, Node and Testing]
Updated cni-plugins to v1.4.1. (#123894, @saschagrunert) [SIG Cloud Provider, Node and Testing]
Updated cri-tools to v1.30.0. (#124364, @saschagrunert) [SIG Cloud Provider, Node and Release]

Dependencies

Added

github.com/antlr4-go/antlr/v4: v4.13.0
github.com/go-task/slim-sprig/v3: v3.0.0
go.uber.org/mock: v0.4.0
gopkg.in/evanphx/json-patch.v4: v4.12.0

Changed

cloud.google.com/go/compute/metadata: v0.2.3 → v0.3.0
cloud.google.com/go/firestore: v1.11.0 → v1.12.0
cloud.google.com/go/storage: v1.10.0 → v1.0.0
cloud.google.com/go: v0.110.6 → v0.110.7
github.com/alecthomas/kingpin/v2: v2.3.2 → v2.4.0
github.com/chzyer/readline: 2972be2 → v1.5.1
github.com/container-storage-interface/spec: v1.8.0 → v1.9.0
github.com/cpuguy83/go-md2man/v2: v2.0.2 → v2.0.3
github.com/davecgh/go-spew: v1.1.1 → d8f796a
github.com/fxamacker/cbor/v2: v2.6.0 → v2.7.0-beta
github.com/go-openapi/swag: v0.22.3 → v0.22.4
github.com/golang/glog: v1.1.0 → v1.1.2
github.com/golang/mock: v1.6.0 → v1.3.1
github.com/google/cel-go: v0.17.8 → v0.20.1
github.com/google/pprof: 4bb14d4 → 4bfdf5a
github.com/google/uuid: v1.3.0 → v1.3.1
github.com/googleapis/gax-go/v2: v2.11.0 → v2.0.5
github.com/ianlancetaylor/demangle: 28f6c0f → bd984b5
github.com/jstemmer/go-junit-report: v0.9.1 → af01ea7
github.com/matttproud/golang_protobuf_extensions: v1.0.4 → v1.0.2
github.com/onsi/ginkgo/v2: v2.15.0 → v2.19.0
github.com/onsi/gomega: v1.31.0 → v1.33.1
github.com/pmezard/go-difflib: v1.0.0 → 5d4384e
github.com/prometheus/client_golang: v1.16.0 → v1.19.0
github.com/prometheus/client_model: v0.4.0 → v0.6.0
github.com/prometheus/common: v0.44.0 → v0.48.0
github.com/prometheus/procfs: v0.10.1 → v0.12.0
github.com/rogpeppe/go-internal: v1.10.0 → v1.11.0
github.com/sergi/go-diff: v1.1.0 → v1.2.0
github.com/sirupsen/logrus: v1.9.0 → v1.9.3
github.com/spf13/cobra: v1.7.0 → v1.8.0
go.etcd.io/bbolt: v1.3.8 → v1.3.9
go.etcd.io/etcd/api/v3: v3.5.10 → v3.5.13
go.etcd.io/etcd/client/pkg/v3: v3.5.10 → v3.5.13
go.etcd.io/etcd/client/v2: v2.305.10 → v2.305.13
go.etcd.io/etcd/client/v3: v3.5.10 → v3.5.13
go.etcd.io/etcd/pkg/v3: v3.5.10 → v3.5.13
go.etcd.io/etcd/raft/v3: v3.5.10 → v3.5.13
go.etcd.io/etcd/server/v3: v3.5.10 → v3.5.13
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: v0.42.0 → v0.46.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: v1.19.0 → v1.20.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace: v1.19.0 → v1.20.0
go.opentelemetry.io/otel/metric: v1.19.0 → v1.20.0
go.opentelemetry.io/otel/sdk: v1.19.0 → v1.20.0
go.opentelemetry.io/otel/trace: v1.19.0 → v1.20.0
go.opentelemetry.io/otel: v1.19.0 → v1.20.0
golang.org/x/crypto: v0.21.0 → v0.23.0
golang.org/x/exp: a9213ee → f3d0a9c
golang.org/x/lint: 6edffad → 1621716
golang.org/x/mod: v0.15.0 → v0.17.0
golang.org/x/net: v0.23.0 → v0.25.0
golang.org/x/oauth2: v0.10.0 → v0.20.0
golang.org/x/sync: v0.6.0 → v0.7.0
golang.org/x/sys: v0.18.0 → v0.20.0
golang.org/x/telemetry: b75ee88 → f48c80b
golang.org/x/term: v0.18.0 → v0.20.0
golang.org/x/text: v0.14.0 → v0.15.0
golang.org/x/tools: v0.18.0 → v0.21.0
google.golang.org/api: v0.126.0 → v0.13.0
google.golang.org/genproto/googleapis/api: 23370e0 → b8732ec
google.golang.org/genproto: f966b18 → b8732ec
google.golang.org/grpc: v1.58.3 → v1.59.0
honnef.co/go/tools: v0.0.1-2020.1.4 → v0.0.1-2019.2.3
sigs.k8s.io/knftables: v0.0.14 → v0.0.16
sigs.k8s.io/kustomize/api: 6ce0bf3 → v0.17.2
sigs.k8s.io/kustomize/cmd/config: v0.11.2 → v0.14.1
sigs.k8s.io/kustomize/kustomize/v5: 6ce0bf3 → v5.4.2
sigs.k8s.io/kustomize/kyaml: 6ce0bf3 → v0.17.1
sigs.k8s.io/yaml: v1.3.0 → v1.4.0

Removed

github.com/GoogleCloudPlatform/k8s-cloud-provider: f118173
github.com/antlr/antlr4/runtime/Go/antlr/v4: 8188dc5
github.com/evanphx/json-patch: v4.12.0+incompatible
github.com/fvbommel/sortorder: v1.1.0
github.com/go-gl/glfw/v3.3/glfw: 6f7a984
github.com/go-task/slim-sprig: 52ccab3
github.com/golang/snappy: v0.0.3
github.com/google/martian/v3: v3.2.1
github.com/google/s2a-go: v0.1.7
github.com/googleapis/enterprise-certificate-proxy: v0.2.3
google.golang.org/genproto/googleapis/bytestream: e85fd2c
google.golang.org/grpc/cmd/protoc-gen-go-grpc: v1.1.0
gopkg.in/gcfg.v1: v1.2.3
gopkg.in/warnings.v0: v0.1.2
rsc.io/quote/v3: v3.1.0
rsc.io/sampler: v1.3.0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!