As the final Kubernetes release of 2023, Kubernetes 1.29 is a new release of the popular container orchestration platform. It offers a number of new features and improvements that will help platform engineers and DevOps engineers manage their Kubernetes clusters more effectively. Here are some of the highlights of this release.
As a longstanding CNCF member, Layer5 has donated two of its open source projects to the CNCF: Meshery and Cloud Native Performance. As an end-to-end, open-source, multi-cluster Kubernetes management platform, Meshery makes Day 2 Kubernetes cluster management a breeze. Run Meshery to explore the behavorial changes of this Kubernetes release and what they really mean to you.
While there are a number of enhancements tracked in this release (72), you need to be aware that a number of these are code freezes or deprecations in 1.29. In this article, we will focus on some highlighted enhancements, and important deprecations, so that you can be confident before upgrading your clusters.
Let's breakdown new K8s features between networking and security categories.
Networking in Kubernetes 1.29
Gateway API reaches 1.0 [Stable]
The Gateway API has reached v1.0 in between Kubernetes 1.28 and 1.29. The Gateway API is the eventual successor to the Ingress API and significantly augments the Service API. The Kubernetes Service API is one of the oldest and most used APIs in Kubernetes. The Service API, in fact, was present in the very first public commit of Kubernetes in 2014, and almost everyone who deploys workloads in Kubernetes ends up using Services. Nearly ten years later in 2024, While Service API offers a kitchen sink full of functionality, it’s nearly ten year old design is limiting what can be done in Kubernetes networking.
The Gateway API takes a fresh approach in offering expressive, extensible, and role-oriented interfaces. Gateway API is expressive and extensible in that it offers core support for advanced traffic management (like traffic weighting, header-based matching) that is only possible through customizations in Ingress API. Gateway API is role-oriented in that it defines three main roles: infrastructure provider, cluster operator, and application developer, who need to work together to use and configure Kubernetes service networking.
Sidecar Containers [Beta]
The new sidecar feature enables restartable init containers and is available in alpha in Kubernetes 1.28. The concept of a “sidecar” has been part of Kubernetes since nearly the very beginning. Sidecar containers have become a common Kubernetes deployment pattern and are often used for network proxies or as part of a logging system. Until now, sidecars were a concept that Kubernetes users applied without native support. The lack of native support has caused some usage friction, which this enhancement aims to resolve.
The new feature adds a new restartPolicy field to init containers that is available when the SidecarContainers feature gate is enabled. The field is optional and, if set, the only valid value is Always. Setting this field changes the behavior of init containers as follows: The container restarts if it exits. Any subsequent init container starts immediately after the startupProbe has successfully completed instead of waiting for the restartable init container to exit. The resource usage calculation changes for the pod as restartable init container resources are now added to the sum of the resource requests by the main containers. Pod termination continues to only depend on the main containers. An init container with a restartPolicy of Always (named a sidecar) won’t prevent the pod from terminating after the main containers exit.
Beginning in Kubernetes 1.29, if your Pod includes one or more sidecar containers (init containers with an Always restart policy), the kubelet will delay sending the TERM signal to these sidecar containers until the last main container has fully terminated. The sidecar containers will be terminated in the reverse order they are defined in the Pod spec. This ensures that sidecar containers continue serving the other containers in the Pod until they are no longer needed.
The behavior of this beta capability in 1.29 is that sidecar containers that have a PreStop hook will be notified when the Pod has begun terminating by executing the PreStop hook. Once the last primary container terminates, the last started sidecar container is notified by sending a SIGTERM signal. The next sidecar (in reverse order) is notified by sending a SIGTERM signal after the previous sidecar container terminates. This continues until all sidecar containers have terminated, or the Pod’s termination grace period expires. In the latter case, all remaining containers are notified by a SIGTERM, followed by a fixed grace period of 2 seconds and finally terminated. The Pod will be terminated after that.
Note that slow termination of a main container will also delay the termination of the sidecar containers. If the grace period expires before the termination process is complete, the Pod may enter emergency termination. In this case, all remaining containers in the Pod will be terminated simultaneously with a short grace period.
Similarly, if the Pod has a preStop hook that exceeds the termination grace period, emergency termination may occur. In general, if you have used preStop hooks to control the termination order without sidecar containers, you can now remove them and allow the kubelet to manage sidecar termination automatically.
Transition SPDY to WebSockets [Alpha]
This change aims to deprecate SPDY in favor of WebSockets for Kubernetes API server communications. WebSockets provide a more modern and scalable protocol that can improve the overall reliability and maintainability of Kubernetes communications. This can enhance security by making sure that the communication protocols used by Kubernetes are robust and well-supported.
These general enhancements contribute to a more secure and efficient Kubernetes environment. By improving Kubernetes’ underlying reliability, performance, and operational control, the above features lay the groundwork for a more secure infrastructure platform capable of hosting sensitive and critical workloads.
Security in Kubernetes 1.29
Ensure Secret Pulled Images [Alpha]
Container images often contain sensitive components, making the security of image pull operations critical. The new alpha feature makes sure that images are always pulled using Kubernetes secrets of the Pod using them. This is important when Kubelet pulls an image for one Pod and another Pod is pointing to the same image. Up until now, the second Pod’s image request did not need authentication since it still used the first one’s credentials. This could lead to unauthorized access. By securing the image pull process, this enhancement prevents attackers from intercepting or tampering with container images, which is vital for maintaining the integrity of workloads.
SignedSigning Release Artifacts [Beta]
Every Kubernetes release produces a set of artifacts such as binaries, container images, documentation, and metadata. Since the 1.24 release, the artifacts have been signed as an alpha feature. In the 1.26 release, artifact signing graduates to beta to increase software supply chain security for the Kubernetes release process and mitigate man-in-the-middle attacks.
Reduction of Secret-Based Service Account Tokens [Beta]
BoundServiceAccountTokenVolume has been GA since version 1.22: Service account tokens for pods are obtained via the TokenRequest API and stored as a projected volume. The new enhancement, in beta, eliminates the need to auto-generate secret-based service account tokens. In addition, Kubernetes will warn about using auto-created secret-based service account tokens, and purge the unused ones.
Reduction of Secret-Based Service Account Tokens [Beta]
In addition to “Bound Service Account Token Improvements” enhancement and narrowing the scope of service account tokens, this improvement seeks to reduce the reliance on long-lived secret-based service account tokens. By limiting the use of these tokens, the potential attack surface is significantly diminished.
This move aligns with the broader industry trend of short-lived, just-in-time credentials that minimize the chances of token leakage leading to unauthorized access.
Structured Authentication Configuration [Alpha]
Similar to the authorization counterpart, Kubernetes 1.29 provides another alpha feature for structured configuration for authentication mechanisms. This enhancement offers users a more maintainable and secure approach to managing authentication, allowing administrators to implement complex authentication schemes more efficiently and with less room for error.
With the new enhancement, it is now possible to configure multiple OIDC providers, clients, and validation rules:
1apiVersion: apiserver.config.k8s.io/v1alpha12kind: AuthenticationConfiguration3jwt:4- issuer:5 claimValidationRules:6 ...7 - expression: 'claims.exp - claims.nbf <= 86400'8 message: total token lifetime must not exceed 24 hours9 claimMappings:10 username:11 expression: 'claims.username + ":external-user"'12 groups:13 expression: 'claims.roles.split(",")'14 …15 userValidationRules:16 - expression: "!user.username.startsWith('system:')"17 message: username cannot used reserved system: prefix18 - expression: "user.groups.all(group, !group.startsWith('system:'))"19 message: groups cannot used reserved system: prefix
Last Kubernetes release of 2023
Kubernetes is an ever-evolving platform. For those of you running workloads on Kubernetes taking detailed note of API changes and enhancements is an important activity as you endeavour to keep your clusters upgraded with release releases. A more secure, scalable, and flexible Kubernetes is our collective goal. Dign into more details about deprecation, removals, and the latest changes in the 1.29 release notes.
On behalf of the Layer5 community and all of the CNCF projects that its contributors steward, thank you to everyone who participated in this Kubernetes release, and congratulations!
As an end-to-end, open-source, multi-cluster Kuberentes management platform, Meshery makes Day 2 Kubernetes cluster management a breeze. Run Meshery to explore the behavorial changes of this Kubernetes release and what they really mean to you.
What's ahead for Kubernetes in 2024?
AI/MLOps
Generative AI came into the world with a big splash this year and helmsman, Kubernetes, is here helping navigate the AI waters. While some people suggest that the CNCF and Kubernetes ecosystem is in catch up mode in context of GenAI, they perhaps, forget that OpenAI is running a 7,500 node Kubernetes cluster in the development and hosting of GPT-3, CLIP, DALL·E, and research on other models. Where I think that Kubernetes and the CNCF can improve is in embracing and enabling AI/ML as a forerunner of our workload use cases. The CNCF has a massive developer ecosystem that will that can be enabled with MLOps and new model training projects like Kubeflow with its Kartib and KFServing sub-projects.
Looking past YAML
DZone’s annual trend report on Kubernetes in the Enterprise has 55% of respondents identifying "Maintaining YAML files” as one of the more significant, ongoing pain-points in their experience with Kubernetes. In 2024, I’m looking forward to CNCF projects like Meshery addressing this pain-point with a visual designer for Kubernetes resource and workflow configuration. The inability for some developers to either be capable of or have time to comprehend the intricacies of their virtualized applications by staring at endless lines of YAML is poised to significantly improve in 2024.
