In the year Kubernetes celebrates a decade since its inception, more projects from CNCF’s incubated tier are preparing to graduate, promising more tools for the ever-expanding cloud-native ecosystem. The Backstage community worked on a more robust architecture, and Crossplane aimed to improve its development experience (DX). KubeFlow and Volcano, both tools that promise to improve AI adoption within the Kubernetes ecosystem, are working on easier installation and more features.
Jorge Castro, head of open source community and developer relations at CNCF, said at the event that “Incubated CNCF projects are toys that are already vouched for by the community, having gone through several rounds of feedback“.
Backstage is an internal developer portal that aims to be a “single pane of glass” that brings together all the information a developer needs to submit, PRs, build pipelines, documentation and more, according to Timbonicus Hansen, a staff engineer at Spotify. The project traces its roots back to ten years ago when Spotify embarked on a journey to remove the biggest obstacle identified in one of their developer surveys: context switching.
The biggest changes currently planned for Backstage are structural: the team is overhauling the system, both front-end and back-end, to provide an improved way to create plugins. The clear goal is to graduate as soon as possible.
Although the project was the clear winner in terms of the number of end-user contributions to the CNCF project in the last 4 months, the main contributions still come from Spotify. An important prerequisite for graduating from CNCF is that contributions come from multiple companies. Recently, Red Hat joined the effort.
Vulcan William Wang, an architect at Huawei and a contributor to volcano.sh, defined it as a collective tool for the cloud-native era. It is built on top of Kube Edge which provides mechanisms such as advanced scheduling rules (fair sharing, topology scheduling, etc.), mechanisms to include SLAs, preempt and backfill mechanisms.
He also highlighted the importance of serial processing in fields such as genomics, machine learning or bioinformatics.
He points to the latest version’s implementation of job queues: “Even if it’s a simple feature, it’s necessary when considering batch processing, and it wasn’t available in the Kubernetes ecosystem.” According to contributors from both teams, one project that Volcano seamlessly integrates with is KubeFlow.
KubeFlow is a project that aims to “abstract the details of infrastructure in the modern era of intelligent applications”. Being able to give ML teams a way to take advantage of all the portability, scalability, and composability of K8s without the hassle of being a Kubernetes expert,” said Chase Christensen, employee solutions engineer for TileDB and contributor to KubeFlow.
With KubeFlow, teams can move between ML frameworks, use Jupyter notebooks, and design hyperparameter tuning jobs, without leaving the context of the framework.
Targeted for a mid-year release are the Kubeflow model registry, new LLM fine-tuning APIs, support for Kubernetes 1.29, and pipelines merged into a single GitHub repo. Asked about community feedback in the age of artificial intelligence, he said: “We’ve also heard a lot of feedback about improving the install experience and are discussing improving that as well as compatibility with distributions.”
CrossPlane’with the mission is to enable provisioning and management of cloud resources in a more efficient way directly from your Kubernetes cluster. One additional benefit, according to Ezgi Demirel, senior distributed systems engineer at Upbound, is the identification of duplicate service requests that are then throttled, ensuring that exactly the right number of resources are created. Another added benefit is the ability to import using Terraform schemas (currently neither OpenTofu nor Pullumi are supported.)
CrossPlane’with the mission is to enable provisioning and management of cloud resources in a more efficient way directly from your Kubernetes cluster. One additional benefit, according to Ezgi Demirel, senior distributed systems engineer at Upbound, is the identification of duplicate service requests that are then throttled, ensuring that exactly the right number of resources are created. Another added benefit is the ability to import using Terraform schemas (currently neither OpenTofu nor Pullumi are supported.)
When asked about the most important feature of the latest version, she emphasized the function of the composition. They are called by Crossplane to determine which resources it should create when you create a composite resource. The feature is in beta starting with version 1.14.
Demirel also mentioned that when the project applied for graduation, it aimed to consolidate the platform based on lessons learned so far: improving DX and adding observation capabilities. In this way, he believes that the project will be more suitable for more mature production systems.
One common theme that can be seen in all the incubated projects is the pursuit of better usability, developer experience, and more robust architectures. As more CNCF projects are completed, more capabilities are added to the natural cloud landscape. CNCF’s goal is to ensure that it is ready for a future full of challenges: from the rapid adoption of artificial intelligence to the growing problem of the carbon footprint of cloud infrastructure.