Telekom prepares Kubernetes 5G core with GitOps


The Kubernetes infrastructure was just the most basic part of Deutsche Telekom’s plans for a new cloud-native 5G core service; As Telekom prepares for launch, its IT teams say GitOps is just as important.

GitOps refers to a set of organizational practices and technical tools that store both application code and infrastructure as code files in the same Git version control repositories. With this approach to automating the IT infrastructure, the state of production resources is constantly synchronized with a desired state expressed in code. GitOps has reached mainstream enterprise adoption thanks to the ubiquity of containers and Kubernetes, which of course are great for managing apps and infrastructure through code, but the two are not the same.

5G cellular technology, which offers high performance and low latency, is expected to enable a significant increase in the amount of data wireless networks can transmit. This creates new types of Business applications, especially in areas like Internet of things and edge computing.

“Running a Kubernetes-based application does not mean that it is cloud-native,” said Michal Sewera, Squad Lead for the cloud-native 5G core project at Deutsche Telekom (DT), known as 5G Standalone (SA). “Cloud native to me means that you are able to survive continuous updates [and] System upgrades with no impact on customers, [and] to be able to completely detach your mindset from the underlying hardware. “

GitOps increases 5G network speed

In 2019, DT began planning core telecommunications services for the first time to run on its Kubernetes platform, which it calls “Das Schiff” or “The Ship”. This happened in part because server virtualization providers and the Kubernetes open source community had improved support for Network Function Virtualization (NFV), a set of software frameworks that enable virtual machines to support high-performance network devices such as routers and load balancers that previously required specialized hardware. DT also had to wait for telecommunications software vendors to update their apps to support VMs and containers.

By the end of 2019, DT had moved conventional IT applications to Das Schiff, but hosting 5G SA this presented a unique challenge beyond NFV’s ability to emulate hardware-based performance. The company’s software applications and software-defined infrastructure updates also had to be faster.

“One second at 5G is like an eternity,” said Sewera. “This is a huge difference to any other IT application, and not just from the point of view of the [sensitivity to] Failures, but especially when you think of runtime integration where thousands of nodes are connected and you have to respond in milliseconds. “

During the construction of Das Schiff, DT’s infrastructure team had started working with the open source Flux CD GitOps tool to manage its Kubernetes clusters, which now number in the hundreds, with a potential fleet in mind of thousands.

When Sewera’s team started planning 5G SA’s move to Das Schiff in 2020, GitOps found it to be an indispensable part of that leap. Although manual intervention from DevOps and IT infrastructure teams is rarely required, a GitOps workflow means that the ship updates itself to a large extent throughout the life cycle of each application, automatically synchronizing resources and configurations of the production environment with changes expressed in code and less prone to the human error that comes with manual testing and deployments.

“They change everything in very small steps, and there is a concept that I call non-stop acceptance testing, which means that there is no longer a fixed point in time when your app is frozen.” [for testing]said Sewera [other] Possibility to control the quality … and to react immediately to a customer request. “

DT’s path to the Kubernetes 5G core

In 2019, DT’s infrastructure team began considering various commercial Kubernetes management platforms and a self-managed environment based on upstream open source. After opting for the latter option, the infrastructure team also began experimenting with GitOps and hired Weaveworks, Flux CD’s commercial backer, to assist and advise on implementing the tool on a large scale.

“They early adopted some of the technologies we are currently using, such as the Cluster API, in their own enterprise Kubernetes platform,” said Vuk Gojnic, squad leader for DT’s Kubernetes engine. “We had other candidates for this type of role, and they were very many [HashiCorp] Terraforming. For our small team, it didn’t look like something that could be scaled back then. “

With Flux, DT now manages approximately 200 Kubernetes clusters with just 10 full-time engineers and plans to scale to thousands of clusters without adding more than one or two other members to the infrastructure team.

DT mutual fund, Telekom Investment Pool, later invested in Weaveworks in a Series C financing round of $ 36 million in 2020 along with other investors such as Ericsson Ventures and Orange Ventures.

DT’s collaboration with Weaveworks helped the company make the transition between Flux Version 1 and Flux Version 2, which first released in 2020 and added the coveted support for multi-tenancy, but marked a major overhaul for the utility.

“We had a lot of interactions with them [Weaveworks] Engineering team and gave them a lot of feedback that was built into the upstream product – a number of things specific to how [we] Manage application delivery profiles, “said Gojnic.

Weaveworks has integrated Flux version 2 into its Weave GitOps Enterprise product, which launched in June and was updated to version 2021.11 this week.

DT was an early user of Flux version 2 but signed contracts with Weaveworks to provide technical support for Flux prior to the release of Weave GitOps Enterprise, Gojnic said, and may consider the commercial platform in the future.

“We run a custom stack that we have support for,” said Gojnic. “It’s not an off-the-shelf product, but we also hope to make this use case a reality [work]. “

Beth Pariseau, Senior News Writer at TechTarget, is an award-winning IT journalism veteran. She can be reached at [email protected] or on Twitter @PariseauTT.


Comments are closed.