How Centris and ipt used Red Hat OpenShift to build a reliable, repeatable and resilient deployment process

Do you remember the "telephone game" from childhood? The version I played required each person to illustrate something on the next player's back, so each round produced unique and unpredictable results. It works as a party game, but in IT deployments such effects are not at all fun. In IT, you want stable, repeatable and predictable results from running software.

Red Hat customer Centris and Red Hat partner ipt have worked hard to transform what could be an unpleasant game of "telephone" into a reliable pipeline for the deployment process. In this article, I'll examine why it's important to analyze your existing process, how we designed a new process and the technologies we used to implement the future pipeline. When they introduced our new order process to customers, Centris successfully completed their first journey toward IT automation and can now confidently tackle further improvements and challenges.  

A new journey: From telephone game to full confidence

Centris is a leading provider of IT platforms and software products for health insurance companies in Switzerland. They embarked on a journey to transform a complex deployment process into a modern, fully automated product ordering application. With the support of ipt (a local Red Hat partner and service provider), Centris introduced new cloud-native technologies, such as Tekton pipelines, and new processes, such as GitOps with ArgoCD on Red Hat OpenShift, to simplify and streamline deployment of platforms and products.

The journey begins: Analyzing the existing deployment process

As a first step, we analyzed the existing deployment process by reading documentation and conducting interviews with all involved stakeholders. The result was a diagram of the current process.

The key findings in the diagram were:

• Multiple duplication of deployment manifests with custom templating mechanisms
• Multiple steps in the process with media breaks and manual copy and paste
• Involvement of non-DevOps teams slowed down the process
• No pre-deployment environment checks and no post-deployment smoke tests for verification

With these key areas for improvement, the next steps to take could be addressed.

The journey continues: Outlining a new deployment process

In designing the new process, we followed DevOps principles, including a shift-left approach. This meant eliminating media breaks and manual copy and paste, and also keeping the involvement of non-DevOps teams to an absolute minimum.

To achieve these goals, we decided to implement new technologies:

• A machine-readable product and platform description format to replace the existing Excel and Confluence-based description
• Replace configuration management scripts run by an Ops team (non-product team) with Tekton pipelines run by the product team
• Replace all custom manifests with Helm Charts and consolidate all values including secrets by introducing Sealed Secrets
• Introduce ArgoCD for a GitOps approach to platform and product deployments

These changes allowed us to begin implementation as the biggest step in the journey.

The journey gets physical: Implementing the new deployment process

The first thing we implemented was the requisite multi-tenancy capability of the new deployment process, meaning multiple ArgoCD instances with configurations for each tenant. Using a master ArgoCD instance to bootstrap the entire multi-tenant infrastructure did the trick, allowing a single command to set everything up.

While setting up the infrastructure, new components were created to handle new product and platform descriptions. With everything running on Red Hat OpenShift, we decided to write our microservices in Quarkus, because it provided the best developer experience.

In addition, two new Tekton pipelines were designed and implemented. The first validated the environment to ensure the installability of platforms and products, and also generated ArgoCD resources from the new platform and product descriptions, which were automatically applied once they were pushed to the Git repository. The second pipeline performed smoke tests, validating the current deployment to ensure everything was up, running and stable.

As a side project, a small portal was created to provide all tenants with an overview of the installed platforms and products, as well as the API documentation for each component in the products.

The journey ends: Introducing the new process to customers

Centris provides IT platforms and software products to customers, so the first "real life" deployment outside of the development environment was the milestone to achieve.

When that day arrived, everyone on the project team was on tenterhooks as the new deployment process ran for the first time. The first pipeline, creating ArgoCD resources after validating all environments and configurations, passed without a hitch. Checking the differences in the Red Hat OpenShift manifests in the ArgoCD console proved the success of this first part.

The second pipeline was more interesting because most of the project was focused on improving all kinds of tests, such as smoke tests or data tests, to increase the quality of the platforms and products. The moment when all test cases turned green was the most satisfying.

Recap the journey: What we accomplished and our key takeaways

Remember the telephone game analogy at the start of this article? For Centris, that's history. The journey has been exciting, sometimes hard, but mostly fun. It was also very educational. Here are our most important takeaways from this trip:

• Eliminate media breaks and middlemen: Use the same manifests and configurations from development to production with a single source of truth to avoid configuration drift
• Implement end-to-end automation: Use pipelines and a control loop to ensure reproducibility and to eliminate manual changes
• Ensure environmental compatibility and deployment stability: Use dependency management and consistent versioning for all components to always check for compatibility

The new process gives customers full confidence in the quality of the platform and products. Time to market has been dramatically reduced, with improved automation and stability giving developers full confidence in their process, allowing for more regular releases in the future.

The next journey: Improve and expand

With the success of the first journey, our adventure with Centris is far from being over. We still have a lot of work to do. We're extending the pipeline capabilities to support more testing technologies, replace the custom developer portal with Red Hat Developer Hub, and add more platforms, products and components to the new pipeline. The new process will serve as a role model for future developments at Centris, winning more customers and delivering superior products and services.