OCTOBER 16, 2018
What CICD tool should I use?
In our ongoing series, the Kubernetes FAQ, where we’ve been answering some commonly asked questions in the community, this week we discuss what you need to consider when choosing a CICD tool.
There are a ton of CICD tools out there to choose from - both open source solutions as well as commercial ones. In here, we highlight some of the most important considerations to make when setting up a continuous delivery pipeline.
In this post you’ll learn:
- Why you need automated pipelines
- Components of a typical deployment pipeline
- CD pipeline features to consider
- How to incorporate GitOps
Why create automated CICD pipelines?
There are a number of benefits to automating your CI/CD pipeline:
- Reduce your time to market from weeks and months to days or hours. With an automated pipeline, development teams improve both the velocity of releases as well as the quality of the code. New features and fixes added continuously in small increments can result in a product with fewer defects.
- A stronger development team. Because all stages of the delivery pipeline are available for anyone on the team to examine, improve upon and verify, a sense of ownership over the build is created that encourages strong teamwork and collaboration across your entire organization.
- Reduced risks and costs of software development. Automation encourages developers to verify code changes in stages before moving forward, reducing the chances of defects ending up in production.
- Less Work in Progress. A CD pipeline provides a rapid feedback loop starting from development through to your customers. This iteration cycle not only helps you build the right product, but it allows developers to make product improvements more quickly, leaving less works in-progress to linger.
The CNCF landscape for CICD tools
What a typical deployment pipeline looks like
A CD pipeline is made up of a few different stages; one tool will not satisfy all of these steps.
These are the steps that make up most automated pipelines:
- Write code on laptop, push it to a source code repository like Git.
- Code goes through unit, integration, and other automated tests. If these pass, a new Docker image is built.
- Image is pushed to an image repository.
- New image is rolled out to the cluster.
CD pipeline features to consider
One of the difficulties in creating a pipeline is successfully gluing together the tools that you would like to use. These are the major features to consider when selecting tools for your pipeline:
- End-to-end security
- Ability to rollback with a fully reproducible audit trail
- Built-in observability and alerting
- A fast Mean Time to Deployment as well as a Fast Mean Time to Recovery
- Simple developer experience and workflows
A secure end-to-end pipeline
Among the CICD tools on the market, some combine the CI and CD portion into one tool. Or even worse, those who have been tasked with creating a CICD pipeline will assemble a chain of hand-forged scripts that push code through the pipeline all in one direction or perform what has been dubbed as CIOPs.
You don’t want to do this for many reasons. This is an anti-pattern because cluster secrets and other credentials are left outside of the cluster, and this can be insecure. The recommended approach is to use a tool that implements a Kubernetes Operator.
A Kubernetes Operator deployed to the cluster can watch for new images that land in your repository. Taking this one step further, if your entire cluster state (via declarative manifests) are kept in Git, a `diff alert` can be the impetus for ‘pulling’ a new image from the repository and deploying it to the cluster. This is not only a more secure method of deployments, but it also provides a simpler way for developers to apply and rollback changes to production.
Rollbacks with a full audit trail
Keeping track of the diff history, as well as managing the rollbacks of new and old deployments when you are working on a large application within a team can be challenging. You need a tool that can easily handle such scenarios. It also helps if you have a fully auditable trail that can show you who did what when and that also doubles for SOC 2 compliance regulations.
Observability and alerting
What does it mean to incorporate observability into your pipeline?
In order to increase your velocity, your pipeline needs to incorporate observability in order to answer these questions:
- If a change is released automatically, how do I know it worked?
- In a complex distributed system how do I understand issues, diagnose them and manage incidents - especially when you need to rollback?
Combining Continuous Delivery with real-time observability, enables your development team to make better decisions before they deploy new features.
New features and fixes are pushed to Git and trigger the deployment pipeline, and when they are ready to be released, ideally they should be observed in real-time against the running cluster. This allows developers to make decisions based on that feedback. They can return to the beginning of the pipeline or deploy the updated image to the production cluster.
Read more in GitOps Prt. 3 - Observability.
Faster mean time to deployment and recovery
In addition to being able to deploy frequently and dependably, another important scenario to consider is recovery for when your cluster goes down. This needs to be quick, painless and possibly carried out confidently by developers who may not be Kubernetes experts.
Simple developer experience and Git workflows
To go fast in today’s cloud native world, developers have to take control of the pipeline from end to end. Gone are the days where they submit a ticket to IT and wait four days for someone to respond. It makes sense to build your pipeline from tools that developers use all the time. Tools like Git.
With GitOps there are three basic principles:
#1. Everything that can be described must be stored in Git.
By using Git as the source of truth, it is possible to observe your cluster and compare it with the desired state. The goal is to describe everything: policies, code, configuration, and even monitored events and version control it all. Keeping everything under version control enforces convergence where changes can be reapplied if at first they didn’t succeed.
#2. Don’t use kubectl directly.
As a general rule, it's not a good idea to deploy directly to the cluster using the command line utility kubectl. Many people let their CI tool drive deployment, but by doing that you’re potentially giving a notoriously hackable thing access to production.
#3. Use a Kubernetes controller that follows an operator pattern.
With a Kubernetes controller that follows the operator pattern, your cluster always stays in sync with ‘the source of truth’ via its configuration files that are checked into Git. And since the desired state of your cluster is kept in Git, it can also be observed for differences against the running cluster.
Apply GitOps workflows to your pipeline
By adopting GitOps developers can manage both infrastructure provisioning and software deployments and rollbacks through pull requests. When the source of truth differs from what’s running in the cluster, the cluster is automatically synchronized with what’s kept in Git.
Weave Cloud’s deployment agent runs inside the cluster. This means that credentials for container image registries, and the cluster API never leave the cluster’s domain. By using Git as the source of truth, the desired state and the current state can be compared and alerted on. This not only ensures that the cluster is kept up-to-date, but it also provides a way to quickly recover from disaster if the cluster melts down.
Learn more about how to implement it in your pipeline through this set of resources called GitOps: What You Need to Know.
Need help?
For the past 3 years, Kubernetes has been powering Weave Cloud, our operations as a service offering, so we couldn’t be more excited to share our knowledge and help teams embrace the benefits of cloud native tooling and git-based workflows.
Contact us for more details on our Kubernetes support packages.
Also some responces from a DEVOPISH Telegram group from fellas DevOpsers =)
Michael Traves, [21.10.18 18:01]
One of the recent articles linked to on Twitter was this - What CICD tool should I use? https://cshort.co/2q1KEP2
Michael Traves, [21.10.18 18:01]
So, I’m curious. What are people actually using?
Sathya Bhat, [21.10.18 18:05]
Jenkins for now
Jon Barber, [21.10.18 18:24]
Concourse recently
Billy Foss, [21.10.18 19:48]
Jenkins for us with custom pipeline.
Sam Briesemeister, [21.10.18 19:53]
GitLab in some teams. Jenkins in others.
Join here or there https://devopsish.com/
