Introduction
Overview
Teaching: 5 min
Exercises: 0 minQuestions
What is continuous integration / continuous deployment?
Objectives
Understand why CI/CD is important.
Learn what is possible with CI/CD.
Find resources to explore in more depth.
What is CI/CD?
Continuous Integration (CI), and Continuous Deployment (CD) are two related concepts in the field of DevOps, a sub-domain of software engineering.
- CI
- Continuously integrates source code changes into your repository while testing to ensure the changes do not "break" anything.
- CD
- Continuously deploys a service based on your code.
A common example of a CI/CD workflow or pipeline in a repository would be one which—after each push of a set of commits—tests the codebase (via CI) and deploys documentation (via CD) on a website.
CI/CD workflows can be configured to run after a push, after a merge-request, or on a schedule. The broader puprose of CI/CD is to automate anything repetitive that doesn’t need to be done manually, and can be thought of a labour-based manifestation of the DRY (don’t repeat yourself) principle in programming.
Catch and Release
This is just like a fishing practice for
conservationpreservation!
Breaking Changes
What does it even mean to “break” something? The idea of “breaking” something is pretty contextual. If you’re working on C++ code, then you probably want to make sure things compile and run without segfaulting at the bare minimum. If it’s Python code, maybe you have some tests with pytest that you want to make sure pass (“exit successfully”). Or if you’re working on a paper draft, you might check for grammar, misspellings, and that the document compiles from LaTeX. Whatever the use-case is, integration is about catching breaking changes.
Don’t know
pytest?To learn more about
pytestvisit its documentation or follow this training module.
Deployment
Similarly, “deployment” can mean a lot of things. Perhaps you have a Curriculum Vitae (CV) that is automatically built from LaTeX and uploaded to your website. Another case is to release Docker images of your framework that others depend on. Maybe it’s just uploading documentation. Or to even upload a new tag of your Python package on pypi. Whatever the use-case is, deployment is about releasing changes.
Workflow Automation
CI/CD is the first step to automating your entire workflow. Imagine everything you do in order to run an analysis, or make some changes. Can you make a computer do it automatically? If so, do it! The less human work you do, the less risk of making human mistakes.
Anything you can do, a computer can do better
Any command you run on your computer can be equivalently run in a CI job.
Don’t just limit yourself to thinking of CI/CD as primarily for testing changes, but as one part of automating an entire development cycle. You can trigger notifications to your cellphone, fetch/download new data, execute cron jobs, and so much more. However, for this lesson, we’ll focus primarily on setting up CI/CD with tests for existing code.
CI/CD Solutions
Now, obviously, we’re not going to make our own fully-fledged CI/CD solution. Plenty exist in the wild today, and below are just a popular few:
For this lesson, we’ll only focus on GitLab’s solution. However, be aware that all the concepts you’ll be taught today: including pipelines, stages, jobs, artifacts; all exist in other solutions by similar/different names. For example, GitLab supports two features known as caching and artifacts; but Travis doesn’t quite implement the same thing for caching and has no native support for artifacts. Therefore, while we don’t discourage you from trying out other solutions, there’s no “one size fits all” when designing your own CI/CD workflow.
Parallel lesson on GitHub CI/CD
We also have a training on GitHub actions, the CI/CD system of GitHub
Key Points
CI/CD is crucial for any reproducibility and testing.
Take advantage of automation to reduce your workload.