I just finished the book, Continuous Delivery by Jez Humble and David Farley. It’s a great book, but it is a bit wordy (512 pages), so here is the gist of it.
Continuous delivery builds on a simple foundation, continuous integration,
automation, and version control. But in order to get it to work you must
version control and automate everything. It’s that simple! Version control and
automate everything! Everything means different things to different people,
but to Jez and David it means a lot.
You should version control:
- Source code and test code
- Assets, images, media files, base data, etc.
- Build scripts
- Deployment scripts
- Secret configuration settings, like keys and passwords
- Environments, development, production, test, staging, …
The first three you are hopefully doing already, but the fourth is unusual
and the fifth and sixth, I rarely see anywhere. Secret settings should be kept
in version control too, just don’t put them into the public repository. Here
are some question to ask yourself:
- Can you set up a complete development machine with one command?
- Can you set up the deployment pipeline machine with one command?
- Can you set up a fresh test environment with one command?
- Can you set up a complete production environment with one command?
- Can you upgrade the application, including data migrations, with one command?
- Can you roll it back, in case it fails, with one command?
The Deployment Pipeline
Continuous Delivery uses a Continuous Itegration server to drive the
Deployment Pipeline. In the simplest form of application, a deployment
pipeline may have only one step. This step, runs all the tests and if all goes
well it will deploy the application. But as soon as the application grows this
will take too long, the single step needs to become a pipeline.
A deployment pipeline may have arbitrary many steps and they should be as
automated as possible. A common setup of a deployment pipeline contains the
- The Commit Stage (automatic)
- The Integration Test Stage (automatic)
- The Acceptance Test Stage (automatic)
- Exploratory Testing Stage (manual)
- Perfomance Testing Stage (automatic)
- Deployment Stage (triggered manually)
The main reason for the deployment pipeline is to provide feedback as
fast as possible. The commit stage typically only contains unit tests
that run very fast. The steps after that only run if the step before
The steps above may contain everything you need, too much, or in the
wrong order. Feel free to remove and re-order as you please. If
performance testing is cheaper than exploratory testing, in your case,
switch them. There is no one-solution-fits-all when it comes to
pipelines. Customize it, to fit your needs.
One important aspect of the deployment pipeline that I rarely see is that the
final product should be created by the first commit stage and then resused by
the other stages. Very often I see that the separate stages check out and
build a new version from scratch, albeit a tagged version but still. The reuse
of the same artifact through all the steps not only ensures that the same
artifact has passed all the stages, it also speeds up the deployment pipeline,
removing redundant work.
Another thing worth noting is that the manual steps described above should be
as automatic as possible. If a tester want to test the latest build that has
passed the acceptance test stage, he should be able to do it with one click.
That one click may involve:
- Create a fresh production like environment.
- Install the selected artifact into the new envrionment.
- Apply all the relevant configuration.
If the tester decides that the build should fail or succeed, he should be able
to do that with one click. And, the feedback with the result should be fed back
into the deploynment cycle so that the developers know that it failed, and why,
or the operation guys knows that there is a new version ready to deploy.
The book is a good read and I highly recommend it. In the process of describing
continuous delivery, it provides tons of useful tips on keeping your
code and data under control. Among other things it describes.
- Release strategies, such as canary relasing and blue-green deployment.
- Test strategies for acceptance tests, capacity test, etc.
- Why not to use long-lived feature branches and how to avoid it.
- Dependency management