The master branch is considered as always in a releasable state
Each project should contain a runbook detailing:
- Prerequisites required for deploying the service
- Instruction on how to deploy and rollback the service
Deployment runbooks should either be stored within the source control of the project, or an attached wiki such as Github.
All deployment artificats should be generated by the CI server. No locally built or otherwise generated artificat should be deployed to any controlled environment. The format of the deployment code should either be an installable debian package, or a tar gzipped archive.
Method of deployment
We currently use either Puppet or Ansible for deployments. Which method to use is dependant on the type of service to be deployed. Ansible is more favourable for deployments that require a greater degree of orchestration.
All deployments should notify their actions in the Slack #releases channel.
Notification should also be sent to New Relic or a Graphite handler detailing the version released.
As deployments are not atomic, all deployments should be able to co-exist with the current live version.
The deployment process should include a canary staged deployment. This can include but is not limited to the following stages:
- Deployed to production environment only accessible internally for validation
- Deployed to a percentage of the production environment to receive live traffic
The deployment should be able to roll back leaving the production in its previous state. Specific instructions on how to revert the production environment should either be in the release plan or the service runbook.
Following a canary release a complete deployment should be actionable to complete the release process.
Non Listening Services
You should refrain from using a scheduler such as cron for running these type of services. Service style process management such as ‘upstart’ is a more manageable form.
Services such as job workers and batch processes that are not externally accessible should be monitored. See monitoring for examples.