Continuous Delivery03 Feb 2018
A basic software team collaboration workflow that uses GitHub looks like this:
Finally, if you’ve set up a Continuous Integration (CI) server (such as was developed in yesterday’s example), you’ve got a test flow:
These can be unified into this deployment diagram:
Continuous Delivery (CD) changes this diagram, where the deployed software is “the version that a team member most recently manually deployed”, to the following, where the deployed software is “the latest version that passed automated testing”:
[Later in the course, we’ll look at a couple of variants of diagram: where a CD server pulls from the repo instead of being pushed from the CI server; and where there’s more than one deployment environment.]
Twilio → MQTT Gateway: Unit Tests
In yesterday’s post, we configured Bear-as-a-Service for CI by (1) adding unit
tests, and (2) connecting the repository to a CI server (Travis). The first step
assert, PyTest, and some test functions
that we wrote. The second step involved adding a configuration file to the repo,
and using the Travis web dashboard to tell Travis about the repo.
Bear-as-a-Service doesn’t have a component that runs in the cloud (this functionality was moved to the Twilio → MQTT Gateway). So today we’ll be using Twilio → MQTT Gateway for Continuous Delivery. The first step is to add Continuous Integration to the Gateway. And the first step of that is to add unit tests.
d2057f5 adds unit testing to the Twilio MQTT
Gateway. This uses the same tools and techniques as discussed yesterday, with these differences:
twilio_mqtt_gatewaymodule needed some changes in order to run in a test environment:
- Beyond PyTest, and unit testing in general, additional mechanism is necessary to test a Flask application. Google lead me to Flask’s testing docs.
These two steps are often necessary.
- Code that’s functional may need changes in order to be testable. In this case, these changes were trivial3. In general, you may find that you need to refactor your code in order to expose the functionality that you want to test. Also in general, this refactoring improves the design of your system, even if you wouldn’t have noticed this without the impetus of testing — or if you had noticed it, but wouldn’t otherwise have paid for it.
- Frameworks that take charge of running your application often come with their own test instructions or utilities.
Twilio → MQTT Gateway: Continuous Integration
Once unit testing was set up, configuring Travis to run the repo was
straightforward. Commit #
e4aaf04 implements this configuration.
(For comparison, Commit #
c35067f to the Bear repo configures that repo for use with Travis
As you cana see, these commits are basically the same.)
You can see the Travis build page here.
Twilio → MQTT Gateway: Continuous Delivery
Finally, we set Travis to push successful builds to Heroku. This final step just applies Travis’s Heroku deployment instructions.
Install the Travis command-line interface (CLI) tool
in order to follow along. You can also do this by editing the
.travis.yml file directly,
but this gets a bit complicated when it comes to adding the encrypted Heroku API key.
We need to add three things to the Travis configuration file: the name of the host (Heroku); the name of the Heroku repository that Travis should push to; and the credential (the Heroku API key) that Travis presents to Heroku in order to authorize the push.
All of these are added to the
.travis.yml configuration file. The result
looks like this:
deploy: provider: heroku app: twilio-gateway api_key: secure: …
app is the name of the Heroku app. You can retrieve this from the Heroku
dashboard, or from the command line by reading the name of the remote
repository from the
heroku … (push) line:
$ git remote -v git remote -v heroku heroku https://git.heroku.com/twilio-gateway.git (fetch) heroku https://git.heroku.com/twilio-gateway.git (push)
In this case, the name of the app is
twilio-gateway. Add the following to
.travis.yml file, and save the file.
deploy: provider: heroku app: YOUR-APP-NAME
Since this configuration file goes in the repository, and our repositories are public, it would be a bad idea to include the unencrypted API key in the configuration. Install the Travis CLI and run the following command, to add a key that only the Travis service can decrypt:
travis encrypt $(heroku auth:token) --add deploy.api_key
deploy section of configuration file should now include:
api_key: secure: ENCRYPTED-VALUE
Commit this change, push it to GitHub, and Travis should update your app when, and only when, a new commit passes CI.
You can still push directly from your workstation to Heroku. This is faster than waiting for CI, and therefore handy for development, and against a non-production server. Faster yet is to do as much development and validation on your workstation. This is generally easy for web applications (that present web pages to the user). We’ll look at how to do this for web hooks, such as Twilio uses, tomorrow. [Spoiler: ngrok. Which happens to be written in Go.]
- Continuous Delivery web site. Jeff Humble is the author of the Continuous Delivery book, also recommended.
- Continuous Delivery essay, Martin Fowler.
- Martin Fowler’s Continuous Delivery page has additional discussion and links to videos.
- Continuous Delivery, Wikipedia
Otherwise the tests run, but never stop. This took me a little while to figure out. ↩
This took me about half an hour with print statements to figure out. Now that I’m more familiar with Python’s mocks, hopefully I’ll recognize this more quickly next time. ↩
To make, if not — because of where I am on the learning curve for these particular tools and packages — to discover. ↩