AWS CI/CD Tools
Last updated
Last updated
First up some background / terminology. AWS have a bunch of different products in this area grouped into their “CodeSuite” developer tools :
CodeCommit is their managed Git service. Think “GitHub Light”, useful if you want to keep your entire SDLC (Software Development LifeCycle) infrastructure in one AWS account.
CodeBuild is their CI tool / managed build service
CodeDeploy is their deployment automation tool — think EC2 services, staggered release, etc.
CodePipeline is their deployment pipeline service.
CodeStar wraps all these 4 things up into a newbie-friendly-ish combination.
CloudFormation is a “close cousin” of these services, and is the the AWS Infrastructure-as-Code (IaC) tool + service
I’m going to focus for the remainder of this article on CodeBuild and CodePipeline.
CodeBuild is what is typically referred to as a Continuous Integration / CI tool. You tell it where to get some source code from, give it a bunch of scripts to run in a shell environment (build, package, test, etc.), and it will run them in a container for you. It’s cheap, simple, does the job. You can think of it as the AWS version of Travis. You can use various different sizes of build host (small, medium, large); you can use one of the predefined images as your build environment, or you can define your own docker image to use; and you can, if you want, cache data across build runs.An example of using AWS CodePipeline
CodePipeline is a CD pipeline orchestrator. You don’t give it scripts to run — you give it a sequence of actions, which are links to other services. A typical pipeline would be:
Source → Build + Test → Deploy
The Source action can be Github, CodeCommit, etc. Build + Test can be … well, I’ll get to that. For Deploy we typically use CloudFormation (which may partially use CodeDeploy under the covers, implicitly, but ignore that for now). CodePipeline can also have manual actions — e.g. for manual testing, approval, and/or to model manual steps in a deployment value stream map.
Both CodeBuild and CodePipeline can themselves be completely defined in their own CloudFormation templates, so you can use Infrastructure-as-Code (IaC) principles for these components too. Which means your CD definitions themselves are version controlled. Fab.
Some confusion can arise when looking at these two tools since CodeBuild can either run entirely independently — defining its own Source Control configuration, OR it can be an embedded action within a CodePipeline, where Source Control config is CodePipeline’s responsibility. But since CodeBuild can run any arbitrary scripts, why would you want to add the complexity of using CodePipeline as well?
Use standalone CodeBuild if you don’t want pipeline orchestration. E.g. “Doing CD in a CI tool”. As I said above it’s basically AWS Travis.
Use CodeBuild embedded in CodePipeline if you want to be able to retry pipelines at particular points, want to spin up multiple actions in parallel (e.g. for cross region build, test or deployment), etc. CodePipeline can also be more efficient than CodeBuild by itself since it can overlap concurrent executions.
The nice thing is that there’s no inescapable decision here — you can start with (1) and migrate to (2) as needs arise (and back again.) That being said at Symphonia we typically always start with a CodePipeline instance anyway, but technique (1) is fine in many cases.
Either way you’ll very likely end up using CodeBuild to define how your application is built, packaged, and tested. CodeBuild can run any number of sequenced actions, but we recommend putting them all into a shell script so that you can more easily test your build and test flow.
If you want to dig into CodePipeline and CodeBuild a little more we have some open source repos that you might find interesting:
Simple getting started Github + CodePipeline: https://github.com/symphoniacloud/github-codepipeline
One CodePipeline deploying to multiple regions: https://github.com/symphoniacloud/multi-region-codepipeline
“Meta” CodePipeline to CD your CD: https://github.com/symphoniacloud/meta-code-pipeline