Moving towards full automation for build, test and deploy
A couple of topics which have interested me for a while now is CI (Continuous Integration) and automation. I’m introducing tools to provide developer support using these approaches for my team to enhance the experience of new application builds and regression testing. The latter is of particular important as this removes the need to hassle the original developers of applications whenever a change has to be made, to a large extent. There are also benefits in the area of handing over support more effectively to first and second line support teams. However, achieving CI and automation is a complex challenge because our applications are loosely coupled n-tier, spread across three core platforms: Liferay Portal 6.1 EE, Orbeon forms 4.0 PE and Weblogic 12c middleware (coming soon!).
For those who are not familiar with CI, this is the practice of integrating code changes frequently into a shared code branch, often your mainline (a.k.a. trunk or master). The theory is that by integrating frequently, you will be integrating smaller changes, and in larger teams this means a decreased likelihood of code branch merge conflicts. As it turns out, this is true. It is made easy with support from powerful SCM (Source Code Management) solutions like Git where branching, and more importantly merging, is a low cost activity. To get the full benefit it is recommended that you integrate your SCM with an issue tracker and branch for every issue, no matter how small, and make include issue IDs in commit messages when you commit the code changes that represent fixes. This facilitates automatic generation of release notes for builds and issue trackers are often able to tell you which commits resolved which issues.
Automation is a natural next step once you have achieved a level of CI. This works best if the automation happens on a server rather than the developer’s desktop and the builds / tests / deploys are triggered as a consequence of new code being committed to nominated branches in a common repository. With this approach you are guaranteeing that all source code is committed to the repository before any deploy takes place to a testing or production environment. This is of course if all developers agree not to deploy any code without using configured jobs on the automation server. I haven’t discussed automated testing in this post, but implementing automated testing is crucial to realising many of the benefits of automation. Especially in the areas of regression testing and effective support handover to first and second line support teams.
I would suggest watching “Jumping from Continuous Integration to Continuous Delivery with Jenkins Enterprise” by Mark Pritchard (@mqprichard) and Andrew Philips (@xebialabs) if you’re interested in learning more about how this can be implemented in software.
If you are new to CI and automation, a good place to start is Wikipedia.
- Wikipedia over of SCM (Source Code Management)
- Wikipedia overview of CI (Continuous Integration)
- Wikipedia overview of build automation
On to discussions!
There are a variety of possible solutions for a CI + automation approach, and the solution largely depends on the platforms you are building on. Automation in particular is an intrusive change to the development lifecycle, which can easily become seen as obstructive without the right level of buy in from the core stakeholders: your developers! So the only real way to implement it is through a team collaborative exercise. So my focus is on putting in place a framework that enables us to have a baseline for on-going conversation and improvement. And this is also why I am writing this blog post to a large extent. It would be great to hear from other technical leads who have attempted or achieved CI and/or automation already, particularly for the platforms mentioned. It would be great to hear about your experience, so please leave a comment!
If you work as a technical lead or are simply a motivated developer working for a local authority then it would be great to have you as a member of the IT Solutions Architecture group on the Knowledge Hub