Continuous development

In the past year there have been two major productivity boons in my workflow:

  • git which I now use exclusively and which I could only use privately before (and was thus using it in a half-assed way)
  • using modules (npm, RubyGems and to some minor extension CocoaPods)

(well, these two plus going back to the terminal after so many years in GUI environments)

I needed time to get used to both of these but with the difference that git obviously is tightly integrated into development workflow whereas using modules also has significant impact on deployment.

I wrote more before about impact on deployment especially on Heroku but I was now bitten by package versions during development so I wanted to write more about that.

We as developers are either using continuous integration or should be. I have been strong proponent of it since forever (oh Ant how I don't miss your XML configuration files) though I have to admit that I haven't always been successfull in pushing it throught organizations where I've been involved, even on projects where I excersiced almost total control. But it's 2014 now and we have many, many prepackaged solutions (CodeShip, Travis-CI, Circle-CI, Xcode CI and so on). The fact is that we no longer have any excuses.

But what does continuous integration really mean? It really means continuous deployment into a non-production environment in order to shrink the gap between different work environments. I know I'm not saying anything new here (see Continuous Delivery) - I'm just exposing my realization flow.

So with that in mind what should a developer do when running tests in local? Update the package versions as that is what will any CI environment first do. Update and update and update practically every time a npm test (or similar) is run.

But this is slow even before taking into account the fact that I live in South America and suffer from speed of light limitations. So I usually don't do npm update before running the tests... and by usually I mean almost never.

This is wrong. It's wrong to import sudden changes to external dependencies and then it's doubly wrong to not take sufficient care to maintain it. So starting right now I'm adopting module versioning rules for development that I had already adopted for deployment:

  1. We assume that all modules follow the semantic versioning. If some don't (factually - I don't care about intentions but actual effects) then they are the exception to these rules and have to be handled specifically.
  2. Versions for all modules are declared in such a way as to enable automatic updates of only backward-compatible bug fix patching. In Node you do that by prefixing the version number with "~".
  3. Automatic package updating should be run after the git commit but before git push (or git merge + git push as I usually do things). This separates the development with a certain set of package versions from any updates that might be necessary after the update.
  4. Once packages are updated, run all unit tests. Only if that step passes proceed with merge and push if these are warranted.
  5. The only valid reasons to manually update a package version (switching thus to a different minor or major version) is if there is a bug fix that isn't being reflected in the patch versions or if there is some new functionality that is really needed in the project.
  6. When manually updating a package, everything needs to be committed before the change. Once the change is performed and the package has been updated, run unit tests. Only if that passes commit the changes. Update the packages one by one unless there is some higher-level depencency between them and your project.
  7. If unit tests are failing with the new package version, consider the work involved in fixing unit tests now vs forking the module and patching it on your own. Tend strongly toward updating unit tests as that usually minimizes the complete integral of maintainance time function but you might be in a hurry and forking would provide a better short-term gain especially if you are alraedy familiar with the module.


Ivan Erceg

Software shipper, successful technical co-founder, $1M Salesforce Hackathon 2014 winner