The joys of automating your build and deployment

Hi everyone,

Alright, so you know like everyone I'm sure (if not, start) I like to browse stackoverflow and it's more subjective version programmers. Take a look at this question about the benefits of nightly builds for a one man team.

Having gone all these years on my own projects without a real build server setup -- I can say that I would never hesitate to use one for a non-trivial project, whether it was just me or N other people. Now the post above is about "nightly builds," but the precursor to that is an automated build. After that, you can run it nightly or by the hour -- there's no difference.

For me, using a tool like eg. Hudson makes your life a lot more easier. It guarantees you have a "working", reproducible copy of your project and if you want ready for deployment. It allows you to do other fun things like run your unit/integration tests on SCM changes (because you will forget!), run static code analysis tools and other metrics, and see trends over time.

We (my friend and I working on Recordings) have a separate Amazon EC2 instance running Hudson 24/7 just for that purpose. Our project is (relatively) tiny and there's only two of us. We've spent some hours setting it up -- and I wouldn't hesitate to do it again. We have it doing all kinds of stuff, such as:

• Monitoring our dev branch for changes; building and running unit tests whenever a change is detected
• Monitoring our production branch for merges (which have been previously tested); building, running unit and integration tests, minifying js/css code, and packaging the project into a WAR file for deployment. 
• Jobs to synchronize our local test databases with the production database
• A job that polls the website every so often to make sure it's still online
• A job that runs findbugs and compiles tasks (ie  TODOs) in source code
• Lastly: a job that deploys the last successful build from production to our production server

 We use this Hudson tray tracker tool to see the status of everything at a given time, so we always know what's going on.

This allows us to be as lazy as possible, because the last thing you want to do is waste your time manually going through the build process when you're trying to fix bugs or merge new features!

And finally, as I said I find this useful even for solo projects that still have actual users* (other than yourself). Having your CI server able to build your application, create installers/docs/etc., and generate a setup file ready for distribution is very handy and will save you lots of time.


* I've used Hudson for the Recordings java webapp and some smaller C# desktop applications.

Let's use hudson as our CI server on Amazon EC2

Hey everyone,

Since we're both lazy, we don't want to build our project manually. So we've been using Hudson as our CI server. Over the past few weeks we've been working on the build process and it's coming along pretty nicely. Here's our situation:

• 2 person "distributed" team
• Can't setup a physical server ourselves (lack of equipment and connection guarantees)
• Need remote access so we can both use it

Well, after searching around we decided to settle on an Amazon EC2 instance. The biggest reason is that you can try a micro instance free* for a year. That allows us to see how it performs in the meantime.

So we've set it up, and it's pretty decent so far. We have a ubuntu instance deployed running Hudson, with our jobs polling Mercurial for changes and building the project.

A couple things I've run into:

• Don't install ant via apt-get. Don't ask me why (maybe I'm doing something wrong), but when I install via the package manager it doesn't get all the required libraries -- it seems to be missing some tasks.
• CPU Usage - During the running of our unit tests CPU usage occasionally goes 'past' 100%. This really slows down the tests some times. Our tests that hit the db take about 2 minutes normally but took 14 minutes at one point.

I guess the problem might be solved by upgrading the instance, but it's not a big problem for us. Since the db tests only run on production builds (which run nightly) speed isn't vital for them. Our regular unit tests which run when the Test branch is changed run pretty fast.

And finally, I'd like to give a shout out to this hudson tray tracker application. It puts an icon on your system tray that shows the status of your hudson jobs. You can also bring up a window to run builds or view the output.


Hudson Tray Tracker.

 It's pretty convenient, although I wish it had TFS style build notifications where you get a pop up when every a build is started or completed. Overall though I can't complain.

Next time I will share the shell script we use to set up hudson on our instance -- this includes installing ant, tomcat, mysql, mercurial, and a couple other things. But we really want that process automated.







* actually, it seems I currently owe $0.31 for I/O requests. I still think it's worth it...