Coding and Dismantling Stuff

Don't thank me, it's what I do.

About the author

Russell is a .Net developer based in Lancashire in the UK.  His day job is as a C# developer for the UK's largest online white-goods retailer, DRL Limited.

His weekend job entails alternately demolishing and constructing various bits of his home, much to the distress of his fiance Kelly, 3-year-old daughter Amelie, and menagerie of pets.

TextBox

  1. Fix dodgy keywords Google is scraping from my blog
  2. Complete migration of NHaml from Google Code to GitHub
  3. ReTelnet Mock Telnet Server à la Jetty
  4. Learn to use Git
  5. Complete beta release FHEMDotNet
  6. Publish FHEMDotNet on Google Code
  7. Learn NancyFX library
  8. Pull RussPAll/NHaml into NHaml/NHaml
  9. Open Source Blackberry Twitter app
  10. Other stuff

Creating a Simple Build Pipeline with Jenkins/Hudson and NAnt

Hi all,

Anyone who's following the FhemDotNet twitter account (at the time of writing that will be... me) may have noticed a flurry of tweets the last few days.  I've been experimenting with ways to improve the build feedback and performance I'm getting from Jenkins, using multiple build jobs and the "Copy Build Artifacts" plugin. I've now got the ability to check in a code change, Jenkins will run a build job, and then fire off two test jobs simultaneously (unit tests and integration tests), tweeting passes and fails for each of the three jobs, and giving me quicker and more detailed feedback along the way.

It took a fair few stabs to get it right, so best I share my learnings.

Prerequisites

Although it took a bit of trial and error, this is really a simple task.  I'm going to make the following assumptions before we start:

  1. Source Code Repository - Your code lives in a source code repository that your CI server can access.
  2. Jenkins / Hudson CI Server - You have full access to a Jenkins CI or Hudson CI server, you can create jobs and edit existing jobs.
  3. Multiple Build/Integration Steps - You have multiple build and/or integration steps to run over the same code - for example a build process followed by one or more test processes, perhaps followed by some sort of deploy process.
  4. Copy Artifact Plugin - This plugin, available for both Jenksin and Hudson, massively simplifies the job of passing build artifacts between jobs.

Step 1 - Create your Build Job

The first step is to create your Build job.  This first job needs to check out your project, build it, and archive the build results (plus a few other bits) for use by subsequence jobs.

  1. From the Jenkins homepage, click "New Job", I've called my job "ProjectName_Build" and clicked "Build a free-style software project"
  2. On the job configuration screen that appears, first configure your source code repository.  The following is for a Subversion repository, but it doesn't matter whether you're using Git, CVS, etc.
    1. Enter your subversion repository location, note this only needs to be accessible from your build server, so if like me you're writing this blog post on a train where you can't reach your repository, no worries, your server can.
    2. A useful tip here, when using Subversion, enter a single full-stop in the "Local module directory" field, otherwise your workspace will include a "trunk" folder (or whatever the last folder is in your Subversion path) that you probably don't want.
    3. The stuff checked out from your repository will become your workspace for this build.
  3. Select the tick-box "Poll SCM", and enter the frequency you want to poll your source code repository.  I'm impatient and want to poll every minute, so I've entered "* * * * *"
  4. Next you need to configure the build steps.  For me, this was:
    1. Click "Add Build Step" and choose "Execute NAnt Script"
    2. Enter the path to your build file, if your build file is in your project root you just need to enter the filename here
    3. Indicate the "build" target.
  5. Don't select to "Archive the artifacts" just yet, let's get the project to build first.
  6. Click Save to save your changes.

Now that you've saved your build job, it should start to run immediately. You need this build to pass so that you can take a look at your workspace and make sure you're happy with it. If you have any problems with the build go back and correct these.

  1. From the build job homepage, click the "Workspace" icon, the files and folders showing here are the build artifacts that you can select to archive for your next job.
  2. Note down which files and/or folder subsequence test or deploy jobs will need.  In my case, I need to copy the configuration files in the root of my project, the "tools" folder and the "build" folder, which contains the output from my first NAnt script.
  3. Go back to your project configuration
  4. You can now select the "Archive the artifacts" option under "Post-build Actions", a text box should appear where you can specify which files and folders you want to keep for for use in subsequence build steps, in my case I enter:
    build/**, tools/**, *.*
  5. Save your configuration again, you're now ready to move onto the next step.

Step 2 - Your Test Job

This second job is a little simpler than the first.  We simply need to link the test job to your build job, get the build artifacts from the build job, and run the tests. Again, let's run through the process.

  1. From the Jenkins homepage, click "New Job" again, this call your job something like "ProjectName_Test", again select the job type "Build a free-style software project".
  2. On the job configuration screen that appears, this time the first thing is select "Build after other projects are built", and enter the name of your build project.
  3. Rather than digging code out of a repository, we're going to reuse the artifacts from the previous build job:
    1. Click "Add Build Step" and choose "Copy artifacts from another project".
    2. Enter the name of first build job you setup.
    3. For Which Build, make sure you've selected "Last successful build".
    4. All the other options can be left blank, this will copy all of the artifacts that you indicated should be archived in your build job to their original locations.
  4. Finally you need to configure the test part of your build process, again I used a Nant script to run NUnit, so:
    1. Click "Add Build Step" and choose "Execute NAnt Script"
    2. Enter the path to the NAnt script containing your test build target (in my case I've used the same build file for both).
    3. Indicate the target you want to run (in my case I've called this "test").
  5. Click Save to save your changes.

That should be it!  Now that you've saved your test job, it should run immediately using the build artifacts from the previous job, and indicate a pass or fail based purely on your tests.

Step 3 - More Jobs

This technique really comes into its own when you can run multiple sets of tests after your build job, as Jenkins will run these jobs simultaneously, giving you faster feedback.  In my case, after setting up my NUnit tests, I'm going to set up a some integration tests to run as well.  The integration testing job is exactly the same as the unit testing job, the only difference is that the NAnt script will first deploy the built website to a test server, and then run NUnit against my Selenium tests.

Hope that helps, any queries or suggestions as ever comment below.

Russ


Permalink | Comments (3)

Comments (3) -

Geoff Bullen Australia

Sunday, June 05, 2011 5:36 AM

Geoff Bullen

Good post.  Have you had a look at the build pipeline plugin?  https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin

Can be a great way of visualizing a pipeline of several steps.

russell United Kingdom

Sunday, June 05, 2011 10:54 AM

russell

Hey Geoff,

I had actually come across your plugin, but it looked a little OTT for where my project is at the moment!  I have some colleagues in the office, on the other hand, who might be really interested in this. I'll point them at it.

Thanks for the tip!

eonarts United States

Wednesday, January 04, 2012 11:52 PM

eonarts

Thanks for the great blog post - just what I needed. I'm using the plug-in mentioned to have a visual of the project.  My question for you -
How to set the calling main build job (that does the code checkout & build) as FAILED if any of the following test builds fail?

I have junit testing on the main build job but then run extensive tests on jenkins slaves. I only want the calling build to really pass if all the tests pass. Possible?

Pingbacks and trackbacks (1)+

Add comment

  Country flag

biuquote
  • Comment
  • Preview
Loading