Things I Wish I Knew Before Using Jenkins Pipelines

I started playing with Pipelines using the web interface, then hit a block as I didn’t really know the ropes.

Here’s some things I wish I’d known first:

Wrap Steps in a Node

All code that does steps in a pipeline should be wrapped in a node block:

node() {
  sh('env')
}

If the node is not specified, eg:

def myvariable='blah'

then by default it will run on the master.

You can specify the node by supplying an argument:

node('mynode') {
  [...]
}

If the pipeline code is not in a node block, then it’s run on the master in some kind of lightweight node/thread.

Checkout scm applies only when called from source

This was a gotcha for me. ‘checkout scm’ is a great single line that checks out the source the Jenkinsfile is taken from

But when updating the script in the browser this won’t work!

When storing your source in source control, you can then switch to using ‘checkout scm’. Otherwise use the ‘git’ function.

Wrap in try / catch

Your code can be wrapped in a try/catch block.

I use this along with timeout() to see whether a node is available before using it:

def nodetest() { 
  sh('echo alive on $(hostname)') 
} 
// By default we use the 'welles' node, which could be offline.
usenode='welles' 
try { 
  // Give it 5 seconds to run the nodetest function
  timeout(time: 5, unit: 'SECONDS') { 
    node(usenode) { 
      nodetest() 
    } 
  } 
} catch(err) { 
  // Uh-oh. welles not available, so use 'cage'.
  usenode='cage' 
} 
// We know the node we want to use now.
node(usenode) {
  [...]
}

Oh yeah, and functions are available to you too.

Handy, eg for seeing whether a node is available.

Wrap things in stages

Want those neat stages to show up in the Jenkins job homepage?

Then wrap your stuff in stages, eg:

[...]
stage('setupenv') {
  node(nodename) {
    sh 'mkdir -p ' + builddir
    dir(builddir) {
      checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: false, recursiveSubmodules: true, reference: '', trackingSubmodules: false]], submoduleCfg: [], userRemoteConfigs: [[url: 'https://github.com/ianmiell/shutit']]])
    }
  }
}
stage('shutit_tests') {
  node(nodename) {
    dir(builddir + '/shutit-test') {
      sh('PATH=$(pwd)/..:${PATH} ./run.sh -s tk.shutit.shutit_test shutit_branch master -l info 2>&1')
    }
  }
}
[...]

It’s a good idea to keep stages discrete, as they can be isolated from one another – for example, you could switch one stage to another node if you want (but then you might want to look into stash()ing files…).

WTF the Deal is with Pipeline Syntax vs Groovy?

Pipeline syntax may be preferable to groovy, but is newer. See here:

https://jenkins.io/blog/2016/12/19/declarative-pipeline-beta/

The docs confusingly assume a familiarity with both.

It’s Still a Bit Buggy

I tried to change the branch the Jenkinsfile was pulled from, but the old branch persisted, and it wouldn’t pick up changes from the new one. I ended up having to create a new job and delete the old one

Input

Want to force user input before continuing?

Simple:

input('OK to continue?')

But – seemed to work better for me when I had defined stages first!

Examples

There are Jenkinsfile examples here but they look a bit unloved.

Some gists were more useful to me.

This intro was pretty good too.

And the canonical reference is here.

 

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

3 Responses to Things I Wish I Knew Before Using Jenkins Pipelines

  1. tmnatsakanyan says:

    Nice post Ian, may be worth adding retry function which can be used to wrap flaky or long running steps, I think it is still missing from pipelines. I was going around it with something like this

    def withRetry(block) {
    waitUntil {
    try {
    block()
    return true;
    } catch(error) {
    echo “ERROR: Problem running the pipeline: ${error}”
    input “Stage failed, retry?”
    return false;
    }
    }
    }

  2. Jeff says:

    Are you saying that jobs will run on the master, unless in a node stanza, even when the master has been explicitly assigned 0 executor slots?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s