Jenkins: How to use shared libraries for configuration

By Gerald Mücke | January 3, 2019

Jenkins: How to use shared libraries for configuration

This article discusses how Jenkins pipelines that use shared libraries can be be updated structurally upon changes on the shared library without running a full build.

By using shared libraries common or recurring elements can be reused across pipelines which simplifies maintenance of project build pipelines. More information on the topic can be found in the Jenkins book

The library has to be configured in Jenkins’ System configuration

The shared libraries contain in it’s resources folder a set of json files to configure various build scripts. As the shared library is fetched from git right after fetching the Project’s Jenkinsfile, any data that affect the build parameters will only come into effect after the script is executed. In order to update build parameters (i.e. choice lists) by making changes to the configuration file, we decided for the following approach:

You need a trigger that is fired upon commits to the shared library. There are two options:

  • Use the Jenkins default mode, that is, if you include a shared library to your pipeline, the default behavior of Jenkins is that it triggers the build on changes on shared library, unless you import it with changelog: false flag.
  • use a dedicated job that watches changes on the shared library repository and add a trigger to your project that fires when this “watcher-job” is completed. The watcher job itself can be empty empty.

Dry-Run

Either way, you need to add to all your pipelines that should support such an update a “dryRun” mode. A dryRun contains all the steps and stages, but guards the body of each stage with a condition if the dryRun is enabled.

The following example describes such a dryRun flag as build parameter. In other jobs I named this flag “confirm”, i.e. as extra protection for unintended job runs.

booleanParam(name: 'dryRun',
             defaultValue: false, 
             description: 'Do a dry run and refresh pipeline configuration')
...

stage("name") {
	if(params.dryRun ?: false){
		//do actual work here
	}
}

The example above defaults to false if the dryRun parameter is null, which may be the case on the first execution of a job, where parameters are not yet initialized.

Shared Library default behavior

The default behavior of an included shared library is to trigger the including job on every commit to the shared library and include the changes of the shared library to the commit summary of the job. In setup with lots of jobs that all use the shared library this may cause an immediate trigger of all jobs and a saturation of the build queue. This can be be prevented when using a tagged version of the shared library or using a specific watcher job.

Watcher Job

A watcher job is a dedicated job that is triggered by every commit to the shared library. Jobs that want to get notified on these changes may use an upstream build trigger.

pipelineTriggers([
        upstream('SharedLibraryWatcher')
  ])

if the build is triggered by a shared library commit or the SharedLibraryWatcher, the dryRun is enabled. This results in the build to be executed, all stages remain, but nothing is actually executed. The only effect is, that the parameters for the next build execution are updated

def dryRun = trigger.startedBySCM() || //triggered by SharedLibWatcher or shared lib changelog
            env["OnlyRefreshParameters"] == null ||  //first run to initialize
            Boolean.valueOf(env["OnlyRefreshParameters"]) //if forced by user, setting the dry-run falg

if the build was in dryRun mode, is marked as “ABORTED” in the end to be visually distinguishable from regular builds

if (dryRun) {
	println "DRY RUN BUILD, NO STAGE IS ACTIVE!"
    currentBuild.result = 'ABORTED'
}
...