Automating DataPower Build/Deployment with Jenkins and DPBuddy

You can easily automate your DataPower deployments with Jenkins and DPBuddy. DPBuddy comes with a comprehensive set of Ant tasks, so you can simply create an Ant script defining your build/deployment process and then invoke the script from Jenkins using the “Invoke Ant” build step.

Before getting started with Jenkins and DPBuddy, we recommend putting all your DataPower-related artifacts under version control along with the Ant script(s) that will be used for deploying the DataPower artifacts. These artifacts should include DataPower configuration, XSLT files, certificates (but not the keys) and so on.

Once this is accomplished, you’ll be able to utilize a typical Jenkins build workflow — checkout files from version control, build, run tests and deploy. In the case of DataPower, deploying configuration and associated files into a domain for testing can be considered a “build” step. Once the testing is done, the domain can be exported and the result of the export can be promoted to other environments (the deployment step).

You’ll need to create one Jenkins job to do the build and one or multiple jobs to promote the build.

This post is focusing on setting up the build job; we’ll cover promotion/deployment jobs and creation of the build/deployment pipeline in future posts.

You should define your DataPower connection properties and other variables in a property file(s) that will be loaded by the build script. You could also use environment prefixes and the “dp:environment” task to point to different appliances and domains from the same build script/Jenkins job. In our examples we use the <device name>.<domain name> prefix to denote the target for our Ant script (and for all the DPBuddy tasks executed by the script). The property file should also be stored under version control. You can find the sample build script here.

Once you’re done with the preparation, creating a job in Jenkins to manage the build is straightforward:

• Create a job in Jenkins and define your version control configuration to checkout your DataPower files and the build script (if needed, install the appropriate SCM plugin).
• Add the “Invoke Ant” build step and point it to your Ant script. Specify the main target that will be used for the build. Our sample build script uses the “clean.deploy.services” target for the build.
• Your Ant script should be exporting the entire domain using “dp:export” task as the last step of the build process. The output of the export should be saved so that it can later be promoted from another Jenkins job. To do so, add “Archive the artifacts” post-build action and define the files to the archive. Use a pattern such as this one: “**/builds/*.zip”.

If you want to use the same job to build/deploy to different appliances/domains, you can parameterize your job:

• Check “This build is parameterized” and add a “Choice” parameter. You can name it “Environment”. List all the environment prefixes that you used in the DPBuddy’s properties file. For example, “devtest.dev”, “devtest.test”, “qa.test”, etc.
• You can also add a “Password” parameter if you don’t want to store passwords in the properties file for security reasons. Regardless of your password management mechanism, install Mask Passwords Plugin and make sure that the passwords are masked in the Jenkins console output (DPBuddy itself never prints passwords).
• Add a property to your “Invoke Ant” step to pass the environment parameter’s value to Ant, e.g.,” env=${Environment}”. Make sure that in your build file the “dp:environment” tasks uses this property as an environment prefix:

  <dp:environment prefix="${env}"/>

You will now be able to select the target appliance/domain for your build whenever you run the job.

If you’re using DPBuddy’s version tracking capabilities or a DPBuddy audit, you can provide release information from Jenkins to your build. Define an appropriate property in your build script (e.g., “dp.release.info”) and initialize it in Jenkins as part of the “Invoke Ant” step configuration. For example, “dp.release.info=2.0.1.${BUILD_NUMBER}.${SVN_REVISION}” (see more details about Jenkins variables here). DPBuddy will inject this information into the comments field of your services in DataPower.

You can also provide Jenkins user information to DPBuddy’s audit facility. Install Build User Vars Plugin and define the following property in your Jenkins “Invoke Ant” configuration: “dp.local.user=${BUILD_USER_ID}”. DPBuddy will automatically populate the local user field in its audit file with this information. To enable audit in DPBuddy, set “dp.audit=true” in the properties file for your build.

You may also want to run your SoapUI tests as part of the build as described here. You can use Jenkins to generate the test report based on your SoapUI tests. Simply add “Publish JUnit test report” post-build action and specify the location of the XML report files produced by SoapUI, e.g., “**/testReports/*.xml”.

After everything is configured, your build page may look like this (we’ll cover downstream projects in a future post):

Finally, you can use the same Jenkins job for continuous integration by adding the “Poll SCM” build trigger to the job’s configuration. You may need to create a separate domain/environment for running continuous integration jobs so they don’t collide with the manual build process.

In real life, a build/deployment process would include other components besides just DataPower, such as backend Java EE applications, message broker (integration bus) applications, database alter scripts, and so on. Most likely, you will need to create several different jobs to handle these components. In a simple case, however, some of the components could be built/deployed as part of the job used to build DataPower components. This can be accomplished by adding build steps for building/deploying these components.

If you’re interested in automating your DataPower builds and deployments, download a trial version of DPBuddy and give it a try.