DataPower Buddy Release 3.2-Beta

We're pleased to announce that DPBuddy 3.2-beta is now available. The main feature of this release is the ability to execute any DPBuddy task or command against multiple devices and/or domains. Simply define the URLs of your devices using environment properties and supply a list of environment prefixes to a command or an Ant task:

dpbuddy import -file -env "dev,test"

DPBuddy will execute the command against all the devices in the list, and, in case of any errors, it would optionally rollback either just the failed domains or all the devices/domains in the list.

You can find more details in the DPBuddy documentation.

Other new features include:

The release also contains several important bug fixes.

The general availability of DPBuddy 3.2 is expected in January 2015. Meanwhile, please let us know if you're interested in becoming part of our beta program.

DPBuddy Updates for June, 2014

Updated Online Documentation

We’ve converted our documentation to the online HTML format, which is much easier to navigate than the old PDF file. Future releases of DPBuddy will be bundled with this documentation instead of PDF.

Support for DataPower Firmware Version 7

We ran our full test suite against the recently released DataPower Version 7 and did not see any issues. If you’re planning to migrate to version 7 in the future, all your DPBuddy scripts and commands should continue working.

Support for GatewayScript

Support for JavaScript (a.k.a. GatewayScript) is the most noticeable new feature in DataPower firmware Version 7. GatewayScript can be used as part of any processing policy as an alternative to the XSLT-based rules.

Please see this page for the tips on developing and deploying GatewayScript files.

DPBuddy Updates for May, 2014

New and Updated Guides and Tutorials

* Getting started with DPBuddy CLI. As this tutorial shows, getting started with DPBuddy CLI is very easy and only takes a few minutes. Download DPBuddy, follow this guide and you’ll be able to run your first automated deployment in no time.
* Automating testing of DataPower services using SoapUI. This post is a must read if you’re looking to implement automated testing of your services.
* How to run DPBuddy tasks from Jenkins. Jenkins provides a nice UI along with many other important features such as job management, log retention and security. If you’re looking to implement continuous integration, automated (or even continuous) delivery of your DataPower artifacts, or if you’re simply running all of your Ant/DPBuddy scripts from command line, follow this guide to create your Jenkins jobs to deploy to DataPower.

DPBuddy Development Update

We’re currently working on the release 3.1.1. This release will have some minor changes and a few bug fixes. It will also contain an updated HTML-based User Guide.

We’re also planning to implement the support for DataPower firmware version 7.0 as soon as it becomes available. This will include supporting DataPower GatewayScript and other new features.

In the meantime, you can always download DPBuddy 3.1 from this link and give it a try.

If you have any questions, suggestions or feature requests, please comment on this post. We appreciate your feedback.

Automating Testing of DataPower Services

Any automated build/deployment process should be accompanied by automated testing. This is required for automated (let alone for continuous) delivery of any application. Services residing in DataPower appliances should also be covered by automated testing. This can be achieved by invoking SoapUI test cases as part of an automated deployment, after all the services have been imported. This article explains how to do it.

DPBuddy Release 3.1

We're pleased to announce that DPBuddy 3.1 is now available. The main feature of this release is the DPBuddy command-line interface (CLI). All of DPBuddy's Ant tasks can now be executed as shell commands without having to create or run Ant build files.

For example, to import a file, you can run the following command:

dpbuddy import -file -assertObjectsUp -save

All attributes of DPBuddy's Ant tasks are translated into command line options, so that DPBuddy commands have the same capabilities as Ant tasks.

Another important feature is the substantially expanded "import" command/task. It provides the following new capabilities:

  • Automatic rollback to a checkpoint in case of an import failure.
  • Creating the checkpoint before the import.
  • Quiescing the domain before the import and unquiescing it after the import.
  • Checking that the ports used by the imported services are open after the import.
  • Resetting the domain's configuration before the import to clean the domain.
  • Setting the domain's comments, which could be used for version tracking.
  • Resolving template variables without having to provide an empty "transform" element.

Other new features in this release include the following:.

  • A simplified way of specifying DataPower objects in export, assertState, quiesce and other tasks/commands. You can now provide regular expressions defining object types and names via attributes instead of having to define a nested element in Ant, e.g., <dp:export file="${}" classPattern="(WSGatew.*|.*Firewall.*)"/>
  • Copy command/task now supports the "incudes" attribute/option so you don't have to specify the nested fileset.
  • Copy/command task now supports the "flushCache" attribute/option to automatically clear the XML cache after copying.
  • You can specify the environment prefix using the "env" attribute/option instead of having to use the "environment" task.
  • Export now gets rid of unused namespace declarations, so the resulting XML file is cleaner.
  • The ability to delete all objects related to a particuar service. DataPower does not do it by default, but using DPBuddy you can now clear all artifacts related to a particular service.
  • "assertOpenPorts" task/command to verify that specific ports are actually open on the device.
  • "testConnection" task.

Download DPBuddy 3.1

DPBuddy Update for April 2014

New and Updated Guides and Tutorials

If you’d like to try implementing version traceability, download the trial version of DPBuddy if you haven’t done so already.

Upcoming DPBuddy Release

We’re on track to release DPBuddy 3.0.1 later this month, April, 2014. Some of the newly added features include:

  • Ability to delete all objects related to a particular service. DataPower does not do it by default, but using DPBuddy you will be able to remove all artifacts related to a given service or multiple services.
  • Ability to update the domain’s “comments” field as part of “import” — this will simplify version tracking of deployed domains.
  • Ability to check that specific ports are actually open on the device after import. Simply provide “assertPorts” attribute/option to the “import” command and DPBuddy will make sure that the ports are active.

Other new features coming in 3.0.1 include:

  • Command line support.You’ll be able to run any DPBuddy task directly from the command line without have to go through Ant. For example, to import a file, you’ll be able to simply run this command directly from shell/DOS prompt:

    dpbuddy import -file -assertObjectsUp -save 

    Read the rest of this post »

How to Track Versions of DataPower Services

A version traceability mechanism should allow us to see versions of all our application components. An applications with UI front-end could have a “help/about” page or use other ways to show its version to an end user or to a developer. On the back-end, the version might be embedded into a manifest file or an application-specific property/configuration file. The version is typically associated with a code baseline stored in a version control repository.

This article explains how to track version of DataPower objects and services.

DataPower Secure Backup with DPBuddy

Secure backup is the only way to backup the entire DataPower device, including keys and certificates. Therefore, it is desirable to run it on a regular basis.

You can easily automate secure backup using DPBuddy. This post explains how.

Easy-to-Use Alternative to DataPower Deployment Policies

DataPower deployment policies allow for “tweaking” device and domain configuration for different environments. Let’s face it, there are always differences between environments. Sometimes, these differences are small, such as different back-end hosts, and in other cases these differences could be significant, such as different security policies.

Deployment policies do a decent job of dealing with differences between environments. Deployment policies support changing, adding and deleting configuration, so it is possible to implement fairly complex transformations.

However, dealing with deployment policies could be confusing. Deployment policy’s match rules utilize xpath, but the syntax of the rules is not pure xpath. Consider this simple deployment policy match rule:


The part before “?” looks like xpath. But what schema is this xpath based on? There is no “protocol” element in the DataPower XML management schema. The part after “?” that uses name-value parameters is even more odd. Why use this instead of proper xpath? After all, DataPower has an XML-processing engine with full xpath support, so it would certainly be more logical to rely on XML standards.

The bottom line is that while deployment policies are useful, they have limitations. They have to be developed using Deployment Policy builder in WebGUI. They can only be applied to configuration elements supported in WebGUI. For example, creating a deployment policy that updates RemoteEndpointPort of a Web Services Proxy proves to be a non-trivial task.

This is why we added support for xpath-based transformations to our DPBuddy DataPower management tool. Instead of dealing with the obscure syntax of DataPower deployment policies, developers can simply look at the configuration export file and specify an xpath expression against this file. It is also possible to specify filters based on the types and names of DataPower objects.

Read the rest of this post »

Uploading Files to DataPower from SoapUI Tests

SoapUI is the de facto standard tool for testing Web services and APIs. It is used by many DataPower developers. However, when SoapUI is used to test DataPower services that rely on XSLT files or other artifacts that have to be authored locally, having to manually upload these files to the device could substantially impair a developer’s productivity.

File uploads and many other test preparation steps can be easily automated using DPBuddy. All that is required is to invoke an Ant target containing DPBuddy’s copy task from the setup script of a SoapUI test step or a test suite. This guide explains how to do it.

How to Version Control DataPower Configuration

It is desirable to keep the configuration of DataPower objects and services under version control. The are many advantages to this approach: being able to re-install a release of an application (of which DataPower configuration could be a component), reverting to an older version of the configuration, being able to diff versions (including diffing across environments) and so on. DataPower itself has built-in support for checkpoints and configuration diffing, but it’s no match for what can be done with a version control tool, especially the one that has a light-weight branching model such as Git.

You can find more details about version-controlling DataPower configuration here.

Using Eclipse for DataPower Development

If you’re using Eclipse for developing IBM WebSphere DataPower artifacts (xslt, schemas, etc.) you can easily configure DPBuddy to automatically copy your artifacts to the appliance and do various other chores. All you need to do is to define an Ant file with DPBuddy tasks and configure the Ant file as a builder for your project.

You can find the detailed steps for configuring Eclipse and DPBuddy here.