How to Version Control DataPower Configuration

It is desirable to keep the configuration of DataPower objects and services under version control. The are many advantages to this approach: being able to re-install a release of an application (of which DataPower configuration could be a component), reverting to an older version of the configuration, being able to diff versions (including diffing across environments) and so on. DataPower itself has built-in support for checkpoints and configuration diffing, but it’s no match for what can be done with a version control tool, especially the one that has a light-weight branching model such as Git.

We should also point out that calling exported DataPower objects and services “configuration” is a bit misleading. These objects and services contain policies and rules, so they should be treated as code (including baselining them as part of an application release), and not purely as configuration.

A DataPower configuration could be exported using WebGUI and then manually checked in. But here are several problems with this approach:

• The exported configuration could contain shared objects, such as a default XML manager, that are not relevant to the declarations of objects and services that we’re interested in. Besides, we may not want to make default objects part of the subsequent import. Instead, we may choose to rely on the default/shared objects already present in the target domain.
• The configuration file could contain environment-specific values, such as a proxy port in the XML firewall configuration. Saving these values “as is” makes tracking changes more difficult.
• If the export was done using XML format (as opposed to ZIP), the configuration is going to contain embedded base64-encoded files.

DPBuddy’s export task easily solves all these problems. This task automatically removes the embedded files and replaces the environment-specific values with variables. As an added bonus, DPBuddy formats exported files using proper indentation and removes unused namespace declarations. In other words, the export task produces a clean “templatized” output file that can then be immediately checked in.

Let’s look at an example:

<dp:export file="./XMLFirewall.xml">
    <exportObject class="XMLFirewallService" name="testFirewall" />
    <transform>
        <dpExclude class="(HTTPUserAgent|XMLManager)" name="default"/>
        <setText xpath="//LocalPort" value="$${xmlfirewall.port}" />
    </transform>
</dp:export>

This task does a few things:

• It exports an XML firewall named “testFirewall”.
• It removes the definitions of default user agent and XML manager.
• It replaces the value of the LocalPort with the Ant variable “xmlfirewall.port” — notice how we had to escape the dollar sign to prevent Ant from attempting to resolve this variable.
• As was mentioned above, this task will also delete the base64-encoded files and indent the output.

A fragment of the resulting file looks like this:

<XMLFirewallService name="testFirewall">
    <mAdminState>enabled</mAdminState>
    <LocalAddress>0.0.0.0</LocalAddress>
    <UserSummary>an example XML Firewall Service</UserSummary>
    <Priority>normal</Priority>
    <LocalPort>${xmlfirewall.port}</LocalPort>
    <HTTPTimeout>120</HTTPTimeout>
    <HTTPPersistTimeout>180</HTTPPersistTimeout>
    <DoHostRewrite>on</DoHostRewrite>

The ${xmlfirewall.port} reference will be automatically substituted by DPBuddy’s “import” task.

It might be desirable to apply some of these transformation rules, such as getting rid of “default” objects, to all export commands, not just to the one we used to export the XML firewall.
This can be achieved by declaring a reusable “transform” rule:

<dp:transform id="remove.default">
    <dpExclude class="(HTTPUserAgent|XMLManager)" name="default"/>
</dp:transform>

Our task can then reference the reusable rule:

<dp:export file="./XMLFirewall.xml">
    <exportObject class="XMLFirewallService" name="testFirewall" />
    <transform refid="remove.default">
        <setText xpath="//LocalPort" value="$${xmlfirewall.port}" />
    </transform>
</dp:export>

The number of version controlled configuration files depends on the number of developers working with DataPower, as well as the number of objects and services being developed. In the most basic case, the entire domain configuration could be exported as one file. When there are multiple developers involved, each developer should be checking in the file containing the services that they’re working on.

Once development for the release has been completed and all the configuration files have been checked in, the next step is to build an integration testing domain from the version controlled files. This is analogous to building a binary from source code. The configuration files should be checked out from the appropriate branch/label/baseline and imported into the domain.

An automated export process helps instill the discipline of frequent commits in the course of development. For developers using Eclipse, this could be as simple as running an export target from the project’s build file and then commiting changes as usual using Eclipse’s version control support.

If you’re interested in automating DataPower configuration export, as well as the rest of your DataPower deployment and administration tasks, download DPBuddy and give it a try. Here you can find various examples of DPBuddy’s tasks, including export-related examples.