Using DPBuddy to Transform DataPower Configuration and XML Files

There is often a need to apply environment-specific modifications to the DataPower configuration. This may include changing ports, removing or adding some configuration elements, or other changes.

DataPower offers the deployment policy-based mechanism which can be executed on the device when importing the configuration. DPBuddy fully supports deployment policies, and also provides additional XML transformation capabilities.

DPBuddy supports many transformation actions that can be executed as part of the import, export, modifyConfig or copy tasks. The actions include “add”, “update”, “delete”, “setText”, “replaceText”, “include” and “exclude”. They are explained in detail later in this section.

All actions except “dpInclude”, “dpExclude” and “replaceText” rely on XPath expressions. The actions are applied to all elements or attributes that match the expression.

The simple example below changes the local port of the front-side HTTP handler to the value provided by the ws.port Ant property. The transformation is applied to the “export.xml” file located inside the zip file.

<dp:import file="${wsproxy.zip.file}" >
    <transform>
        <setText xpath="//LocalPort"  value="${ws.port}" />
    </transform>
</dp:import>

See the samples/ant-tasks/transform.xml file located in your DPBuddy distribution or online for more examples.

DPBuddy’s transformation actions have many powerful features:

  • Can be applied to any XML file, not just DataPower configuration files. You can transform any XML file included in import or copy.
  • Support Ant and Groovy/Gradle variables inside XPath or in XML fragments.
  • Reusable: you can define a reusable group of transformation actions, then reference it from any other group.
  • Can be turned on and off depending on a Boolean expression that can reference any Ant property or Groovy/Gradle variable. For example, a transformation action could be triggered only for certain environments.
  • Support Groovy expressions for complex transformations, e.g., you can apply a Groovy/Java method to an existing value.
  • Easy to debug: transformed files are saved locally and also logged using verbose/debug mode.
  • Can be developed locally without having to connect to a DataPower device. Both the import and copy tasks support “transformOnly” mode, which runs transformations but does not connect to the device.
  • Support repeatable XML fragments.

Any transformable file could have Ant/Groovy/Gradle variables references in any text node or in any attribute. DPBuddy’s transformation logic always attempts to resolve Ant/Groovy/Gradle variables after all transformation actions have been applied. The transformation will fail if any of the variables remains unresolved.

In case you have many transformation rules, it might be beneficial to specify the transformations in a separate file and then include this file in your main DataPower build/deploy script.

transform Type

transform is the container for all transform actions, such as “add”, “setText” and so on. Transformation actions are executed in the order they are specified inside transform.

transform is supported by the following DPBuddy tasks: import, export, copy and modfyConfig.

When used with import, transform could be specified within the transformFiles element. transformFiles determines which files are going to be transformed.

For other tasks, such as copy or modifyConfig, transform is specified within the fileset or dpFileset element, so the transformations are applied to all the files that matched the “include” and “exclude” patterns of the fileset.

transform supports refid and id attributes, similar to Ant’s path element. transform with id can be defined outside of any Ant target. transform can also contain any number of nested transform elements with refid. DPBuddy will first execute actions referenced by nested transforms.

Attributes/Options

Name Description Required
id Unique ID of this transform. Required if this is a reusable transform defined outside of any task or as part of the transform definition file. No
refid Reference to a transform element defined elsewhere. Transformations defined by the referenced transform will be executed prior to the transformations defined by this transform. No
verbose

Print name/values of transformed elements/attributes to standard out when Ant is running in non-verbose mode. This can also be enabled globally by setting property dp.verbose.transform to true.

Defaults to false.

 No

Attributes Common to All Transform Actions

Attribute Description Required
xpath

XPath expression the action will be applied to.

Not applicable to the “dpInclude”, “dpExclude” and “replaceText” actions.

 Yes
matchRequired

Fail if xpath doesn’t match any element/attribute.

Defaults to true.

 No
if

A Groovy Boolean expression (has to return true or false). The expression has access to all Ant properties as Groovy variables. In addition, there is a special dp variable containing the following properties: url, username and domain. These properties are populated with connection information for the device and domain to which the task executing this transform action is being connected. Note that all Ant property names containing dots are changed into lower camel case. E.g., “dp.env” can be referred to as “dpEnv” in Groovy. There is also a special variable antProject of type org.apache.tools.ant.Project. This variable gives an entry point for accessing Ant APIs. E.g., this code snippet antProject.getProperty('prop name') returns the value of the specified property.

The expression has to evaluate to true in order for the transform action to be executed.

 No
verbose Print name/values of transformed elements/attributes to standard out when Ant is running in non-verbose mode. No

Transform Definition File

You can specify your transform logic in one or multiple XML files and then reference these files directly from the import CLI command.

E.g., you can create this transform definition file:

<project name="transforms" xmlns:dp="antlib:com.myarch.dpbuddy" >

    <dp:transform verbose="true" id="set.fw.port">
        <setText xpath="//*[@name='testFirewall']/LocalPort" value="3087"/>
    </dp:transform>

</project>

And then refer to it in your import command:

dpbuddy import -file dpconfigs/XMLFirewall.xml -save -transformDefs transformDefs.xml -transformId "set.fw.port"

setText Transform Action

If the action’s XPath expression matched elements, setText sets the value of the text node of these elements.

If the action’s XPath expression matched attributes, setText sets the value of these attributes.

Attribute Description Required
value Value to set matched elements or attributes to. Yes, unless expression was provided.
expression A Groovy expression-returning String. The expression has access to all Ant properties as Groovy variables. Note that all Ant property names containing dots are changed into lower camel case. E.g., “dp.env” can be referred to as “dpEnv” in Groovy. The expression can also use the special variable currentValue, which is set to the current value of the matched text node or the attribute. Yes, unless value was provided.

replaceText Transform Action

replaceText searches for the provided search string in all attributes and text nodes. If the string is found, it will be replaced with the value specified in the replaceWith attributes.

The search is case-sensitive.

This action does not use XPath, so all attributes and text nodes will be checked.

Attribute Description Required
textToReplace Text to replace. Yes
replaceWith Replacement string. Yes

add and update Transform Actions

add and update add an XML fragment nested within the action tags to the matching elements. In other words, the root element of the XML fragment becomes the child of the matched element. update removes all children of the matching element prior to adding.

add and update cannot be applied to attributes.

add and update support repeaters. Repeaters allow for generating XML fragments in a loop based on a JSON array or an object defined in a HOCON configuration file.

For example, you may need to change the definition of a Load Balancer Group depending on the target environment since each environment can have a different number of LBG members.

You can define your LBG members in JSON/HOCON as following:

lbg.backend:{ _env: true
    // LBG backend config for Dev
    dev:[ {host: server1, port:8080, weight:1} ]
    // LBG backend config for test - 2 servers
    test: [
        {host: server1, port:8080, weight:1},
        {host: server2, port:8090, weight:2}
    ]
}

You can then use the following update transform action with repeat attribute:

<update xpath="${lbg.xpath}" repeat="lbg.backend">
    <LBGroupMembers>
        <!-- host, port, etc. are all part of a json object in the array-->
        <Server>${host}</Server>
        <Weight>${weight}</Weight>
        <MappedPort>${port}</MappedPort>
        <Activity/>
        <HealthPort/>
        <LBMemberState>enabled</LBMemberState>
    </LBGroupMembers>
</update>

Repeater supports two special variables, ${_index_} and ${_key_}. “index” is is the iteration count, starting with “1”. “key” is the key of the item. “key” can only be used with JSON objects, it is undefined for arrays.

delete Transform Actions

delete simply removes all matching elements.

delete cannot be applied to attributes.

dpExclude and dpInclude Transform Actions

These actions don’t use XPath. Instead, they support regular expressions matching DataPower configuration classes and objects.

dpExclude removes all configuration objects matching class names/object names. Conversely, dpInclude removes all configuration objects that do not match class names/object names specified in the action.

Attribute Description Required
class Regular expression matching names of DataPower configuration classes. Yes, unless name was provided.
name Regular expression matching names of DataPower configuration objects. Yes, unless class was provided.

namespace Nested Element

All namespace prefixes used in XPath expressions of the transform actions have to be explicitly defined using the namespace element. Note that transforming DataPower configuration files does not require any namespace definitions.

Attribute Description Required
prefix Prefix of the XML namespace that will be used inside XPath expressions. Yes
uri XML namepsace’s URI. Yes