.. highlight:: xml .. index:: Configuration Transformation .. _transform: Transforming DataPower Configuration ==================================== There is often a need to apply environment-specific modifications to the DataPower configuration. This may include changing ports, removing or adding some configuration elements, or other changes. DataPower offers the `deployment policy-based mechanism `_ which can be executed on the device when importing the configuration. DPBuddy fully supports deployment policies, and also provides additional XML transformation capabilities. DPBuddy supports many transformation actions that can be executed as part of the ``import``, ``export``, ``modifyConfig`` or ``copy`` tasks. The actions include "add", "update", "delete", "setText", "replaceText", "include" and "exclude". They are explained in detail later in this section. All actions except "dpInclude", "dpExclude" and "replaceText" rely on XPath expressions. The actions are applied to all elements or attributes that match the expression. The simple example below changes the local port of the front-side HTTP handler to the value provided by the ``ws.port`` Ant property. The transformation is applied to the "export.xml" file located inside the zip file. .. code-block:: xml See the ``samples/ant-tasks/transform.xml`` file located in your DPBuddy distribution or `online `_ for more examples. DPBuddy's transformation actions have many powerful features: * Can be applied to any XML file, not just DataPower configuration files. You can transform any XML file included in :ref:`import` or :ref:`copy`. * Support Ant and Groovy/Gradle variables inside XPath or in XML fragments. * Reusable: you can define a reusable group of transformation actions, then reference it from any other group. * Can be turned on and off depending on a Boolean expression that can reference any Ant property or Groovy/Gradle variable. For example, a transformation action could be triggered only for certain environments. * Support Groovy expressions for complex transformations, e.g., you can apply a Groovy/Java method to an existing value. * Easy to debug: transformed files are saved locally and also logged using verbose/debug mode. * Can be developed locally without having to connect to a DataPower device. Both the ``import`` and ``copy`` tasks support "transformOnly" mode, which runs transformations but does not connect to the device. * Support repeatable XML fragments. Any transformable file could have Ant/Groovy/Gradle variables references in any text node or in any attribute. DPBuddy's transformation logic always attempts to resolve Ant/Groovy/Gradle variables after all transformation actions have been applied. The transformation will fail if any of the variables remains unresolved. In case you have many transformation rules, it might be beneficial to specify the transformations in a separate file and then include this file in your main DataPower build/deploy script. .. _transform_elt: ``transform`` Type ------------------ ``transform`` is the container for all transform actions, such as "add", "setText" and so on. Transformation actions are executed in the order they are specified inside ``transform``. ``transform`` is supported by the following DPBuddy tasks: ``import``, ``export``, ``copy`` and ``modfyConfig``. When used with :ref:`import`, ``transform`` could be specified within the :ref:`transformFiles ` element. ``transformFiles`` determines which files are going to be transformed. For other tasks, such as :ref:`copy` or :ref:`modifyConfig`, ``transform`` is specified within the ``fileset`` or ``dpFileset`` element, so the transformations are applied to all the files that matched the “include" and "exclude" patterns of the fileset. ``transform`` supports ``refid`` and ``id`` attributes, similar to Ant's `path `_ element. ``transform`` with ``id`` can be defined outside of any Ant target. ``transform`` can also contain any number of nested ``transform`` elements with ``refid``. DPBuddy will first execute actions referenced by nested transforms. Attributes/Options ^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 20 80 8 :header-rows: 1 * - Name - Description - Required * - id - Unique ID of this transform. Required if this is a reusable transform defined outside of any task or as part of the transform definition file. - No * - refid - Reference to a ``transform`` element defined elsewhere. Transformations defined by the referenced ``transform`` will be executed prior to the transformations defined by this ``transform``. - No * - verbose - Print name/values of transformed elements/attributes to standard out when Ant is running in non-verbose mode. This can also be enabled globally by setting property ``dp.verbose.transform`` to ``true``. Defaults to ``false``. - No Attributes Common to All Transform Actions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 20 80 8 :header-rows: 1 * - Attribute - Description - Required * - xpath - XPath expression the action will be applied to. Not applicable to the "dpInclude", "dpExclude" and "replaceText" actions. - Yes * - matchRequired - Fail if xpath doesn't match any element/attribute. Defaults to ``true``. - No * - if - A Groovy Boolean expression (has to return true or false). The expression has access to all Ant properties as Groovy variables. In addition, there is a special ``dp`` variable containing the following properties: ``url``, ``username`` and ``domain``. These properties are populated with connection information for the device and domain to which the task executing this transform action is being connected. Note that all Ant property names containing dots are changed into lower camel case. E.g., "dp.env" can be referred to as "dpEnv" in Groovy. There is also a special variable ``antProject`` of type ``org.apache.tools.ant.Project``. This variable gives an entry point for accessing Ant APIs. E.g., this code snippet ``antProject.getProperty('prop name')`` returns the value of the specified property. The expression has to evaluate to ``true`` in order for the transform action to be executed. - No * - verbose - Print name/values of transformed elements/attributes to standard out when Ant is running in non-verbose mode. - No .. _transformDefFile: Transform Definition File ^^^^^^^^^^^^^^^^^^^^^^^^^ You can specify your transform logic in one or multiple XML files and then reference these files directly from the ``import`` CLI command. E.g., you can create this transform definition file: .. code-block:: xml And then refer to it in your import command: .. code-block:: bash dpbuddy import -file dpconfigs/XMLFirewall.xml -save -transformDefs transformDefs.xml -transformId "set.fw.port" ``setText`` Transform Action ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If the action's XPath expression matched elements, ``setText`` sets the value of the text node of these elements. If the action's XPath expression matched attributes, ``setText`` sets the value of these attributes. .. list-table:: :widths: 20 80 8 :header-rows: 1 * - Attribute - Description - Required * - value - Value to set matched elements or attributes to. - Yes, unless ``expression`` was provided. * - expression - A Groovy expression-returning String. The expression has access to all Ant properties as Groovy variables. Note that all Ant property names containing dots are changed into lower camel case. E.g., "dp.env" can be referred to as "dpEnv" in Groovy. The expression can also use the special variable ``currentValue``, which is set to the current value of the matched text node or the attribute. - Yes, unless ``value`` was provided. ``replaceText`` Transform Action ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ``replaceText`` searches for the provided search string in all attributes and text nodes. If the string is found, it will be replaced with the value specified in the ``replaceWith`` attributes. The search is case-sensitive. This action does not use XPath, so all attributes and text nodes will be checked. .. list-table:: :widths: 20 80 8 :header-rows: 1 * - Attribute - Description - Required * - textToReplace - Text to replace. - Yes * - replaceWith - Replacement string. - Yes ``add`` and ``update`` Transform Actions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ``add`` and ``update`` add an XML fragment nested within the action tags to the matching elements. In other words, the root element of the XML fragment becomes the child of the matched element. ``update`` removes all children of the matching element prior to adding. ``add`` and ``update`` cannot be applied to attributes. ``add`` and ``update`` support repeaters. Repeaters allow for generating XML fragments in a loop based on a JSON array or an object defined in a :ref:`HOCON configuration file `. For example, you may need to change the definition of a Load Balancer Group depending on the target environment since each environment can have a different number of LBG members. You can define your LBG members in JSON/HOCON as following: .. code-block:: js lbg.backend:{ _env: true // LBG backend config for Dev dev:[ {host: server1, port:8080, weight:1} ] // LBG backend config for test - 2 servers test: [ {host: server1, port:8080, weight:1}, {host: server2, port:8090, weight:2} ] } You can then use the following ``update`` transform action with ``repeat`` attribute: .. code-block:: xml ${host} ${weight} ${port} enabled Repeater supports two special variables, ``${_index_}`` and ``${_key_}``. "index" is is the iteration count, starting with "1". "key" is the key of the item. "key" can only be used with JSON objects, it is undefined for arrays. ``delete`` Transform Actions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ``delete`` simply removes all matching elements. ``delete`` cannot be applied to attributes. ``dpExclude`` and ``dpInclude`` Transform Actions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ These actions don't use XPath. Instead, they support :ref:`regular expressions ` matching DataPower configuration classes and objects. ``dpExclude`` removes all configuration objects matching class names/object names. Conversely, ``dpInclude`` removes all configuration objects that do not match class names/object names specified in the action. .. list-table:: :widths: 20 80 8 :header-rows: 1 * - Attribute - Description - Required * - class - Regular expression matching names of DataPower configuration classes. - Yes, unless ``name`` was provided. * - name - Regular expression matching names of DataPower configuration objects. - Yes, unless ``class`` was provided. ``namespace`` Nested Element ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ All namespace prefixes used in XPath expressions of the transform actions have to be explicitly defined using the ``namespace`` element. Note that transforming DataPower configuration files does not require any namespace definitions. .. list-table:: :widths: 20 80 8 :header-rows: 1 * - Attribute - Description - Required * - prefix - Prefix of the XML namespace that will be used inside XPath expressions. - Yes * - uri - XML namepsace's URI. - Yes