Installation

DPBuddy Installation

Note

You can skip these installation steps and use DPBuddy docker image instead.

DPBuddy requires Java SE Runtime Environment 8 or better. All JVM types are supported, including Oracle, IBM and OpenJDK. DPBuddy is pure Java, so it runs on any OS where Java is available. In some cases, DPBuddy may require a large heap size so a 64-bit JRE is recommended.

All the latest versions of Java SE or JDK are fully supported.

Both DPBuddy and Apache Ant will first check if the JAVA_HOME environment variable is defined. If it is defined, this Java installation will be used. If JAVA_HOME is not defined, then java executable must be added to the path. To verify, run the java -version command.

Install DPBuddy by following these simple steps:

  • Download DPBuddy and unzip/untar the content of the file into a directory, e.g., c:\dpbuddy or /opt/dpbuddy:

  • Define the DPBUDDY_HOME environment variable and point it to the directory containing the DPBuddy installation, e.g., c:\dpbuddy\dpbuddy-3.5 or /opt/dpbuddy/dpbuddy-3.5.

  • Add %DPBUDDY_HOME%\bin or $DPBUDDY_HOME/bin to your PATH environment variable. On Linux, make sure that $DPBUDDY_HOME/bin/dpbuddy has the “execute” permission, run the appropriate “chmod” command if it does not.

  • Open a command line/terminal window and type dpbuddy -h. You should see a help screen containing the list of commands and options supported by the tool.

On Linux, you can add these lines to your .bash_profile:

export DPBUDDY_HOME=/opt/dpbuddy/dpbuddy-3.5
export PATH=$DPBUDDY_HOME/bin:$PATH

If you already have a previous version of DPBuddy installed, you will need to update your taskdef Ant tag to point to the location of the new version.

DPBuddy does not require root or administrator privileges, it can run under a regular user account.

When running DPBuddy under a non-administrative account on Windows you may see the warning “Could not open/create prefs root node SoftwareJavaSoftPrefs …”. This warning can be ignored. This can also be fixed by running DPBuddy as administrator once (after that you can continue running it as user) or by manually creating necessary keys in the Windows registry as explained here.

You can also follow our Quick Start Guide to complete your DPBuddy setup.

Using DPBuddy Docker Image

DPBuddy docker image contains the latest versions of DPBuddy, Java 8, Apache Ant and Gradle. Use of DPBuddy docker image is fully supported in all environments, including production.

Pull the image from the DPBuddy’s docker repository:

docker pull myarch/dpbuddy:3.5

DPBuddy docker image exposes two volumes, one for DPBuddy logs and another one for your scripts, configurations, exports, etc. You need to create these volumes and provide them to docker to preserve the files between container runs or image updates.

Creating volumes is not required for Docker on macOS.

docker volume create dpbuddy-logs
docker volume create dpbuddy-work

Run the container:

docker run -it --rm --name dpbuddy  -v dpbuddy-work:/opt/dpbuddy/src -v dpbuddy-logs:/var/log/dpbuddy myarch/dpbuddy:3.5

Note the --rm option which means your container will be destroyed upon exit, so make sure that you preserve all your work in volumes.

Alternatively, you can first create the container and then run it using docker start command

docker create -it --name dpbuddy -v dpbuddy-work:/opt/dpbuddy/src -v dpbuddy-logs:/var/log/dpbuddy myarch/dpbuddy:3.5
docker start -i dpbuddy

Note

Run docker with sudo if you get any errors attempting to execute docker commands.

On macOS you can simply point to a directory under one of the shared root directories:

docker run -it --rm --name dpbuddy  -v $PWD/dpbuddy-docker/vols/work:/opt/dpbuddy/src -v $PWD/dpbuddy-docker/vols/logs:/var/log/dpbuddy myarch/dpbuddy:3.5

You can run the following command from the docker’s shell to verify that DPBuddy is up and running:

dpbuddy -h

Next, you will need to create your dpbuddy.conf file. You can use the dpbuddy.starter.conf file in /opt/dpbuddy/conf as the starting point.

Copy it to the /opt/dpbuddy/src first (to make sure it’s not lost after container restart):

cp /opt/dpbuddy/conf/dpbuddy.starter.conf /opt/dpbuddy/src/dpbuddy.conf

Then follow quick start guide to modify it.

DPBuddy docker image predefines several DPBuddy environment variables.

  • DPBUDDY_CONFIG. It points to /opt/dpbuddy/src/dpbuddy.conf.

  • DP_MASTER_PASSWORD. You may want to override it if you intend to use DPBuddy’s encryption/decryption capabilities.

  • DPBUDDY_HOME is set to /opt/dpbuddy.

You need to provide a new environment variable value to the docker create or docker run commands using -e option.

E.g., to provide your master password:

docker run -it --rm --name dpbuddy -e DP_MASTER_PASSWORD=my-secure-password -v dpbuddy-work:/opt/dpbuddy/src -v dpbuddy-logs:/var/log/dpbuddy myarch/dpbuddy:3.5

You can also provide any DPBuddy command on the docker command line, in which case the command will be executed and then the container will be terminated:

docker run -it --rm --name dpbuddy  -v dpbuddy-work:/opt/dpbuddy/src -v dpbuddy-logs:/var/log/dpbuddy myarch/dpbuddy:3.5 dpbuddy connect

Upgrading DPBuddy

To upgrade DPBuddy, simply un-archive the distribution into a new folder and change your DPBUDDY_HOME environment variable to point to the new location.

The DPBuddy 3.5 distribution contains Apache Ant 1.10 jar files. To avoid classloader conflicts, exclude Ant jars from the taskdef used to define DPBuddy tasks. Additionally, DPBuddy starting with version 3.2.4 uses logback logging framework and relies on logback.xml configuration file. “logback.xml” is located in “$DPBUDDY_HOME/conf” directory. In Ant, this directory needs to be explicitly added to the classpath of the DPBuddy’s “taskdef”. The resulting “taskdef” should be defined as following:

<taskdef uri="antlib:com.myarch.dpbuddy">
    <classpath>
        <pathelement location="${dpbuddy.home}/conf"/>
        <fileset dir="${dpbuddy.home}/lib" excludes="ant-*"/>
    </classpath>
</taskdef>

All DPBuddy 3.5 tasks are backward compatible with 3.0, 3.1, 3.3 and 3.4.

Configuring DataPower to Work with DPBuddy

You need to make sure that you have XML Management enabled in all devices that will be managed by DPBuddy. You can do this from DataPower WebGUI (Network/Management/XML Management).

DPBuddy does not use SSH, so SSH can be disabled.

DPBuddy uses the “old” DataPower v2004 message format for file transfer (the 2004 schema is more efficient, since it allows for uploading multiple files at once). This means that the “SOAP Configuration Management (v2004)” checkbox must be checked on the XML management screen.

Be careful performing these changes on the appliances residing in a production DMZ – make sure that the management ports are not available to the outside world.

We also recommend enabling internal logging as documented in the DataPower Info Center. DataPower does not return error messages to the client. If a DPBuddy command fails and you get an “Internal Error” message from the device, you will need to look for the root cause in the DataPower system log in the default domain. DataPower captures the error only if internal logging is enabled. You can enable it by navigating to “Troubleshooting” from the control panel, then to “Logging.” Set “Enable Internal Logging” and click on “Set Log Level.” We recommend setting the level to “Warning”, since anything more detailed (e.g., “Information”) results in needlessly verbose messages.

If you’re creating a new account that will be used by DPBuddy, make sure that the password for the account has been reset. Before running DPBuddy, make sure that you’re able to login to WebGUI using this account.

DPBuddy will work with any “Group-Defined” account that has read/write access to the target domain(s). In general, DPBuddy does not need to access the “default” domain. However, certain features of DPBuddy’s tasks, such as updating domain’s comments during import will not work if the write access to the default domain is restricted. quiesceDomain and unquiesceDomain tasks also require access to the default domain.

Note that by default DataPower uses a self-signed certificate for SSL. By default, DPBuddy ignores certificate validation errors. You can change this behavior by setting dp.trust.all.certs to false as described in Common Attributes/Options and Properties. If you choose this option, you will need to manually add the DataPower certificate to your JVM’s keystore.

You also need to make sure that your DataPower appliance has sufficient internal space. For exports and backups DataPower first generates export files in the internal filesystem. If you’re trying to export large configuration files and there is not enough space, you will get an error. You can checkOld the available temporary space by navigating to “Status/System/Filesystem information” in WebGUI.

It is highly recommended (although not absolutely required) to place all machines hosting DPBuddy and all DataPower appliances in the same time zone. DataPower relies on ZIP format for export/import; this means that file timestamps have no time zone information. This may lead to various issues if an export/import archive needs to be modified during your build/deployment process.

Creating DPBuddy Configuration File

All DataPower connection parameters and other configuration properties have to be defined in the dpbuddy.conf config file. You can follow the quick start guide to bootstrap its creation.

Once you created dpbuddy.conf, you should keep it under version control.

dpbuddy.conf is extremely flexible and should be used for all your environment and configuration settings.

To use dpbuddy.conf from CLI, define DPBUDDY_CONFIG environment variable.

Configuring Apache Ant to Work with DPBuddy

Using DPBuddy from Apache Ant provides more flexibility and better integration with various build/deployment and release management tools than DPBuddy’s CLI. It is also more efficient since there is no need to start a JVM for every command.

DPBuddy requires Apache Ant 1.8 or above.

If you don’t have Apache Ant installed on your system, download and install it as documented at http://ant.apache.org/manual/install.html. Note that on many Linux systems you can also install Ant using a package manager, e.g., you can use apt-get -u install ant on Ubuntu. We also recommend running Ant with the -e option, as it provides a much cleaner output. You can set this option globally by setting the ANT_ARGS environment variable to -e.

DPBuddy tasks are packaged in a single Antlib. You may wish to familiarize yourself with the Ant documentation of the Antlib feature, especially the use of namespaces in Ant, before configuring DPBuddy tasks in your build file.

To declare DPBuddy tasks, add “taskdef” to your build file and point it to the DPBuddy installation:

<property name="dpbuddy.home" location="your location" />
<taskdef uri="antlib:com.myarch.dpbuddy">
     <classpath>
         <!-- Add conf directory to the classpath -->
         <pathelement location="${dpbuddy.home}/conf"/>
         <!-- We need to exclude Ant jars packages with DPBuddy -->
         <fileset dir="${dpbuddy.home}/lib" excludes="ant-*"/>
     </classpath>
 </classpath>

You must also declare the namespace used by DPBuddy tasks:

<project name="your project" xmlns:dp="antlib:com.myarch.dpbuddy" >

The namespace’s URI must match the URI used in taskdef. You must use antlib:com.myarch.dpbuddy as the URI.

You can also copy all the jar files that come with DPBuddy to ${user.home}/.ant/lib directory. If you choose this option, you don’t need to provide an explicit taskdef. Please consult the Ant manual for more details.

Verifying Connectivity with DataPower

To verify DPBuddy’s connectivity with DataPower, run the following command:

dpbuddy connect -env <your environment from dpbuddy.conf>

Advanced Configuration

If you’re planning to use DPBuddy to download or upload large binary files (bigger than 30MB), we recommend increasing the maximum heap size for the JVM used to run DPBuddy.

DPBuddy CLI supports the DPBUDDY_OPTS environment variable, which can be used to pass custom options to the JVM. By default, this variable is set to -Xms256m -Xmx1024m, meaning that the maximum heap size is set to 1GB.

To change the heap size for Ant, set the ANT_OPTS environment variable.

-Xmx1648m is the recommended setting for both Ant and DPBuddy CLI.

If you’re going to run DPBuddy behind a proxy, add the following Java system properties to the DPBUDDY_OPTS or ANT_OPTS variables: -Dhttps.proxyHost=<your proxy host> -Dhttps.proxyPort=<your proxy port> (note “https” prefix), or use Ant’s setProxy task.

DPBuddy License

All DPBuddy downloads are bundled with a 45-day evaluation license. You do not need to request or install the evaluation license; it is generated automatically when you run DPBuddy for the first time.

Once the evaluation license is expired, you’ll need to purchase the license from MyArch or request an extension of your trial license. Please send your request to license at myarch.com or submit it at http://myarch.com/dpbuddy-pricing.

To install a permanent license that you received from us, simply save the license file and set DPBUDDY_LICENSE_FILE environment variable to point to the license file (must be an absolute path to the file, e.g. export DPBUDDY_LICENSE_FILE=/opt/dpbuddy/dpbuddy-license.

You can also save the license file in $DPBUDDY_HOME/conf (%DPBUDDY_HOME%\conf on Windows) or, if you’re using our docker image, under /opt/dpbuddy/src. In this case, you don’t need to specify any environment variables.

Alternative, you can add the property dpbuddy.license.file to your configuration file or Ant build file. You can also provide the license option or attribute to a DPBuddy command or task.

Upgrading from DPBuddy 2.3

DPBuddy 3 is mostly backward compatible with DPBuddy 2.3 with a few exceptions:

  • The namespace for the DPBuddy antlib has changed to com.myarch.dpbuddy. Make sure to update it in the project tag and in the taskdef of your build file, e.g., xmlns:dp="antlib:com.myarch.dpbuddy".

  • The request task was renamed to somaRequest.

  • The setConfig task was deprecated; use import instead.

  • modifyConfig no longer supports the nested configFile element. You can replace it with the new transform element.

  • toDir is now a required attribute of the copy. You can also provide toDir at the nested dpFileset level.

  • Environment prefix mechanism has been substantially enhanced; you may need to change the way your properties are named to take advantage of the hierarchical prefixes.