DPBuddy Roadmap

We're currently working on the next version of dpbuddy, our popular tool for managing IBM WebSphere DataPower appliances.
For now, we're planning on the following features:

• Task for secure backup (execute secure backup and download the resulting file).
• Tasks for quiescing domains and objects.
• DPBuddy's xpath-based alternative to deployment policies can be used with import. Currently this feature is only supported with setConfig task, whereas majority of dpbuddy users rely on import.
• Command-line interface as an alternative to Ant tasks.
• Integration of DataPower-related environment variables (e.g., a port number) with the prefix-based device definition.

What else would you like to see in the new version? Please comment on this post or in our google group.

Posted in dpbuddy

DataPower Buddy 2.3 Release

You can find DPBuddy 2.3 announcement here.

Posted in DataPower, WebSphere, XML Appliances, dpbuddy

Security Hardening of WebSphere Application Server Installations

It is a known fact that an out of the box a WAS installation with security enabled is not entirely secure. There is a number of steps that has to be taken in order to "harden" the installation. Most of the steps are documented in the WebSphere hardening guide, which should be closely studied and followed by any WAS administrator or developer. The hardening guide, however, is a little light on the specifics regarding how to secure WAS at the OS level. For example, hardening should normally include changing file and directory permissions to restrict access to sensitive configuration files.

Why bother hardening the installation at the OS level? This helps defend against external threats, i.e., a server hosting the installation is compromised and against internal threats, i.e., an unauthorized access from inside.

Here are some of the things you can do to secure your installation on a Unix/Linux platform:

• If you installed WAS as a non-root account, make sure that this account is not part of the "users" group. Create a separate group for all administrators, say "wasadmins". If you're doing this the post-install, make sure that you changed the ownership of all WAS files and profiles. To do that, bring all WAS processes down and run chgrp -R wasadmins $WAS_INSTALL_ROOT.
• Refrain from running WAS using root. There is no reason to do it on Unix/Linux, unless you absolutely have to use low ports. Yes, you can use sudo to tighten down access to root, but it is safer just to stay away from it. You can actually install WAS using root so it can be properly registered with the OS, but you can later change the installation to be owned by a non-root account.
• Following the principle of least privilege, remove read access from "others". Why do you need to do it? There are many sensitive files that are part of WAS install, including files containing encoded passwords (see below). It's better to err on a side of caution and disallow general read access altogether. We should revoke execute access as well. On the other hand, we may want to grant write/execute to all members of wasadmins groups, assuming the membership in this group is going to be tightly controlled:
  chmod -R 770 $WAS_INSTALL_ROOT
• You may want to allow access to logs to "others" so that developers can view logs. Since we just revoked read permissions from the root WAS install directory and from all profile directories, you can't just "chmod" the "logs" directory. You need to redirect logs to some other directory which is not under "$WAS_INSTLALL_ROOT" or profile roots. In order to do it, change LOG_ROOT WAS variable. You can do it from the console (Environment/WebSphere variables). By default, "LOG_ROOT" is defined at a node level. If the location is going to be the same on all nodes (which it should), just create LOG_ROOT at the cell level. You'll need to restart WAS servers after this. You can redirect logs to, say, /var/log/was. Make sure that "wasadmin" system account has write permissions to this directory.
• Do not allow login to the system account used to run WAS. You can read this guide explaining how to do it for various login mechanisms. You also need to disable "sudo su wasadmin" and only allow sudo for the specific commands/scripts that are needed to run WAS. You will need to add something like this to the sudoers file:
  %wasadmins ALL=(wasadmin) WAS_INSTLALL_ROOT/bin/, PROFILE_ROOT/bin/
  This configuration allows all member of "wasadmins" group to run commands from "bin" using "sudo -u", e.g., sudo -u wasadmin ./startServer.sh server1.
  You may relax this a little bit by allowing sudo access without having to enter password every time:
  %wasadmins ALL=(wasadmin) NOPASSWD: WAS_INSTLALL_ROOT/bin/, PROFILE_ROOT/bin/
  It's a bit of a pain having to run all commands via sudo (although, of course, it can be scripted), but it gives you complete audit trail of who did what in the environment. By the way, by default sudo logs via syslog, but you can redirect sudo's log into its own file by adding this directive in sudoers:
  Defaults logfile=PATH_TO_LOG_FILE
• Secure "sensitive" files. As any WAS administrator knows, passwords stored in WAS are not really encrypted, they are just encoded, so they can be decoded pretty easily (there is a way to plug in custom encryption, but that's a different story). Therefore, it is imperative to properly protect files containing encoded passwords. To do that, we're going to revoke any access to these files from "wasadmins" group. That way, only the "wasadmin" system account (which we can't even login into) or root can read these files. Encoded passwords are stored in "security.xml" and "wim/wimconfig.xml" (if you have LDAP configured). All these files reside under "$PROFILE_ROOT/config/cells/$CELL_NAME". "fileRegistry.xml" also contains passwords, although these passwords are properly encrypted.
  You may also want to protect keystores (key.p12). Note that in addition to the cell key store, each node has its own keystore. You don't need to worry as much about trust stores (trust.p12), in fact it might be convenient to keep group-level access to trust stores so that administrators can modify them outside of WAS (e.g., by adding a signer cert using ikeyman). While you're on it, make sure that you don't use the default password (WebAS) for any of the keystores/truststores.
  To revoke group access to any of the files mentioned above, just run "chmod 600" on them, e.g., chmod 600 security.xml. Note that you'll have to repeat it for each profile, including deployment manager and application server profiles (unless you have a standalone install).
  Node synchronization built into WAS fully supports changing permissions on config files. The changes will not disappear after node sync (provided you updated the DM profile).
• If your environment has a DMZ utilizing IHS, make sure that you don't have WAS installation on the DMZ boxes. It is convenient to have node agents running on the IHS servers, so you can manage IHS from admin console, but, unfortunately, it is a security risk. Deployment manager pushes all configuration files, including "security.xml" to all nodes, and we certainly don't want this file sitting in the DMZ (even with "600" permissions).
• Repeat all changes on every server containing WAS installation. You may want to script them or, better yet, use a tool like Puppet to automate OS-level configuration changes across multiple machines. For example, see this guide on how to centralize management of the sudoers file.

This post is part of the series on WebSphere Application Server administration. Please subscribe to our blog if you'd like to receive updates.

Note: We offer professional services in the area of WebSphere architecture, implementation and operations. If you're looking for help with any of these tasks, please let us know.

Posted in WebSphere, WebSphere Application Server Administration

DataPower Secure Backup with DPBuddy

Secure backup is the only way to backup the entire device, including keys and certificates. Therefore, it is desirable to run it on a regular basis.

You can easily do it with dpbuddy using its support for "do-action" command. Secure backup is simply one of the actions with a few additional parameters.

Here's how it could be expressed using Ant and dpbuddy. Note that we're using "local://" filesystem as a destination; in reality you probably want to use "ftp:" instead.

<target name="secure-backup" description="Backup the entire device">
    <tstamp/>
    <dp:action>
        <SecureBackup>
            <cert>test-cert</cert>
            <destination>local://sbackup-${DSTAMP}</destination>
        </SecureBackup>
    </dp:action>
</target>

Posted in DataPower, dpbuddy

WebSphere Application Server Tutorial for Developers — WAS in 5 Minutes

If you're developing applications for WAS and you're new to it, this is what you need to know:

• How to view the logs. Logs can be found under install_root/profiles/profile_name/logs/server_name. The default profile name is AppSrv01 and the default server name is server1. Example:/usr/IBM/WebSphere/AppServer/profiles/AppSrv01/logs/server1. SystemOut.log is the file containing everything that was logged to standard out. Logs can also be viewed from the admin console by navigating to Troubleshooting/Logging and Tracing/server_name/Runtime.
• How to deploy an application. In admin console, navigate to Applications/Application Types/WebSphere enterprise applications, click on "Install new application", select "Fast path", accept all the defaults except that on "step 2" make sure that you targeted correct servers (if you have multiple servers/clusters in your environment). Note you can deploy a WAR file directly, you don't have to build an EAR. In this case, make sure that you set a context root on "step 4" screen.
• How to change context root of a Web application. Go to Applications/Application Types/WebSphere enterprise applications/application_name/Context Root For Web Modules in the console. Re-start the application after the change.
• How to change the order of classloaders. If you're getting a ClassNotFoundException when you're starting the app, changing the order of classloaders is the first thing you may want to try. Go to Applications/Application Types/WebSphere enterprise applications/application_name/Manage Modules/module_name and make the appropriate selection in the "Class loader order" drop-down (this assumes you're doing it for a WAR module).
• How to enable dynamic class reloading. If you need to frequently update your deployed application (e.g., you use a local WAS installation for development), enabling dynamic reloading could be a huge time saver. Go to your application in the console, "Class loading and update detection", set "Override class reloading settings ..." and set polling interval to 2 seconds. See this post for more details on how to configure your development environment to support class reloading.
• How to find a host name and a port of the server. Go to Server/Server Types/WebSphere application servers. You'll find the host name in the Host Name column. To find a port, click on your server, and expand Ports. WC_defaulthost is the HTTP port and WC_defaulthost_secure is the HTTPS port.
• How to start/stop a server. If you're dealing with the "Network Deployment" type of installation (multiple application servers running under the control of the "deployment manager"), your can start/stop a server from the console. Otherwise you have to do it from command line. Go to install_root/bin and run ./startServer.sh server_name, e.g., ./startServer.sh server1 (this assumes that your installation has only one profile defined, otherwise you may need to "cd" to the profile_name/bin directory). Make sure that you run all commands using the appropriate system account. To stop the server, run ./stopServer.sh server_name -username user_name -password password. user_name and password is the credentials of an admin account, typically the same one you use to login to the console.
• How to browse JMS messages. Go to Buses/Your bus name/Destinations/Your destination/Queue points/Your queue point/Runtime/Messages.

This post is part of the series on WebSphere Application Server administration. Please subscribe to our blog if you'd like to receive updates.

Note: We offer professional services in the area of WebSphere architecture, implementation and operations. If you're looking for help with any of these tasks, please let us know.

Posted in Developer productivity, WebSphere, WebSphere Application Server Administration

Why Pay for DPBuddy Support?

You can find information on our support services here.

Posted in DataPower, dpbuddy

Keeping Multiple DataPower Devices in Sync

Most DataPower installations in production contain two DataPower devices with identical or similar configuration.
It is actually quite easy to keep the two devices in sync using Apache Ant and DPBuddy administration tool.
DPBuddy supports a notion of an environment prefix. The prefix groups together connectivity properties for each device:

dp.username=dpbuddy
dp.password=123 
dp.domain=deployTest
prod1.dp.xmlmgm.url=https://dp1.prod
prod2.dp.xmlmgm.url=https://dp2.prod

The above property list defines dp.username, dp.password and dp.domain properties that are common to all environments and two XML management URLs for our DataPower cluster.

Then the Ant target for importing configuration into both devices could look like this:

<parallel>
    <dp:import envPrefix="prod1" file="${import.file}" overwriteFiles="true" overwriteObjects="true" />
    <dp:import envPrefix="prod2" file="${import.file}" overwriteFiles="true" overwriteObjects="true" />
</parallel>

Note that for speed we chose to execute both imports in parallel; it is perfectly save to do it with dpbuddy as all of its tasks are thread-safe.

If your "dp:import" tasks is more complex (e.g. because of use of deployment policies), you may want to create an Ant macro for each task to avoid duplicating parameters.

MyArch, Inc. offers paid support for dpbuddy, including implementing custom enhancements. If you're interested, please contact us as info at myarch.com.

Posted in DataPower, dpbuddy

DataPower Buddy Google Group

There is now a Google group for Data Power buddy: http://groups.google.com/a/myarch.com/group/dpbuddy.
DPBuddy users can use this group to discuss questions and issues related to the dpbuddy tool.

Posted in dpbuddy

DataPower Buddy 2.2.2 Release

DataPower Buddy 2.2.2 is available from this link. This release contains several bug fixes, mostly around "import" Ant task.

MyArch, Inc. offers paid support for dpbuddy, including implementing custom enhancements. If you're interested, please contact us as info at myarch.com.

Posted in Ant, DataPower, dpbuddy

Message Acknowledgement or Transacted Session?

JMS is one of the oldest Java EE specifications (JMS 1.0 specification is dated 11/1999), however, questions about the difference between message acknowledgement and transacted session still come up. The difference is especially subtle when programmatic client acknowledgement (Session.CLIENT_ACKNOWLEDGE) is used since Message.acknowledge() and Session.recover() are similar to Session.commit() and Session.rollback(). So how are these APIs different?

The bottom line is that there is no difference if you deal only with a single resource (Queue or Topic) within a session. If all you do is consuming messages from a single queue, it does not matter whether you use acknowledgements or transacted sessions (although in my opinion it is more intuitive to use Session.commit/rollback). Session.commit() invokes Message.acknowledge() under the covers and Session.rollback() invokes recover().

However, if you're dealing with multiple JMS resources withing the same session (or multiple consumers/producers), the transacted session mechanism is what you want to use. For example, you may consume messages from one queue and then put messages on a different queue using the same JMS session. The transacted session will treat all consumed and produced messages as part of a single transaction and will commit or rollback all messages at once. Message acknowledgement on a non-transacted session will "commit" consumed messages independently of the produced ones.

Transacted sessions are limited to JMS resources; container-managed transactions and JTA is required to managed JMS and non-JMS resources (e.g., getting a message from a queue and updating a database).

As a summary, the different mechanisms discussed here differ in terms of the types of resources managed as part of transactions:

• Message acknowledgement: messages consumed from a single destination.
• Transacted session: multiple JMS resources within the same JMS session.
• Container-managed transactions: multiple JMS and non-JMS resources.

For a more in-depth discussion of different transaction mechanisms, refer to this article.

Posted in JMS, Java EE

Tips for Developing JMS Client for WebSphere Application Server

Following are some tips for developing a JMS client for WAS.

• First, we need to put the right jar files on the client's classpath. Following jars are required: com.ibm.ws.sib.client.thin.jms_<WAS version>.jar, com.ibm.ws.ejb.thinclient_<WAS version>.jar. If you're using Oracle/Sun JDK, you'll also need com.ibm.ws.orb_<WAS version>.jar as IBM libraries rely on IBM ORB implementation. IBM JDK has it built-in but Oracle's obviously does not. These jars are available from the "runtimes" directory of your WAS installation. It is a good idea to make sure that the version of your client jars matches the version of your target WAS installation, including the fixpack level. You may want to update your client jars every time a new fixpack is installed.

• If you're not using DNS, you'll need to add the host name of the WAS server to the host file on your client machine, otherwise you're bound to get a cryptic ORB error message. You can find WAS host name by going to Server/Server types/ WebSphere Application Servers in admin console.
  By the way, most ORB error messages are cryptic; it is very rare to see the root cause of the problem in the message. The way to deal with it is to enable ORB tracing. Add -Dcom.ibm.CORBA.Debug=true to your JDK parameters; other debug/tracing parameters are explained here.

• Another frequent source of errors is a missing bootstrap property in the connection factory configuration. It is specified in the "provider endpoint" field and has a form of <host>:<SIB_Port>:BootstrapBasicMessaging. If this is not provided, WAS will attempt to bootstrap the messaging engine using localhost.

• Most JMS examples use JNDI lookup in order to get a queue object. It is more efficient to use session.crerateQueue() method. Note that in this case you need to provide the name of the JMS destination as opposed to the queue JNDI name. This is the "Queue name" parameter on the queue configuration screen in WAS admin console.

• Finally, not all developers are aware that WAS admin console has a built-in JMS queue/topic browser. It takes a few steps to get to it from the console: Buses/Your bus name/Destinations/Your destination/Queue points/Your queue point/Runtime/Messages.

Posted in Java EE, WebSphere, WebSphere Application Server Administration

DataPower Deployment Policies and XPath

Deployment policy in IBM DataPower appliance is a very nice concept. Deployment policies allow for "tweaking" device and domain configuration for different environments. Let's face it, there are always differences between environments. Sometimes, these differences are small, such as different back-end hosts, and in other cases these differences could be significant, such as different security policies. It is commendable that IBM recognized this fact and implemented a mechanism for dealing with this very common problem directly in their product.

Deployment policies do a decent job of dealing with differences between environments. Deployment policies support changing, adding and deleting configuration, so it is possible to implement fairly complex transformations.

However, dealing with deployment policies could be confusing. Deployment policy's match rules utilize xpath, but the syntax of the rules is not pure xpath (note -- I'm using 3.7 firmware. The syntax might be different in 3.8). Consider this simple deployment policy match rule:

*/*/protocol/http?Name=personSreviceHTTP&Property=LocalPort

The part before "?" looks like xpath. But what schema is this xpath based on? There is no "protocol" element in the DataPower XML management schema. The part after "?" that uses name-value parameters is even more odd. Why use this instead of proper xpath? After all, DataPower has an XML-processing engine with full xpath support, so it would certainly be more logical to rely on XML standards.

The bottom line is that while deployment policies are useful, they have limitations. They have to be developed using Deployment Policy builder in WebGUI. They can only be applied to configuration elements supported in WebGUI. For example, creating a deployment policy that updates RemoteEndpointPort of a Web Services Proxy proves to be a non-trivial task.

This is why we added support for "plain" xpath-based overrides in our DPBuddy DataPower management tool. Instead of dealing with the obscure syntax of deployment policies, developers can simply look at the configuration export file and specify an xpath expression against this file. "dp:setConfig" task of DPBuddy will update the matching element of this expression with the new value. For example, to update RemoteEndpointPort, one can define the following task:

<dp:setConfig>
    <configFile file="dpconfigs/config-wsproxy.xml">
        <override xpath="//RemoteEndpointPort" value="${endpoint.port}"/>
    </configFile>
</dp:setConfig>

It is not possible to add or remove configuration using "dp:setConfig"; deployment policies still have to be used for this kind of config changes. However, from our experience, majority of environment differences can be dealt with the xpath-based override mechanism.

MyArch, Inc. offers paid support for dpbuddy, including implementing custom enhancements. If you're interested, please contact us as info at myarch.com.

Posted in DataPower, WebSphere, XML, XML Appliances, dpbuddy

DPBuddy — Tool For DataPower Administrators and Developers

We're pleased to announce the release of our new product, DataPower Buddy (dpbuddy). "dpbuddy" a free command-line tool for automating administration, management and deployment of IBM WebSphere DataPower appliances. The tool supports export/import, file transfer, backups and many other functions.

dpbuddy is implemented as a set of custom tasks for the popular build tool, Apache Ant.

Here is a quick example of dpbuddy in action:

<dp:copy cleanDirectories="true">
    <dpFileset prefix="/apps/services" 
               dir="services" includes="**/*.wsdl **/*.xsd"/>
</dp:copy>

This Ant task will remove remote directories if they exist, reproduce the local directory tree (all folders under "services") on the device and upload the necessary files based on the "includes" pattern.

dpbuddy is completely free; it can be downloaded from the dpbuddy product page

dpbuddy provides many cool features, including:

• Response from the device is presented in a human-readable form as opposed to raw SOAP/XML messages. dpbuddy makes it easy to understand error and status messages.

• Powerful remote "copy" command that automatically reproduces local directory tree on the device.

• Tight integration with Ant. Ant variables can be used inside deployment policies and configuration files.

• Easy-to-use alternative to deployment policies based on XPath.

• Ability to remotely "tail" device logs. It is even possible to automatically get new log messages similarly to Unix "tail -f" command. "tail" task can also check for error patterns.

• "Export" based on naming patterns. You don't need to know types ("classes") of DataPower objects; simply specify a regexp pattern and dpbuddy will export all objects matching this pattern.

• Support for self-signed certificates. No need to add DataPower certificates to the JDK store.

• Support for arbitrary SOMA requests. You can use Ant variables inside a request.

• Parsing of all commands on the client. In case of XML errors, DataPower returns cryptic "internal error" message. The actual error then has to be extracted from the device logs. dpbuddy on the other hand validates management XML commands on the client and displays error messages right away.

Go to dpbuddy product page to learn more.

Posted in Building software, DataPower, IT operations, XML Appliances, dpbuddy

PAnt 2.0.1 is Released

PAnt 2.0.1 contains an important bug fix that was preventing using certain Ant tasks from python. Specifically, the bug affected all tasks that utilize addConfigured method to handle nested elements. This included the "manifest" task and several others.

You can download PAnt 2.0.1 from the PAnt project page

Posted in Ant, PAnt

WebSphere 7 Business Level Applications

Application management, including deployment, has remained mostly unchanged since WAS 5.

WAS 7, however, introduced a new concept -- "Business Level Applications" (BLA). It would be more accurate to use the term "composite application" because this is what BLAs are -- it's a composition of Java EE applications (WARs or EARs) or other "assets".

Unfortunately, the term "composite application" is already used in a different context by IBM.

The word "asset" is another confusing and terribly overused term. In WAS7 it's a "catch all" word for any kind of artifact, including jar, zip, and, with the introduction of OSGi support, an OSGi bundle (you need to have a beta OSGI feature pack installed to be able to use OSGi).

BLAs are created by combining applications and assets together either using admin console or scripting. BLA configuration is saved under $profile_root/config/cells/$cell_name/blas. Assets are saved under $profile_root/config/cells/$cell_name/assets.

I can see two use cases for BLAs:

• BLAs can be used to start and stop multiple Java EE applications at once. This might be useful if operationally several applications have to be treated together or if there are run-time dependencies between applications. This is especially the case if scripting is used; AdminTask.startBLA and AdminTask.startBLA provide an easy way to restart several "regular" applications.

• BLAs can be used to manage shared libraries. There is no way to deploy "traditional" shared libraries ("Environment/Shared Libraries") using Deployment Manager; you have to use scp or ftp to copy binaries to target servers. With BLA you can simply import "assets" (e.g., a jar file) using AdminTask.importAsset command. You can then reference the jar file using "SharedLibRelationship" option of AdminApp install/update commands. This will automatically create a classloader for this library (note--this might require server restart, at least for me it did). Being able to use regular administration APIs for managing shared libraries is certainly a welcome feature.

Unfortunately, BLAs also have limitations:

• I was not able to find a way to create an asset consisting of multiple jars. This severely limits the usefulness of utilizing BLAs for managing shared libraries since a library will almost always consist of multiple jars. It is possible to import jars individually, but this is tedious and difficult to manage.

• It is not possible to deploy multiple Java EE applications using BLAs. You still have to deploy applications individually.

• There is no support for version information in BLA assets. This is unfortunate. I wish IBM though about integrating BLA assets with Maven repositories and supporting true dependency management (e.g., version resolution, transitive dependencies, etc.). As of right now the only way to import multiple version of the same asset is to append a version number to the asset's file name.

To summarize, the BLA idea has some promise but its current implementation is too limited to be truly useful. I'm sure IBM will be evolving BLA in future WAS releases.

You can find more information about BLA in this article and in this webcast

This post is part of the series on WebSphere Application Server administration. Please subscribe to our blog if you'd like to receive updates.

Note: We offer professional services in the area of WebSphere architecture, implementation and operations. If you're looking for help with any of these tasks, please let us know.

Posted in WebSphere, WebSphere Application Server Administration