Category Archives: Building software

Yet Another Build Server

Thoughtworks has finally released a successor to their venerable Cruise Control - Cruise build server. The UI certainly looks nice and it seems quite flexible. There is even a free version (which is limited to two computers), which is great.

What is not clear though is how this product is different from AntHill, Buildforge, Bamboo, TeamCity, Gauntlet and the likes. The field is certainly becoming crowded - and I haven't even mentioned numerous open-source contenders. All these products seem to be doing the same thing - organizing your build scripts, interfacing with version control, running builds on a distributed "build server farm", collecting statistics, publishing reports and providing UI for all these functions.

All these features are important and useful. Ironically, however, what build servers don't do is automatically building or deploying your software. You still have to write Ant or Maven scripts, define and manage configuration parameters (using properties, XML, environment variables), deal with different environments (if I'm not mistaken, AntHill is the only product that has an explicit concept of an environment). For a complex project this could be a lot of work. Granted, every project is unique (if not, just use the default Maven configuration and you're done), so this could be a tough nut to crack. It should however be possible for a build server to have enough intelligence to infer how to build a project directly from the code base.

Going beyond Builds with Build Servers

A build server provides great benefits to a development team, including:

  • Convenient way of sharing build results and artifacts.
  • Ability to run builds without having to mess with command line parameters.
  • Tight integration with version control.
  • History and statistics.
  • Accountability and audit (e.g., who ran what build).

In most instances build servers, such as Hudson, Continuum or LuntBuild are used mostly for continuous integration and development builds. But they can be used for much more than that. A build server provides a generic framework for performing any kind of deployment or operations-related activity, including deployment of an application into acceptance and production environments, configuring of an app server (e.g., setting up data sources and JMS destinations) and even app server monitoring.

Several things make it possible. Build servers have flexible scheduling capabilities allowing to schedule any kind of repetitive activity, such as “pinging” a server every 15 minutes. Integration with version control makes it easier to manage the scripts. Very often, scripts for administration or configuration tasks are created in an ad-hoc fashion with multiple versions scattered around different servers and directories. A build server is able to check out a specific version of a script from a version control system and execute it. In other words, a build server helps treating admin scripts same way with business applications. Additionally, build servers can publish logs. This is powerful, since it lets anyone (who has permissions) see results of configuration activities or monitoring scripts. Finally, build servers do a great job of notifying all interested parties via e-mail, RSS, IM and other means of communication.

Security could be another important benefit of leveraging a build server outside of development builds. Usually, access to production environments is tightly controlled. In many organizations, developers have no access to production whatsoever. Build server can help enforce this policy yet give developers visibility into the details of production deployment. This can be done by creating a separate build configuration just for production deployments.

All the benefits that I mentioned in the beginning of this post also apply to non-build activities. For example, the ability to see history of all production deployments can be very valuable.

Not all build servers are created equal, especially open source ones. Security, for one, could be supported and implemented very differently in different products. Support for various version control features, such as labels and branches, also varies. In other words, careful evaluation is required before embarking on an effort of expanding the build server beyond development. But more importantly, this effort requires buy-in from operations and support organization and other groups that will become users of the build server.

As an example, with one of our clients we’ve been able to implement a number of interesting non-build functions using a build server (we started with LuntBuild and now migrated to Hudson):

  • Deployment of pre-built EAR/WAR files into test and production environments.
  • Configuration of application servers. In our case, most of the app server resources are shared, so we run the configuration script separately from application deployments. In fact, we treat configuration as a separate “unit of release” with its own lifecycle.
  • Monitoring of application servers. We have a script which tests health of application servers using JMX.
  • Log scanning. We’ve implemented a custom log scanning tool which “greps” logs for various patterns. The build server sends notification if a pattern is found in the logs.

Many of these functions can be accomplished using various commercial system management and monitoring products. These products however can get expensive and very often organizations simply don’t have enough licenses to cover all environments. So a combination of custom scripts with a build server provides a viable alternative.

Using Commons Logging from Ant

Here is a common problem with custom Ant tasks. A typical task is implemented using multiple classes, so classes that don't extend class don't necessarily have to have a dependency on Ant APIs. For example, it is pretty easy to pass Ant properties in a hashtable instead of passing the entire project object. This makes the custom task's code more generic and reusable.

One little issue that still remains is logging. Ant users are accustomed to running Ant with -verbose option which tells Ant tasks to print more detailed information. Oftentimes, -verbose is the only way to debug a build.

Unfortunately, using Ant logging requires access to Project or Task objects. As a result, the dependency on Ant APIs permeates the code that otherwise could have remained generic.

My solution for this is to use a simple class that implements org.apache.commons.logging.Log interface so that we can use jakarta commons logging (JCL) APIs instead of using Ant logging directly.
The class is called AntCommonsLogger.

To initialize AntCommonsLogger, users can either invoke "antCommonsLoggerInit" task in the beginning of a project or call AntCommonsLogger.init( getProject()) somewhere in their custom task class. After that, AntCommonsLogger becomes the default logger so that any class can use the the familiar commons logging pattern without any changes:

private static Log logger = LogFactory.getLog(CustomTask.class.getName());

"info" messages (i.e. calls to display during normal Ant execution, "debug" messages display if "-verbose" was specified and "trace" messages display if "-debug" was specified. This is a bit counterintuitive but this is the best we could do given that Ant's "verbose" does not have a direct counterpart in JCL.

To begin using AntCommonsLogger, download myarch-antutil and add antCommonsLoggerInit task definition to your project, e.g.:

    <taskdef resource="com/myarch/antutil/">
            <pathelement path="${basedir}/myarch-antutil.jar"/>
            <!-- lib.dir must contain commons-logging -->
            <fileset dir="${lib.dir}" />

You also need to add myarch-antutil.jar and commons-logging.jar to the classpath of your custom tasks.

Note that calling antCommonsLoggerInit makes AntCommonsLogger the default logger for this JVM instance. This means that all Java classes invoked by this Ant script (e.g., using "java" task) that use JCL, will use AntCommonsLogger instead of java.util logging or log4j. If this is not what you want, call AntCommonsLogger.init(getProject()) at the begging of your custom task and AntCommonsLogger.restorePreviousDefault() at the end.

Download myarch-antutil

Jython Ant Wrapper Examples

Somebody asked me about examples for PAnt Jython wrapper. Here are some. I'll be updating this page with more examples in the future.

Following is a simple echo task. Note that has to be on your "python.path". You can set it by adding it to your ANT_OPTS environment variable (ANT_OPTS=-Dpython.path=python_path).

        <script language="jython">
from pant import *
pant.execTask(name="echo", message="foo")

Following is a copy task example. Note the use of "nested" function to denore nested elements. "expandproperties" is assigned an empty dictionary since it does not have any attributes.

pant = PAnt( project )
pant.copy(todir="${test.prop}", fileset=nested(dir=".", include=nested(name="*.xml")), 
          filterchain=nested(expandproperties={}) ) 

Following is an example of Exec task. Multiple "env" elements are distinguished by adding the suffix "_number". The suffix can in fact be anything, PAnt, simply ignores the substring starting with underbar. This example also demonstrates how you can mix and match python variables and Ant properties in the same piece of code.

pant.exec(dir="${bin.dir}", executable=shellFile,  failonerror="true", resultproperty="result.code",
          env_1=nested(key="key1", value='${val1}'),
          env_2=nested(key="key2", value='${val2}'),
          env_3=nested(key="key3", value='${val3}'),
          arg=nested(line=commandLine) )

MyArch Jython Task provides tighter integration of Ant and Jython. You may want to use it together with PAnt.

Please refer to our official PAnt project page for more information and to download PAnt

Ant Scripts without XML – Jython Ant Wrapper

In my previous post I blogged about my attempts to replace XML-based syntax for invoking Ant tasks using Jython scripts. But I wasn't fully satisfied with the result - I did not like that fact that I had to pass a task name as a parameter to PAnt.execTask, e.g. pant.execTask("mkdir", dir=buildDir). It just was not intuitive enough. In Ant a task is equivalent to a subroutine, so I really wanted to use the task name as a function name. So I played a bit with dynamic dispatching in Jython and after a simple override of __getattr__ I was able to invoke Ant tasks using this syntax:

PAnt (my Ant wrapper) treats any method call as a request to execute an Ant task (except for explicit calls to "execTask" method). This allows for an elegant (in my mind) and concise syntax. The updated PAnt script can be downloaded from here.

Meanwhile, I've started using this wrapper in earnest in a large-scale enterprise build system that I'm working on. So far, I've been absolutely thrilled with the results. This makes writing any non-trivial Ant target so much easier. I'm really hoping that this could make build script development less dull and daunting.

Please refer to our official PAnt project page for more information and to download PAnt

Ant Scripts without XML

It's pretty easy to create an Ant file for a simple project. A simple Ant script typically contains ubiquitous "init", "compile", "test", "war" (or "jar), "build" targets all wired together. It's easy to change and easy to understand and the script's flow has a declarative, rule-based feel to it. The problem is, projects and their build files rarely stay simple for long. Soon we need to add "validate.xml" target, junit reports, deployment to your application server and so on. Then we begin supporting multiple environments; we discover that our local desktop configuration is different from how integration environment is configured so we add numerous property files, "ftp" tasks and multiple "copy" targets for various application files. Before we know it, the build script becomes a convoluted mess of XML tags and there is nothing declarative about it anymore; it's morphed into a full-fledged, very procedural program. Perhaps we even had to resort to using ant-contrib "if" and "for" targets to implement procedural logic in Ant. And nothing is uglier than an "if" with complex conditions expressed in XML.

A better approach would be to implement "procedural" portion of the build script in Java or any of the scripting languages that Ant supports. The problem is, configuring and invoking Ant tasks from Java or a scripting language leads to verbose code. For example:

    execTask = project.createTask("exec")

Doing the same thing in XML is shorter and cleaner:

<exec executable="${execName}" outputPropert="p1" 
    errorProperty="p2" resultProperty="p3">
    <arg line="${params}" />

So what can we do to make task invocation syntax more concise and easier to understand? In fact, the syntax could be drastically simplified with the help of simple Ant "adapters" that can be developed for popular scripting languages since Groovy, Ruby and python all have fairly intuitive syntax for supporting lists, dictionaries and other data structures. I developed such an adapter for jython. It uses python named arguments and dictionary syntax, so executing a "copy" task looks like this:

pant.exTask("copy", todir="testdir",  fileset={"dir":".","includes":"*.xml"} )

"PAnt" is the name of the "ant adapter" class for Jython. The class configures and executes Ant tasks based on the provided arguments using Ant task configuration API.

"pant" module also comes with a simple helper function called "nested" so that named arguments can be consistently used for nested elements. With syntax highlighting supported by most editors/IDEs (e.g., you can try PyDev for jython/python development), it allows for better visual distinctions between attribute names and values:

pant.exTask("copy", todir="testdir",  fileset=nested(dir=".", includes="*.xml") )

To use "PAnt" from Ant, you can develop custom tasks using "scriptdef" or simply embed python code directly into a target:

    <target name="test.pant" >
        <script language="jython">
from pant import *
pant.execTask(name="echo", message="foo")

The "pant" module itself is just a few lines of code as you can see from its code. Don't forget to properly setup your "python.path" if you want to make it globally available to all your Ant scripts.

There is also an open-source project Gant that provides similar (in fact, much more extensive) capabilities for Groovy, but I have not had a chance to play with it; I specifically wanted to use python/jython because jython can also be used for WebSphere Application Server administration.

In my mind scripting language-based approach for writing build files provides for much more flexible and easier to understand and maintain scripts. When you start implementing your Ant logic in python, you'll see that Ant targets become much more coarse-grained, since there is no need to create internal targets (the ones that are never invoked by the users) to simulate subroutines or conditional targets to simulate "if" statements . It is also nice to be able to use all the capabilities of a full-blown programming language as oppose a limited subset of procedural tasks that Ant provides (such as "condition"). Being able to user properly scoped variables instead of inherently global Ant properties is another great benefit.

At the same time, it is still possible to use Ant targets for expressing a flow wiring together major functions of the build script. I would prefer something less XML-like for this purpose too, but that's a task for another day.

Please refer to our official PAnt project page for more information and to download PAnt