PAnt 2.0.1 is Released

PAnt 2.0.1 contains an important bug fix that was preventing using certain Ant tasks from python. Specifically, the bug affected all tasks that utilize addConfigured method to handle nested elements. This included the "manifest" task and several others.

You can download PAnt 2.0.1 from the PAnt project page

PAnt Documentation Has Been Updated

We've just updated PAnt documentation to reflect changes implemented in PAnt 2.0. There are two significat changes:

• Support for Ant targets.
• PAnt is now bundled with Ant Jython tasks that previously were available separately.

You can follow this guide to get started with PAnt.

Creating Ant Targets in Python with PAnt 2.0

We've just released a new version of PAnt, a tool for developing build scripts in python. The key new feature of PAnt 2.0 is the ability to create Ant targets directly from python. This allows for completely getting rid of Ant XML files and for developing build scripts entirely in Python.

Here is an example:
@target(unless="prop.name", depends=ptarget1)
def p_target2():
"""Python project ptarget2"""
ant.echo("echo ptarget2")

You can find more details about creating targets from PAnt here.

PAnt 1.5 is Released

This is a major update to our popular python Ant wrapper.

Notable changes in this release:

• Support for positional (as opposed to named) arguments, e.g., “pant.echo(‘message’)”.
• Support for lists to express repeating elements.
• Support for “ant_” prefix to avoid conflicts with python keywords.

More information is available from PAnt project page

Please subscribe to our feed or follow us on twitter to continue receiving updates about PAnt – new version is coming shortly.

Ant Task without Setters

Ant uses reflection to pass data from XML to the Java class that implement an Ant task. For every attribute in XML, you have to define a setter in the Java task’s class.

This works fine most of the time, however, in some cases there could be a need for a dynamic list of attributes. For example a task can pass attribute values to some external tool that has its own set of parameters that you don’t want to hardcode in Ant. Or you may simply like the flexibility of using dynamic attributes as opposed to predefined setters.

In order to implement dynamic attributes, first you need to override “maybeConfigure” method in your Ant task and have it do nothing:

public void maybeConfigure() throws BuildException {
}

Then in your “execute” method you can access the map of attributes (that represents all attributes set in XML) as follows:

RuntimeConfigurable configurator= getRuntimeConfigurableWrapper();
Map attributes=configurator.getAttributeMap();
String attr1=(String)attributes.get("attr1");

Using Jython 2.2.1 with WSAdmin Tool

In one of the previous posts I complained that wsadmin tool (which is the main WebSphere Application Server administration tool) still relies on Jython 2.1, which is quite old.

The issue became critical when I realized that jython automatically re-compiles classes compiled with a different jython version. In my case, I was using Jython 2.2 for my Ant scripts and Jython 2.1 for wsadmin scripts. Some of the code was shared. This led to the situation of concurrent builds stepping on each other by dynamically re-compiling classes with different jython versions. The error looked something like that:

File "<string>", line 5, in ?
java.lang.ArrayIndexOutOfBoundsException: -4
at org.python.core.imp.createFromPyClass(Unknown Source)

Bugs like that are always fun to troubleshoot.

Going back to 2.1 was not an option for me since I use closures and “new classes” in quite a few places. So I tried putting jython 2.2.1 on wsadmin classpath and it worked without a hitch with thin wsadmin client. All my of wsadmin scripts work without a problem.

Here is a sample wsadmin.bat file that I use. This file utilizes thin client jars. Note how in line 85 my jython.jar (which is jython 2.2.1) is added to the classpath so it would override jython packages supplied with WAS.

One possible downside of this approach is running into issues with IBM support in case if you need to open a PMR related to wsadmin scripting.

Updated Jython Ant Task

I've updated Ant Jython task with a number of new features:

• Jython path is now handled by a separate JythonPath task.

• Jython interpreter is now scoped to Ant project. This means that you can have multiple Jython calls withing the same project that share common imports and variables.

• Jython task now supports nested text, similar to the "script" task.

Ant Jython Tasks (PAnt Tasks)

PAnt build tool comes with several Ant tasks to facilitate the use of Jython/Python from Ant.

PAnt tasks have a number of advantages over built-in <script language="jython"> way of invoking Jython from Ant:

• More graceful exception handling. Jython code invoked using "script" generates long error stack that contains full stack trace of the "script" task itself. Sifting through the traces and trying to distinguish Java trace from Python trace is quite painful. PAnt "jython" task produces brief readable python-only error stack.
• You can use Ant properties as parameters ("jython" task makes them available in the local namespace of the calling script).
• Convenience "import" attribute.
• "jythonInit" task allows for setting python.path using Ant path structure.
• Jython interpreter is initialized once per Ant project. All scripts invoked from the same Ant project reuse the same built-in namespace. So you can define variables and imports in one call and use them in a subsequent call.
• Task name ( the name that prefixes all console output from Ant for a given task) is generated automatically based on the supplied Python code.
• "verbose.jython" property triggers verbose output for jython-related tasks only. This is much easier than trying to scan through hundreds of lines of general "ant -v" verbose log.

Example:

Ant code:

<jythonInit pythonPathRef="python.path" />
<property name="testProp" value="testVal" />

<jython>
print "Property from ant:", testProp
# define a var that we can use in other scripts
s="test"
</jython>

<jython>
print "Var created earlier: ",s
</jython>

<jython  import="from testmodule import *" exec="test(testProp)"  />

"testmodule" python code:

from pant.pant import project 
def test (prop):
    print "Passed parameter: ",prop
    print "Test property: ", project.properties["testProp"]

Please refer to this build.xml file for more examples.

The tasks can be used independently of PAnt python code.

PAnt Ant Tasks Reference

Getting Started

Download PAnt, extract pant.jar and create "taskdef" as described here

"jythonInit" Task

The tasks initializes jython interpreter. Because of the overhead, the interpreter is initialized only once even if jythonInit is invoked multiple times. The repeating calls are simply ignored.
jythonInit automatically adds pant.pant module to PYTHONPATH.

Attributes:

• pythonPathRef - PYTHONPATH (python.path) to use, given as reference to a PATH defined elsewhere. Required if "pythonPath" nested element was not provided.
• pythonHome - location of python distribution (optional). If provided,jythonInit will set python.home system property and will automatically add ${python.home}/Lib to the python path if ${python.home}/Lib exists.
• cacheDir - location of jython cachedir used for caching packages (optional). Defaults to ${java.io.tmpdir}/jython_cache (note-- this is different from default jython behavior).

Nested elements:

pythonPath - python.path to use defined using Ant path-like structure. Required if "pythonPathRef" attribute was not provided.

Special properties:

log.python.path - if set to "true", jythonInit will print python path to Ant log. Default: false.

"jython" Task

Invokes python code.
Note: by default, jython does not print python stack trace in case of an exception. To see the trace, run Ant in verbose mode using "-v" or use "-Dverbose.jython=true" property.

Attributes:

• exec - Python code snippet to execute. Typically, this is a function from a module available from python.path. This has to be a single line, e.g., mod.fun() although you could combine multiple statements separated by ";". Required if "execfile" was not provided.
• import - a convenience attribute for providing "import" statement. Its only purpose is to make the task invocation more readable. Alternatively, you can have "import" as part of the"exec",e.g., exec="import mod;mod.fun()". Optional.
• execfile - path to a python script file. Required if "exec" was not provided.

Nested elements:

Inline text with python code.

Special properties:

verbose.jython - if set to "true", jython will print additional information about executing python code to Ant log. Default: false.

pimport Task

Creates Ant targets from a python module. Functions that will be used as targets have to be marked using "@target" decorator as described here.
Python module name is used as Ant project name. Target overriding works the same way with Ant import task. In other words, targets defined using pimport will override targets previously defined using "import" or "pimport" tasks.

Attributes:
module - python module to create targets from. The module has to be available from python.path specified using jythonInit.

Jython Ant Wrapper Examples

Somebody asked me about examples for PAnt Jython wrapper. Here are some. I'll be updating this page with more examples in the future.

Following is a simple echo task. Note that pant.py has to be on your "python.path". You can set it by adding it to your ANT_OPTS environment variable (ANT_OPTS=-Dpython.path=python_path).

        <script language="jython">
from pant import *
pant=PAnt(project)
pant.execTask(name="echo", message="foo")
        </script>

Following is a copy task example. Note the use of "nested" function to denore nested elements. "expandproperties" is assigned an empty dictionary since it does not have any attributes.

pant = PAnt( project )
pant.copy(todir="${test.prop}", fileset=nested(dir=".", include=nested(name="*.xml")), 
          filterchain=nested(expandproperties={}) ) 

Following is an example of Exec task. Multiple "env" elements are distinguished by adding the suffix "_number". The suffix can in fact be anything, PAnt, simply ignores the substring starting with underbar. This example also demonstrates how you can mix and match python variables and Ant properties in the same piece of code.

shellFile="myshell.sh"
commandLine="options"
pant.exec(dir="${bin.dir}", executable=shellFile,  failonerror="true", resultproperty="result.code",
          env_1=nested(key="key1", value='${val1}'),
          env_2=nested(key="key2", value='${val2}'),
          env_3=nested(key="key3", value='${val3}'),
          arg=nested(line=commandLine) )

Update:
MyArch Jython Task provides tighter integration of Ant and Jython. You may want to use it together with PAnt.

Please refer to our official PAnt project page for more information and to download PAnt

Ant Scripts without XML

It's pretty easy to create an Ant file for a simple project. A simple Ant script typically contains ubiquitous "init", "compile", "test", "war" (or "jar), "build" targets all wired together. It's easy to change and easy to understand and the script's flow has a declarative, rule-based feel to it. The problem is, projects and their build files rarely stay simple for long. Soon we need to add "validate.xml" target, junit reports, deployment to your application server and so on. Then we begin supporting multiple environments; we discover that our local desktop configuration is different from how integration environment is configured so we add numerous property files, "ftp" tasks and multiple "copy" targets for various application files. Before we know it, the build script becomes a convoluted mess of XML tags and there is nothing declarative about it anymore; it's morphed into a full-fledged, very procedural program. Perhaps we even had to resort to using ant-contrib "if" and "for" targets to implement procedural logic in Ant. And nothing is uglier than an "if" with complex conditions expressed in XML.

A better approach would be to implement "procedural" portion of the build script in Java or any of the scripting languages that Ant supports. The problem is, configuring and invoking Ant tasks from Java or a scripting language leads to verbose code. For example:

    execTask = project.createTask("exec")
    execTask.setOutputproperty(outputPropertyName)
    execTask.setErrorProperty(errorPropertyName)
    execTask.setResultProperty(resultPropertyName)
    execTask.setExecutable(execName)
    arg=execTask.createArg()
    arg.setLine(paramString)

Doing the same thing in XML is shorter and cleaner:

<exec executable="${execName}" outputPropert="p1" 
    errorProperty="p2" resultProperty="p3">
    <arg line="${params}" />
</exec>

So what can we do to make task invocation syntax more concise and easier to understand? In fact, the syntax could be drastically simplified with the help of simple Ant "adapters" that can be developed for popular scripting languages since Groovy, Ruby and python all have fairly intuitive syntax for supporting lists, dictionaries and other data structures. I developed such an adapter for jython. It uses python named arguments and dictionary syntax, so executing a "copy" task looks like this:

pant=PAnt(project)
pant.exTask("copy", todir="testdir",  fileset={"dir":".","includes":"*.xml"} )

"PAnt" is the name of the "ant adapter" class for Jython. The class configures and executes Ant tasks based on the provided arguments using Ant task configuration API.

"pant" module also comes with a simple helper function called "nested" so that named arguments can be consistently used for nested elements. With syntax highlighting supported by most editors/IDEs (e.g., you can try PyDev for jython/python development), it allows for better visual distinctions between attribute names and values:

pant.exTask("copy", todir="testdir",  fileset=nested(dir=".", includes="*.xml") )

To use "PAnt" from Ant, you can develop custom tasks using "scriptdef" or simply embed python code directly into a target:

    <target name="test.pant" >
        <script language="jython">
from pant import *
pant=PAnt(project)
pant.execTask(name="echo", message="foo")
        </script>
    </target>

The "pant" module itself is just a few lines of code as you can see from its code. Don't forget to properly setup your "python.path" if you want to make it globally available to all your Ant scripts.

There is also an open-source project Gant that provides similar (in fact, much more extensive) capabilities for Groovy, but I have not had a chance to play with it; I specifically wanted to use python/jython because jython can also be used for WebSphere Application Server administration.

In my mind scripting language-based approach for writing build files provides for much more flexible and easier to understand and maintain scripts. When you start implementing your Ant logic in python, you'll see that Ant targets become much more coarse-grained, since there is no need to create internal targets (the ones that are never invoked by the users) to simulate subroutines or conditional targets to simulate "if" statements . It is also nice to be able to use all the capabilities of a full-blown programming language as oppose a limited subset of procedural tasks that Ant provides (such as "condition"). Being able to user properly scoped variables instead of inherently global Ant properties is another great benefit.

At the same time, it is still possible to use Ant targets for expressing a flow wiring together major functions of the build script. I would prefer something less XML-like for this purpose too, but that's a task for another day.

Please refer to our official PAnt project page for more information and to download PAnt