Beyond the Ant basics

This article was originally published in VSJ, which is now part of Developer Fusion.
In last month's VSJ we learned about a Java-based project build tool known as Ant. We saw how useful Ant is when used to orchestrate the compilation and construction of complex Java software projects, replacing the venerable "make" utility. However, the versatility of Ant isn't limited to the building of Java projects. In this article, we will show how Ant can be extended to everyday computing and management tasks. Ant can take advantage of the dynamic "task" loading and excellent networking support of the Java platform to perform some amazing things. More specifically, we will discover how Ant can:
  • assist a webmaster in synchronising web sites, and run tasks on a remote host
  • access a relational database and then conditionally execute custom logic
  • be extended to automate almost any arbitrary computing or networking task
We will investigate many of the advanced features in Ant during this brief exploration, which will include the design and coding of our very own custom Ant task.

Ant: a webmaster's best friend

Imagine for a moment that you're a professional for-hire web designer. Your part-time business involves the design and management of multiple web sites for clients. Let's assume that the web sites that you manage are relatively typical, and consist of sets of static web pages that you design and create using a web page design tool or text editor. The master copies of these web pages are on your hard disk, while the actual pages that users will access are on a remote web-hosting server. Because of different customer preferences and requirements, not all of your web sites are remotely hosted on the same server. However, all the remote servers can be accessed via the FTP protocol for uploading web pages. This scenario is quite typical.

Figure 1
Figure 1: Hypothetical scenario involving two active customer sites

In Figure 1, you are managing two web sites on different hosting servers. One is for Major Appliances, and the other one for Suzy Flowers. You edit the web pages on each site according to client instructions, and then upload the changed pages to the hosting server(s). Each web site resides in a different directory on your local hard disk. The typical maintenance cycle is:

  1. edit the copy of the web site on your hard disk based on customer's requirements>
  2. connect to remote hosting server and upload any changed files
Ant can handle step (2) with the help of the FTP Ant task (included as one of the optional tasks in Ant). For example, our build file target for Major Appliances' site is:
<target name="majorappliance">
	<ftp server="macarb.exphostbank.com"
		userid="majorapp"
		password="bargain"
		depends="yes"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset dir="mappl/htdocs"/>
	</ftp>
</target>
Since all of the files in the Major Appliance website are located under the mappl/htdocs directory, we specify that as a <fileset> sub-element of the FTP task. These are the files to synchronise. Let us examine some of the frequently used attributes in the FTP task (see Table 1).

Table 1: Attributes in the FTP task
Attribute Description
server The host name or IP address of the remote FTP server to which you are connecting. The FTP task works best with *NIX or Linux based FTP servers. There can be problems with Win32 based FTP servers unless they emulate *NIX server behaviours.
userid The login userid to be used on the remote server.
password The password to be used on the remote server.
depends Specify that a dependency is to be performed on the remote files before transfer. If this is set to "yes", only newly modified files will be transferred. Otherwise, all files specified will be transferred.
verbose Provide a detailed trace while the FTP task goes into action.
ignoreNonCriticalErrors The default behaviour of the FTP task is to halt on any error. Some FTP servers, especially Win32 and wuftp on *NIX, may report non-critical error that will stop the task's operation. Set this attribute to "yes" if you're encountering such problems.

Updating changed pages with the FTP task

The complete build file for managing the two sites is listed below. First, notice that we have created an "updatesites" target that combines the action of both the "majorapp" and the "suzyflowers" target. Each target is called via an <antcall> task. The <antcall> task is used exclusively to invoke targets defined in the same buildfile. Using this technique, we can easily add more clients to the update.
<?xml version="1.0"?>
<project name="VSJWebmaster" default="updatesites" basedir=".">

<target name="updatesites">
	<antcall target="majorappliance"/>
	<antcall target="suzyflowers"/>
</target>

<target name="majorappliance">
	<ftp server="macarb.exphostbank.com"
		userid="majorapp"
		password="bargain"
		depends="yes"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset dir="mappl/htdocs"/>
	</ftp>
</target>
<target name="suzyflowers">
	<ftp server="zeus.hostall.co.uk"
		userid="suzyflower"
		password="rosebud"
		depends="yes"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset dir="suzyf/htdocs"/>
	</ftp>
</target>
</project>
For this to work, make sure you've changed the server, userid, and password attributes to an FTP server that you can access.

Since "updatesite" is defined to be the default target for the project, you can start the update using the command line (in the code\webmaster directory):

ant –f build01.xml
The first time you perform this, all of the files in the web sites will be uploaded to the server(s). Figure 2 shows a sample output.

Figure 2
Figure 2: Sample output

Next, try modifying one or more files in each website and execute Ant again. This time, only the updated files will be uploaded – thanks to the FTP's task built-in dependency check. Figure 3 shows a sample output.

Figure 3
Figure 3: Sample output

Accelerating updates with parallel task execution

If you use the build file above to maintain 10 or 20 average sized web sites, the update to remote server can take quite a bit of time. Most of the delay comes in the form of establishing connection to the remote FTP server and waiting for transfer to complete. If there is some way to perform the web site updates concurrently, we stand to save quite a bit of time. The <parallel> task in Ant is specially designed for this.

The <parallel> tag is a task container in Ant. It specifies to the Ant runtime that each task within the tag is to be run on its own thread. The contained tasks will be executed in parallel, each to completion (either success or failure). In our case, it will enable concurrent update of the web sites, resulting in accelerated completion of the "updatesite" target and the parallel execution of the "majorapp" and "suzyflowers" targets. If you're using Ant to automate the execution of CPU intensive tasks on a multi-processor machine (say DVD video compression), the overall throughput can be dramatically increased with the use of the <parallel> task container.

To add the <parallel> task container to our build file, we simply modify the "updatesites" target:

<target name="updatesites">
	<parallel>
		<antcall target="majorappliance"/>
		<antcall target="suzyflowers"/>
	</parallel>
</target>
Now, if we execute Ant, we can see that the logon and transfer of data for each site occurs simultaneously. Use build02.xml for this:
ant –f build02.xml
Note that if you have high speed direct access to each of the remote web hosting servers the end result may not be too impressive (since each of the transfers will be competing against each other for the available bandwidth). Figure 4 illustrates a typical output, revealing the interleaved upload operations.

Figure 4
Figure 4: Interleaved upload operations

Running remote server commands with a telnet task

Readers familiar with the UNIX or Linux operating system may have experimented with the rsh and rexec commands. These commands can be used to execute commands on remote servers. Since Ant is designed to run on any system that support Java, having a built-in task for rsh and rexec on the system level would be close to impossible. It is possible, however, to get similar results to these commands by using the Ant telnet task. Using the telnet task, we literally:
  1. log on to the remote system
  2. issue the commands that needs to be executed
  3. log off from the remote system
Back to our webmaster application scenario, let's say that after each file update, we must run a script called "udpatelinks" to ensure all our external links are valid from the hosting server. To do this, we can add a telnet task to the "majorapp" and "suzyflowers" targets.
<target name="majorappliance">
	<ftp server="macarb.exphostbank.com"
		userid="majorapp"
		password="bargain"
		depends="yes"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset dir="mappl/htdocs"/>
	</ftp>
	<telnet server="macarb.exphostbank.com"
		userid="majorapp"
		password="bargain"
		>
		<read>majorapp]</read>
		<write>updatelinks</write>
		<read>majorapp]</read>
	</telnet>
</target>
The telnet task is an optional task, but included as part of the Ant distribution, that can be used to execute commands on remote hosts. It works according to a set of alternating <read> and <write> sub-elements. A <read> element will cause the task to wait for a specified string, and the <write> element will send the string specified to the remote server. In the task above, we wait for the Linux prompt from the server via:
<read>majorapp]</read>
...and then we send the command to be executed:
<write>updatelinks</write>
Finally, we wait again for the prompt to return – indicating that the command has completed. This will also enable us to see the output from the remote server on the screen. In the build03.xml file, the same telnet task has been added to the suzyflowers target for executing the updatelinks script.

To make the output less confusing, we should remove the <parallel> task container. If we now run Ant using the build03.xml build file (ant -f build03.xml), we will see how it will now connect to the remote server and issue the updatelinks command immediately after the file upload. Figure 5 illustrates the typical trace output.

Figure 5
Figure 5: Typical trace output

Shutting down non-payment sites

Let's continue our scenario. With Ant's help, the webmaster business has been quite good to us, and we have a sizable clientele. However, on a day-to-day basis, despite active reminders – there are always one or two clients who miss payment deadlines. In respect of these clients, we want to suspend their sites until payment is received.

It would be ideal if Ant could help us to automate this "site shutdown" procedure. Since our customer and accounting data is kept in a simple MS Access database, we need some way of fetching data within an Ant task. If you scan down the list of Ant tasks available, you will come across the Sql task (one of Ant's core task). This task allows us to use JDBC to execute SQL scripts on a RDBMS server. Table 2 shows several frequently used attributes of the Sql task that you must specify.

Table 2: Frequently used attributes of the Sql task
Attribute Description
driver The class name of the JDBC driver to be used to access the RDBMS.
url The JDBC URL used to locate the database instance.
userid User ID for login to the RDBMS.
password Password for the user id specified.

Unfortunately, the output alternative of the Sql task is restricted to either a file, or the screen. This is insufficient for what we want to accomplish. Ideally, we want the task to interpret the result of the Sql select (as paid or unpaid) and then set a property.

Thanks to the thoughtfulness of the Ant designers, however, they have left the core of the Sql task completely extensible. Actually, the Sql task builds on a task called org.apache.tools.ant.taskdefs.JDBCTask. It is this JDBCTask that we will extend to perform our database search.

Most of the functionality of the Sql task, including the handling of the four attributes listed above, the initialisation of JDBC driver, obtaining JDBC connection, etc. are taken care of by the JDBCTask. We can extend this task in our own custom task code – saving us from writing the tedious JDBC driver intialisation, attribute management, and connection management code.

Our custom Ant task, called clientpaid, will perform the following:

  1. take a client name as an attribute
  2. access the MS Access database to determine if the client's account is current
  3. if the account is current, set a property – the name of the property to be set is specified as an attribute of the task
Figure 6 shows the simple layout of our MS Access database, illustrating that the database contains a single table called webclients, and the table has only two columns.

Figure 6
Figure 6: Access database layout

The first column is the name of the client, the second column is a boolean flag indicating whether the client is paid up (true) or not (false).

Figure 7
Figure 7: Decision flow and action of site shutdown automation

Figure 7 reveals the decision flow and action of the site shutdown automation, and we can see that the updatesite target will either:

  1. perform the regular site update if the account is paid up
    OR
  2. clean out all the files in the remote server, and post a delinquency notice as the home page of the site on any delinquent account

Creating a custom DB access Ant task

Any custom Ant task must be a descendent of org.apache.tools.ant.Task. Our task will extend org.apache.tools.ant.taskdefs.JDBCTask which in turn inherits from the required org.apache.tools.ant.Task class. Our custom task is called com.vsj.ant.tasks.CustSqlTask. There are two more attributes that we want to implement for CustSqlTask, beyond those already implemented by JDBCTask. These are:

Attribute Description
client The name of the client whose account is to be verified.
property The property to set if the account is paid. If the account is not paid, no property will be created.

Each of the above properties must have a corresponding set<property name>() method implemented in CustSqlTask.

The following is a listing of the custom task. You'll find the source in the webmaster\src directory of the distribution. The core logic of the task is in the execute() method that the custom task must define. Note the highlighted code:

  1. setting the property if the account is paid in execute()
  2. handling the two new attributes via setter methods.
package com.vsj.ant.tasks;

import org.apache.tools.ant.
BuildException;
import org.apache.tools.ant.
taskdefs.JDBCTask;

public class CustSqlTask extends JDBCTask {
	private String client = null;
	private String propName = null;
	private String valueToSet = "true";

	public void execute() throws BuildException {
		Connection conn = getConnection();
		Statement stmt=null;
		try {
			if (client == null ) {
				throw new BuildException("client must be specified",location);
				}
			if (propName == null ) {
				throw new BuildException("property must be specified",location);
				}
			String exsql =
			"SELECT PAID FROM WEBCLIENTS WHERE CLIENT='" + client + "';";
			stmt= conn.createStatement();
			ResultSet rs = stmt.executeQuery(exsql);
			while (rs.next()) {
				boolean paid = rs.getBoolean(1);
				if (paid) {
					project.
			setProperty(
			propName,
					valueToSet);
					}
				}
			} catch (SQLException e) {

			} finally {
			if (stmt != null) {
				try {stmt.close();}catch (SQLException ex){}
				}
			if (conn != null) {
				try {conn.close();}catch (SQLException ex){}
				}
			}
		}

		public void setClient(String inClient) {
			this.client = inClient;
			}

		public void setProperty(String inPropName) {
			this.propName = inPropName;
			}
		}
The JDBCTask superclass substantially simplifies the development of any custom Ant tasks that needs to work with relational data sources.

To compile and test this custom class, we have created yet another Ant build file. Let us take a look at build04.xml. The default target is "build" for creating the custom task.

<?xml version="1.0"?>
<project name="VSJCustTask" default="build" basedir=".">
The "build" target simply uses the <javac> task (which you will recall from last month's article) to compile the source tree under the src directory – and places the resulting binary under the build directory.
<target name="build" >
	<mkdir dir="build"/>
	<javac srcdir="src" destdir="build"/>
</target>
To make the custom task available to the build file, we need to define a new task. This is done via the <taskdef> task. Here, we create a "declare" target that associates the CustSqlTAsk custom task with a taskname of "clientpaid". Note the dependency on the "build" target.
<target name="declare" depends="build">
	<taskdef name="clientpaid"
		classname=
"com.vsj.ant.tasks.CustSqlTask"
		classpath="build"/>
</target>
Finally, we'll write a simple test target called "testdb". It depends on the "declare" target, since it requires the <clientpaid> task. Note how we assign the attributes to the client and property attributes (managed by our CustSqlTask) and also the driver, url, userid and password attributes (managed by the JDBCTask superclass). After this task executes, a "suzpaid" property will be set if the account is paid up. Then we call the "testsuzy" target.
<target name="testdb" depends="declare">
	<clientpaid client="suzyflowers" property="suzpaid"
		driver="sun.jdbc.odbc.JdbcOdbcDriver"
		url="jdbc:odbc:vsjwebmaster"
		userid="" password="" />
	<antcall target="testsuzy"/>
</target>
Note the use of the if attribute in the "testsuzy" target given below. The if attribute (and the unless attribute) allow you to test for the existence of a property and conditionally execute the task. In this case, the <echo> task will only be executed if "suzpaid" is set previously by our <clientpaid> task.
<target name="testsuzy" if="suzpaid">
	<echo message="suzy is paid up!"/>
</target>
</project>
To build and test this custom task, first ensure that you have created an ODBC system datasource, named VSJWEBMASTER, pointing to the clients.mdb file from the code\db directory of the source code distribution. You can create this datasource using the "Data Sources (ODBC)" applet in the control panel of your Win32 OS.

Once the datasource is created, you can compile and execute the test by using the command line:

ant –f build_04.xml testdb
Since the account suzyflowers is paid up in the database, you should see output similar to Figure 8.

Figure 8
Figure 8: Sample output

You should go into the MDB and clear the PAID field for suzyflowers, and then try running the test again.

Integrating our custom Ant task to drop unpaid sites

Now that the <clientpaid> task has been proven to be working, we can integrate the custom task into our working build file. The complete integrated build file is in build_05.xml. The following are some highlights.

We modified our updatesites target to call the <clientpaid> task before calling either of the customer targets. We are also calling two new targets, "majordrop" and "suzydrop". These targets are used to conditionally shutdown the client's web site.

<target name="updatesites" depends="declare">
	<clientpaid client="majorapp" property="majpaid"
		driver=
"sun.jdbc.odbc.JdbcOdbcDriver"
		url="jdbc:odbc:vsjwebmaster"
		userid="" password="" />
	<antcall target="majorappliance"/>
	<antcall target="majordrop"/>
	<clientpaid client="suzyflowers" property="suzpaid"
		driver=
"sun.jdbc.odbc.JdbcOdbcDriver"
		url="jdbc:odbc:vsjwebmaster"
		userid="" password="" />
	<antcall target="suzyflowers"/>
	<antcall target="suzydrop"/>
</target>
If we examine one of the client targets, say "majorappliance". We will see the following change:
<target name="majorappliance" if="majpaid">
	<ftp server="macarb.exphostbank.com"
		userid="majorapp"
		...
Note that now the web site update will be executed only if the "majpaid" property is set by the <clientpaid> task. Conversely, you will see that the new "majdrop" target will be executed unless the "majpaid" property is set. Here is the segment containing this new "majdrop" target.
<target name="majordrop" unless="majpaid">
	<ftp
		action="del"
		server="macarb.exphostbank.com"
		userid="majorapp"
		password="bargain"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset>
			<include name="**"/>
		</fileset>
	</ftp>
	<ftp server="macarb.exphostbank.com"
		userid="majorapp"
		password="bargain"
		depends="yes"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset dir="unpaid/htdocs"/>
	</ftp>
</target>
This target is executed if the Major Appliance account is unpaid. It uses the FTP task to delete all files at the remote host for the account. It then connects to the server again to upload a home page, from the unpaid/htdocs directory, that requests the owner to call and pay for the account.

Using this technique, we can conditionally shutdown web sites depending on their accounting records stored on an RDBMS data source. Test this by using the command:

ant –f build_05.xml
Try changing the PAID field of the two clients in the MDB file, and observe the changing behaviour of the build file.

This concludes our exploration of the advanced features of Ant, going beyond building Java development projects. However, looking at the resulting build_05.xml file, we realise that it is relatively difficult to maintain. Let us take a quick look at how to tidy up the build file.

Storing passwords and other properties in external files

One major problem in build_05.xml are the string literals for server name, user id, password, etc. that are repeated throughout the build file. We can readily factor them out, and even place them in an external file.

build_06.xml is the cleaned up version of build_05.xml. You will see the following statement in the updatesites target:

<target name="updatesites" depends="declare">
	<property file="accounts.properties"/>

	<clientpaid client="majorapp" property="${mapp.paidprop}"
		driver="${jdbc.driver}"
		...
The <property> task directs Ant to read in the "account.properties" file and set all the properties. Our accounts.properties file contains:
jdbc.driver=sun.jdbc.odbc.JdbcOdbcDriver
jdbc.url=jdbc:odbc:vsjwebmaster
jdbc.user=
jdbc.pass=

mapp.server=macarb.exphostbank.com
mapp.user=majorapp
mapp.pass=bargain
mapp.paidprop=majpaid
mapp.htmlDocDir=mappl/htdocs

suzy.server=zeus.hostall.co.uk
suzy.user=suzyflower
suzy.pass=rosebud
suzy.paidprop=suzpaid
suzy.htmlDocDir=suzyf/htdocs
Subsequently inside the build file, any property may be referenced using the notation:
${property name}

Parameterising an Ant Target for Reuse

Last but not least, you will notice that the "majordrop" and "suzydrop" targets have been removed. Since both targets do the same thing, minus some small variations in parameterisation, we've created a reusable "sub-routine" target called "dropsite" instead. Here is the "dropsite" target:
<target name="dropsite" unless="${dropprop}">
	<ftp
		action="del"
		server="${server}"
		userid="${user}"
		password="${pass}"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset>
			<include name="**"/>
		</fileset>
	</ftp>
	<ftp server="${server}"
		userid="${user}"
		password="${pass}"
		depends="yes"
		verbose="yes"
		ignoreNoncriticalErrors="yes">
		<fileset dir="unpaid/htdocs"/>
	</ftp>
</target>
To perform the former function of the "majordrop" target, we see the use of <param> sub-element in the <antcall> task. This fully parameterises the reusable "dropsite" target to shutdown the Major Applicance site.
<antcall target="dropsite">
	<param name="dropprop" value="${mapp.paidprop}"/>
	<param name="server" value="${mapp.server}"/>
	<param name="user" value="${mapp.user}"/>
	<param name="pass" value="${mapp.pass}"/>
</antcall>
Apart from better maintainability, build_06.xml performs exactly as build_05.xml. You can test it to your satisfaction.

Conclusions

Ant's versatility goes way beyond a simple Java project build tool. In fact, Ant can be used as an automation utility that supports automation of tedious computing, networking, and data access tasks. There are Ant tasks (core or optional) that may satisfy your automation needs already. In the rare cases where the available tasks don't quite do exactly what you want you can always write your own custom Ant task to get the job done.


Sing Li is a consultant, trainer and freelance writer specialising in Java, web applications, distributed computing, and peer to peer technologies. His recent publications include Early Adopter JXTA, Professional JINI and Professional Apache Tomcat, which are all published by Wrox Press.

In order to get the FTP task to work, you need to download an external library that is not supplied in the Ant distribution. Many optional Ant tasks require external libraries (see Ant documentation for more details if another task you try does not work properly).

In our case, download the Net Components library, netcomponents.jar, from www.savarese.org/oro/downloads. Place this JAR file into your <ant installation>/lib file for proper operations.

You might also like...

Comments

About the author

Sing Li United States

Sing Li has been writing software, and writing about software for twenty plus years. His specialities include scalable distributed computing systems and peer-to-peer technologies. He now spends ...

Interested in writing for us? Find out more.

Contribute

Why not write for us? Or you could submit an event or a user group in your area. Alternatively just tell us what you think!

Our tools

We've got automatic conversion tools to convert C# to VB.NET, VB.NET to C#. Also you can compress javascript and compress css and generate sql connection strings.

“There's no test like production” - Anon