• Three different FTP protocols should be supported, namely FTP, FTPS and SFTP
• It should be possible to transmit different files to several different servers
• The files to be sent were generated in runtime, they had different file names and were stored in different directories

Basically, I should implement the following interface:

FtpSender.java

public interface FtpSender {

    /**
     * Uses the {@code ftpProperties} to transmit the provided {@code file} to a remote server
     *
     * @param ftpProperties The FTP properties of the remote server
     * @param file The file to transmit
     */
    void sendFile(FtpProperties ftpProperties, File file);
}

where the FtpProperties was another interface:

FtpProperties.java

public interface FtpProperties {

    /**
     * Gets the protocol
     * @return One of {@code ftp}, {@code ftps} or {@code sftp}
     */
    String getProtocol();

    /**
     * Gets the user name
     * @return The name of the user
     */
    String getUserName();

    /**
     * Gets the password
     * @return The password
     */
    String getPassword();

    /**
     * Gets the FTP host
     * @return The FTP host
     */
    String getHost();

    /**
     * Gets the name of the directory on the server where the file will be transferred
     * @return The name of the remote directory
     */
    String getRemoteDirectory();

    /**
     * Gets the passive mode, e.g. if the server is behind a firewall
     * @return whether or not passive mode should be used
     */
    boolean getPassiveMode();
}

Rather than implementing the entire FTP client from scratch, I investigated the capabilities of existing frameworks. Most solutions recommend using an existing FTP framework such as Apache Commons / Jakarta Commons Net for FTP and FTPS and then wrap it in a SSH layer like Jsch / Java Secure Channel for SFTP. However, soon I discovered Apache Camel, "a powerful open source integration framework based on known Enterprise Integration Patterns with powerful Bean Integration" (their words). They support a various range of components, anything from XQuery and Atom to XSLT and SQL, just to name a few. To my luck, they also support all three FTP protocols that I needed.

Fundamentals

What does the Camel FTP API look like? In its basic form it is a little more than a one-liner:

FtpRouteBuilder.java

public class FtpRouteBuilder extends RouteBuilder {

    @Override
    public void configure() throws Exception {
        from("file://localDirectory").to("ftp://user@host.com/remoteDirectory?password=secret&passiveMode=true");
    }
}

So what is going on here? The RouteBuilder is one of Camel's core classes that defines the Camel specific DSL. In this example, a Route from the localDirectory to a remoteDirectory over FTP with the provided credentials has been created. As can be seen, it is easy to create and configure different endpoints using String URIs. It should also be mentioned that it is possible to fetch files from the remote FTP server by swapping the URIs in the to and from methods.

A few more lines are needed to activate the route:

public static void main(String[]args) throws Exception {
    CamelContext camelContext = new DefaultCamelContext();
    try {
        camelContext.addRoutes(new FtpRouteBuilder());
        camelContext.start();
        // do other stuff...
    }
    finally {
        camelContext.stop();
    }
}

A CamelContext is created for managing the route. The previously described FtpRouteBuilder is instantiated and added to the camelContext which is subsequently started. As a consequence, any file that is placed in the "localDirectory" folder will be automatically transferred to the "remoteDirectory" of the FTP server while the program is running.

Adding Configurability

The requirement of supporting different FTP protocols is solved by changing the URI from ftp:// to ftps:// or sftp:// respectively. It is just as easy to fulfill the second requirement of multiple server support by changing the host, user, port and other parameters of the URI. Likewise, additional settings may be configured by adding more options if needed, see the Camel FTP documentation.

Adding Dynamism

The last challenge was to add the dynamic support of selecting source files and destination servers during runtime. The easiest solution is to use another of Camel's base classes, the ProducerTemplate. Again, the solution is a one-liner:

producerTemplate.sendBodyAndHeader("ftp://user@host.com/remoteDirectory?password=secret", file, Exchange.FILE_NAME, file.getName());

The sendBodyAndHeader() method name reveals a glimpse of Camel's underlying messaging architecture. The parameters used are the destination URI with additional options, the message body, i.e. the File to be sent, a message header parameter that identifies the last parameter as the name of the file it will have on the remote server once it has been transferred.

Spring Integration

With some refactoring and a little Spring magic we have all the bits and pieces needed to implement the previously defined FtpSender interface:

FtpSenderImpl.java

@Service
class FtpSenderImpl implements FtpSender {

    /** Camel URI, format is ftp://user@host/fileName?password=secret&passiveMode=true */
    private static final String CAMEL_FTP_PATTERN = "{0}://{1}@{2}/{3}?password={4}&passiveMode={5}";

    private final ProducerTemplate producerTemplate;

    /**
      * Constructor
      * @param producerTemplate The producer template to be be used
      */
    @Autowired
    FtpSenderImpl(ProducerTemplate producerTemplate) {
        this.producerTemplate = producerTemplate;
    }

    @Override
    public void sendFile(FtpProperties ftpProperties, File file) throws RuntimeException {
        producerTemplate.sendBodyAndHeader(createFtpUri(ftpProperties), file, Exchange.FILE_NAME, fileName);
    }

    /**
      * Creates a Camel FTP URI based on the provided FTP properties
      * @param ftpProperties The properties to be used
      */
    private String createFtpUri(FtpProperties ftpProperties) {
        return MessageFormat.format(CAMEL_FTP_PATTERN,
                ftpProperties.getProtocol(),
                ftpProperties.getUserName(),
                ftpProperties.getHost(),
                ftpProperties.getRemoteDirectory(),
                ftpProperties.getPassword(),
                ftpProperties.getPassiveMode());
    }
}

Using the Camel namespace, the required plumbing work is delegated to the Spring application config:

spring-config.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:p="http://www.springframework.org/schema/p"
       xmlns:camel="http://camel.apache.org/schema/spring"
       xsi:schemaLocation="
		http://www.springframework.org/schema/beans   http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
		http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
		http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">

    <!-- Let Spring create the Camel context and the Camel template, including lifecycle management such as starting and stopping them -->
    <camel:camelContext id="camelContext">
        <camel:template id="camelTemplate" />
    </camel:camelContext>

    <!-- Use Spring component scan to find the FtpSenderImpl implementation -->
    <context:component-scan base-package="com.jayway.ftp" />

</beans>

Conclusions

Camel and Spring provide a simple, yet configurable, way of implementing a dynamic FTP client.

Acknowledgments

Thanks to Claus Ibsen and other members of the Apache Camel Forums for providing valuable and rapid support.

Glossary

• FTP File Transfer Protocol
• FTPS FTP Secure is an extension to FTP that adds support for the TLS, Transport Layer Security, and the SSL, Secure Sockets Layer, cryptographic protocols
• SFTP SSH File Transfer Protocol, i.e. FTP over the Secure Shell protocol

References

Camel API
Camel Components
Camel FTP
Camel ProducerTemplate
Camel Spring
Camel ProducerTemplate and Spring
Camel Enterprise Integration Patterns
Camel Forums

I've used Commons HttpClient in older versions on several occasions in the past and have found it to be extremely useful. I've only just started working with the new version, but there seems to quite a lot of good stuff in there. For example I'm thrilled to see that the Template pattern, heavily used in Spring, has now made its way to the Apache world (e.g. ResponseHandler, a callback interface to be used with HttpClient for handling results).

Now, configuration of HttpClient has always been a little bit tricky. There are so many features in there, and most of these are configured using generic parameters (HttpParams), basically name-value pairs of a setting identifier and its value. This has not been made that much easier in the new version.

In my current project I was struggling to find the appropriate parameters for configuring timeouts with the new version of the framework. It turned out to be quite difficult to find clear information on this - the documentation on the 4.0 version of HttpClient is still quite sparse. Since it took me a while to find out how to do this I figured I wasn't alone, so here goes:

My original solution for this problem - being completely unable to find the proper way to do it - was to use the hard-coded property values and set them as parameters in HttpParams. Fortunately, a reader with signature 'BoD' (see below) knew the right way to do this and was kind enough to share.

Right, the code looks as follows:

 
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(httpParams, connectionTimeoutMillis);
HttpConnectionParams.setSoTimeout(httpParams, socketTimeoutMillis);
 

Simple as that - hard to understand why that should take me so long to figure out . Thanks again to 'BoD' for setting me straight on this.