Prerequisites

In order to keep this reasonably brief I'll have to refer to the getting started guide for information on how to get going with Amazon EC2.

A FactoryBean to Launch EC2 Instances

What we want to achieve here is to launch an EC2 instance transparently and independently of the actual test code. Ideally, the test code should be oblivious of the target server and we want to externalize those details to external configuration. We will use Spring's excellent JUnit test support to automatically wire the integration test setup and make sure that the target server is up and running before the actual test code is running.

A powerful way to transparently perform complex initialization logic when using Spring is the FactoryBean concept. We will create a FactoryBean implementation to launch the EC2 image and set up our target resource:

AbstractEc2InstanceLaunchingFactoryBean.java
public abstract class AbstractEc2InstanceLaunchingFactoryBean extends AbstractFactoryBean {
  private static final int INSTANCE_START_SLEEP_TIME = 1000;
  private static final long DEFAULT_PREPARATION_SLEEP_TIME = 30000;
  private static final Log log = LogFactory.getLog(AbstractEc2InstanceLaunchingFactoryBean.class);
  private String imageName;
  private String awsKey;
  private String awsSecretKey;
  private String keypairName;
  private String groupName;
  private Instance instance;
  private long preparationSleepTime = DEFAULT_PREPARATION_SLEEP_TIME;
  public void setImageName(String imageName) {
      this.imageName = imageName;
  }
  public void setAwsKey(String awsKey) {
      this.awsKey = awsKey;
  }
  public void setAwsSecretKey(String awsSecretKey) {
      this.awsSecretKey = awsSecretKey;
  }
  public void setKeypairName(String keypairName) {
      this.keypairName = keypairName;
  }
  public void setGroupName(String groupName) {
      this.groupName = groupName;
  }
  public void setPreparationSleepTime(long preparationSleepTime) {
    this.preparationSleepTime = preparationSleepTime;
  }
  @Override
  protected final Object createInstance() throws Exception {
      Assert.hasLength(imageName, "ImageName must be set");
      Assert.hasLength(awsKey, "AwsKey must be set");
      Assert.hasLength(awsSecretKey, "AwsSecretKey must be set");
      Assert.hasLength(keypairName, "KeyName must be set");
      Assert.hasLength(groupName, "GroupName must be set");
 
      log.info("Launching EC2 instance for image: " + imageName);
 
      Jec2 jec2 = new Jec2(awsKey, awsSecretKey);
      LaunchConfiguration launchConfiguration = new LaunchConfiguration(imageName);
      launchConfiguration.setKeyName(keypairName);
      launchConfiguration.setSecurityGroup(Collections.singletonList(groupName));
 
      ReservationDescription reservationDescription = jec2.runInstances(launchConfiguration);
      instance = reservationDescription.getInstances().get(0);
      while (!instance.isRunning() && !instance.isTerminated()) {
          log.info("Instance still starting up; sleeping " + INSTANCE_START_SLEEP_TIME + "ms");
          Thread.sleep(INSTANCE_START_SLEEP_TIME);
          reservationDescription = jec2.describeInstances(Collections.singletonList(instance.getInstanceId())).get(0);
          instance = reservationDescription.getInstances().get(0);
      }
 
      if (instance.isRunning()) {
          log.info("EC2 instance is now running");
          if (preparationSleepTime > 0) {
              log.info("Sleeping " + preparationSleepTime + "ms allowing instance services to start up properly.");
              Thread.sleep(preparationSleepTime);
              log.info("Instance prepared - proceeding");
          }
          return doCreateInstance(instance.getDnsName());
      } else {
          throw new IllegalStateException("Failed to start a new instance");
      }
   }
 
  protected abstract Object doCreateInstance(String ip) throws Exception;
 
  @Override
  protected void destroyInstance(Object ignored) throws Exception {
    if (this.instance != null) {
      log.info("Shutting down instance");
      Jec2 jec2 = new Jec2(awsKey, awsSecretKey);
      jec2.terminateInstances(Collections.singletonList(this.instance.getInstanceId()));
    }
  }
}
 

This superclass makes use of the typica library (also available in maven) to launch the EC2 instance and delegates to doCreateInstance for creating the target resource that we will use in our test case. In our setup we want to create a ContextSource, which is the Spring LDAP equivalent of a DataSource:

ContextSourceEc2InstanceLaunchingFactoryBean.java
public class ContextSourceEc2InstanceLaunchingFactoryBean extends AbstractEc2InstanceLaunchingFactoryBean {
  private String base;
  private String userDn;
  private String password;
  @Override
  public final Class getObjectType() {
    return ContextSource.class;
  }
  @Override
  protected final Object doCreateInstance(final String dnsName) throws Exception {
    Assert.hasText(userDn);
    LdapContextSource instance = new LdapContextSource();
    instance.setUrl("ldap://" + dnsName);
    instance.setUserDn(userDn);
    instance.setPassword(password);
    instance.setBase(base);
 
    instance.afterPropertiesSet();
    return instance;
  }
  public void setBase(String base) {
    this.base = base;
  }
  public void setUserDn(String userDn) {
    this.userDn = userDn;
  }
  public void setPassword(String password) {
    this.password = password;
  }
}
 

Note that the AbstractEc2InstanceLaunchingFactoryBean will wait for the instance to start up properly. It will also wait an additional period of time once the instance is up and running to allow for all relevant services to start up properly. Finally it will automatically close down the launched instance once it is properly shut down (which it will be when executing using the Spring automated test support).

Test Case Configuration

We now have the infrastructure to have the EC2 instance launched transparently from a Spring Application Context. All we need to do is configure it:

testContext.xml
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="ldap.properties" />
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
</bean>
 
<bean id="contextSource" class="org.springframework.ldap.ContextSourceEc2InstanceLaunchingFactoryBean">
<property name="awsKey" value="${aws.key}" />
<property name="awsSecretKey" value="${aws.secret.key}" />
<property name="imageName" value="${aws.ami}" />
<property name="groupName" value="${aws.security.group}" />
<property name="keypairName" value="${aws.keypair}" />
<property name="base" value="dc=jayway,dc=se" />
<property name="userDn" value="${userDn}" />
<property name="password" value="${password}" />
</bean>
 
<bean id="ldapTemplate"
  class="org.springframework.ldap.core.LdapTemplate">
  <constructor-arg ref="contextSource" />
</bean>
 

We are referring to ldap.properties for the actual settings:

userDn=cn=admin,dc=jayway,dc=se
password=secret
base=dc=jayway,dc=se
aws.ami=ami-56ec083f
aws.keypair=spring-ldap-keypair
aws.security.group=spring-ldap

The referenced AMI is a public image specifically set up for our test cases. It has OpenLDAP installed and has been pre-populated with the test data that our test cases expect.

Note that we are not specifying any property value for aws.key and aws.secret.key, as this is the Amazon account access information. That will be the account settings of the individual developer and should be supplied at runtime using system properties (e.g. mvn -Daws.key=xxxxxxx -Daws.secret.key=xxxxxx test).

Performing the Test

Now all we need to do is use Spring's automated test support to start our test:

SomeLdapITest.java
@ContextConfiguration(locations = { "testContext.xml" })
public class SomeLdapITest extends AbstractJUnit4SpringContextTests {
  @Autowired
  private LdapTemplate tested;
 
  @Test
  public void testSomething() {
    // Use the automatically injected LdapTemplate instance to perform a test.
  }
}
 

Set to Go

With the above you should be all set to go to start using this powerful approach to integration testing. The actual code is available with the 1.3.0.RC1 release of Spring LDAP; links to downloads and documentation available here.

In the perfect world, each developer would at all times have access to a server park with all of these different server versions installed. Reality is however almost always a different question: Being an Open Source project with limited funding and developers located throughout the world this has been impossible for us, up until now. You're probably already guessing: The answer is in the Clouds.

Having used Amazon EC2 in a couple of customer projects, we suddenly realized this is the answer to our problems. For those of you not completely familiar with the concept, the general idea of Amazon EC2 is that you configure one or several machine images, installing whatever software and data you might need on each image. You will then start an instance from one of these pre-configured images, which will in effect start up a virtual computer somewhere in the amazon cloud, a computer that will then be yours to play around with in any way you might like until you're finished with it and just shut the instance down. All of this is available at a very reasonable pricing - you pay only (a very modest amount) for the time that your instances have been running.

The concept suits us perfectly: we want a number of isolated environments, each with a known start state where we can run our test suite against a particular server setup. So that's what we've started to do: we have now created an EC2 image with an Open LDAP installation, pre-loaded with the data that our test cases expect. We then devised support classes to have the JUnit test cases automatically start an appropriate instance (using the excellent typica library), run the tests against the started virtual machines, and shut down the instance after the tests are finished (regardless of the outcome - we don't want any stray instances running around costing us money ). We still have some more work to do with setting up more images for other server setups, but we're hoping to get that work going as we move on...

I'm planning to dig a little bit deeper into the details of the setup and the code involved in a later post. In the meantime: if you're really interested, feel free to check the code out from the svn repo and have a look at what we've done so far. The relevant code for this is in the test/integration-tests-openldap directory.