First step is to boot an instance. You need the command line tools for this, as neither the AWS Management Web Console nor ElasticFox currently supports this right now (note to developers - please add!).

ec2-run-instances --region eu-west-1 \
     -K pk-YOURKEYHERE.pem \
     -C cert-YOURCERTHERE.pem \
     -b '/dev/sda1=:100' \
     ami-d7a78ca3 \
     -g my-security-group -g common-security-group \
     -t m1.large \
     -k thekeyhere \
     --disable-api-termination \
     --instance-initiated-shutdown-behavior stop \
     -z eu-west-1a

• The command is ec2-run-instances with an 's' in the end. There is another command in the API without the s, which doesn't work.
• -b is the most important flag here. It says to make the boot disk (/dev/sda is Linux speek for C:) 100 GB, despite what the AMI manifest says (which is 30 GB).
• ami-d7a78ca3 is the Windows image, in this case a 64bit Windows 2008.
• -g security-group you should add as many as you like. Remember to think these through, since you cannot change which groups an instance belongs to.
• The flags --disable-api-termination --instance-initiated-shutdown-behavior stop are really nice, because that make sure you cannot accidentally terminate the instance, making the harddrive disappear.
• Note that I'm running in the EU region here.

Ok, so now you have a running Windows server. Once it has booted, you log on using remote desktop (well, as soon as you have the generated password, which takes 15 min). PLease note that the C: drive will show up as 30 GB at this point. The partition is still only 30 GB, even if the virtual harddrive is 100 GB.

1. Go into the windows management console. Go to disk management.
2. Right-click on the C: drive. Choose Extend volume from the context menu.
3. In the wizard that pops up, just accept the defaults, and click finish.

Done! The change is instant (no reboot or anything), and your system drive C: is now 100 GB.

That said, I must also give credit to DreamHost for always being very helpful and supporting when needed.

If you want to know what we did to gain the speed needed for a lift-of, keep on reading.

Exporting the Existing Database

I really like MySQL, but when it comes to encoding for character sets and collations there seems to be such a complexity in the details that users often get lost and configure it the wrong way. The UTF-8 configured database we used on DreamHost worked just fine together with our WordPress installation but it turned out to be impossible to export a correct UTF-8 version for importing into our new environment. And I tried a lot of things. Really. I'm not going to go into details but to sum it up I had to sit down and do kind of manual replacements of the swedish specific characters in the database export. Lucky me this blog is in english.

Anyway, I had ssh access to the Trafficante server and could use this connection to export the database from the DreamHost MySQL server by issueing:

 
mysqldump --default-character-set=UTF8 -u [user] -p -h [host].dreamhosters.com [database] -r [output].sql
 

This resulted in a database dump later used for importing into the new MySQL instance.

Setting up the Amazon EC2 Instance

To make your Amazon life easier start with installing the Firefox plugins Elasticfox and S3 Organizer. These tools give you a GUI view of the somewhat more complex underlying details of Amazon.

Amazons' cloud solution is based on a user account. There are no start-up fees, you only pay for what you use. So if you still haven't got an account, sign-up now. The rest of this article assumes you have signed up and set up accounts for Amazon EC2, Amazon S3 and Amazon EBS. Start by reading the EC2 Getting Started Guide and don't stop reading until you have created an ssh key pair, a security group and have a server instance running.

Now you should also have a private key and a certificate in your ${EC2_HOME} (preferrably ${HOME}/.ec2/) folder of your local computer. These two files should also be uploaded to your instance /mnt folder to make it possible to bundle and upload your images to your S3 "bucket(s)" (buckets are one level deep folders/directories). Make sure you put them in the /mnt folder to prevent them from being bundled with the image and thereby increasing the risk of having your identity revealed.

Bundling, by the way, should be done when you've made changes to your server instance that you don't want to loose. The result of a bundling and uploading is an image with a specific Amazon Image Id (AMI ID) reflecting the state of your server instance at the moment of bundling. You also have to register the image with the manifest file created for you. If you do that you could then safely close down your instance and have it up and running again by issuing the ec2-run-instances command with the AMI ID as parameter. A way to save money.

Bundling and uploading was made easier for me as I used two scripts created by my colleague Ulrik. I put these scripts in the /root folder.

bundle-myself.sh:
 
#!/bin/sh
prg=`basename $0`
dest=/mnt
image=blog-image
 
USAGE="Usage: $prg <datetime>
 
Example:
% $prg 20090415-1000
Bundling into $dest/$image-20090415-1000.manifest.xml"
 
if [ $# -ne 1 ]; then
  echo "$USAGE"
  exit 1
fi
date=$1
echo "Bundling into $dest/$image-$date.manifest.xml"
ec2-bundle-vol --debug -k /mnt/pk-PRIVATE_KEY.pem -c /mnt/cert-CERTIFICATE.pem -u USER_ACCOUNT_ID -d $dest -p $image-$dat
e -r i386
 

upload-myself.sh:
 
#!/bin/sh
prg=`basename $0`
dest=/mnt
image=blog-image
 
USAGE="Usage: $prg <date>
 
Example:
% $prg 20090415-1000
Uploading bundled AMI parts to https://s3.amazonaws.com:443/[S3_BUCKET] ...
Uploaded blog-image-20090415-1000.part.00 to https://s3.amazonaws.com:443/[S3_BUCKET]/blog-image-20090415-1000.part.00.
..."
 
if [ $# -ne 1 ]; then
  echo "$USAGE"
  exit 1
fi
date=$1
ec2-upload-bundle --retry -m $dest/$image-$date.manifest.xml -b blog-images -a [PUBLIC_KEY] -s [PRIVATE_KEY]
 

Another thing I needed was a static IP address for my instance. To solve this I used Amazons Elastic IP service which doesn't cost anything as long as the generated IP address is in use. So, I generated an Elastic IP address and assigned it to the instance.

Installing Necessary Software

The blog require Apache2, Php5, MySQL and the Php GD graphics package, to start with. This was easily installed using Advanced Packaging Tool, apt-get:

 
# sudo apt-get update
# sudo apt-get install apache2 php5 mysql-server php5-mysql php5-gd
 

Setting up the Database for EBS

EBS volumes are, as you might know, more persistent than EC2 instances and will survive if instances go down. Volumes are attached to the instance and could as well be detached and re-attached to new instances. As EBS volumes are not 100% reliable you should also take snapshots of your database data and have it uploaded to an S3 bucket. All this is nicely described in Eric Hammonds' EBS and MySQL tutorial, so I urge you to read it if you haven't yet.

The only thing I sidestepped in the tutorial was the MySQL configuration to point out the data and log directories. Instead I used symbolic links:

 
# ln -s /vol/lib/mysql/ /var/lib/mysql
# ln -s /vol/log/mysql/ /var/log/mysql
 

The tutorial also walks you through automatic snapshots which requires the EC2 command line tools. Here is a nice tutorial on setting these tools up on Ubuntu.

I had to tweak the environment variable settings in ec2-snapshot-xfs-mysql.pl script from

BEGIN {
  $ENV{PATH} = '/bin:/usr/bin:/usr/local/bin';
}

to

BEGIN {
  $ENV{PATH} = '/bin:/usr/bin:/usr/local/bin:/usr/sbin:/root/ec2-api-tools/bin';
  $ENV{EC2_CERT} = '/mnt/cert-CERTIFICATE.pem';
  $ENV{EC2_PRIVATE_KEY} = '/mnt/pk-PRIVATE_KEY.pem';
  $ENV{EC2_HOME} = '/root/ec2-api-tools';
  $ENV{HOME} = '/root';
  $ENV{JAVA_HOME} = '/usr/lib/jvm/java-1.5.0-sun';
}

to get the snapshotting to fully work. I also wrapped the ec2-snapshot-xfs-mysql.pl script for a more convenient way to handle the input:

/root/snapshot-myself.sh

#!/bin/sh
mountpoint=/vol
volume_id=[VOLUME_ID]

su -c "/root/ec2-snapshot-xfs-mysql.pl ${mountpoint} ${volume_id} > /vol/cron.log"

To automatize the snapshotting I set up a simple crontab job by issueing:

 
# crontab -e
 

and then edited it like this

05 */6 * * * sh /root/snapshot-myself.sh

which makes the job run five minutes past every six hours, every day.

I also had to set the timezone info to apply to our local time. The easiest way I've found out of setting this for a Linux server is like this:

 
# ln -sf /usr/share/zoneinfo/Europe/Stockholm /etc/localtime
 

Configuring MySQL

To configure MySQL for UTF-8 content make sure the /etc/mysql/my.cnf file contains the following:

[client]
default-character-set=utf8

[mysqld]
#
# * Basic Settings
#
init_connect='SET collation_connection = utf8_general_ci'
init_connect='SET NAMES utf8'
character-set-server=utf8
collation-server=utf8_general_ci
default-character-set=utf8
skip-character-set-client-handshake

Now, if you haven't yet, go on and start your MySQL server. You need to create your wordpress database. It's as simple as:

 
# mysql -u [user] -p
> CREATE DATABASE [database_name]
 

To create the user with which WordPress will access the database:

 
> GRANT ALL ON [database_name].* TO '[user]'@'localhost' IDENTIFIED BY '[password]';
 

And to import your database dump into the newly created database:

 
# mysql -u [user] -p [database_name] < [output].sql
 

You shouldn't need to specify any character set or collation when importing thanks to the configuration done in /etc/mysql/my.cnf.

In order for us to stick with our domain name and have no downtime we needed to use the instances IP address until we were finished testing. This means I had to replace two properties in the database to prevent the blog from redirecting to the original host; the 'siteurl' and 'blogurl' properties residing in the option_value column of the wp_options table. The commands

 
# mysql -u [user] -p
> UPDATE wp_options SET option_value='http://[ip_address]/wordpress/' WHERE option_name='siteurl'
> UPDATE wp_options SET option_value='http://[ip_address]/wordpress/' WHERE option_name='blogurl'
 

fixed this. As soon as the DNS info was propagated throughout the Internet this was reset to "http://blog.jayway.com/wordpress" from within the WordPress admin interface.

Apache Configuration

Our WordPress installation makes use of permalinks and redirects to FeedBurner for RSS statistics. This is based on .htaccess redirects (at least that's how we've solved it) which in turn is dependent on the Apache redirect module. So, in order to make this to work I had to enable that.

 
# sudo a2enmod rewrite
 

Easy as that! One more thing on redirections. I had to configure Apache to allow redirection directives specified in the .htaccess file by changing the /etc/apache2/sites-available/default configuration of the element from

 
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>
 

to

 
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride All <----------
		Order allow,deny
		allow from all
	</Directory>
 

Securing the WordPress Admin Interface

Again, I choose to refer to detailed and well written sources. Here are the two articles I followed to create a self-signed certificate and to configure Apache to use it.

• Creating a self-signed certificate
• Configuring Apache for SSL on Ubuntu

To enable https access from the "outside world" I also edited the group policy to open up port 443. This could easily be done via the Elasticfox Firefox plugin mentioned earlier.

Installing and Configuring WordPress

The default location for Apache httpd server for reading content is /var/www/. If you, as I did, have an existing WordPress installation you can simply copy this to the new server. I ended up having it located at /var/www/wordpress/ on the Amazon instance.

Once the database connection properties in the wp-config.php wordpress configuration file was adapted to the new environment I was able to access the WordPress installation via a browser and the Amazon instances' IP address. Sweet!

And as we have our WordPress installation in a separate "/wordpress" folder instead of in the document root of Apache we need to specify, in WordPress admin settings, that the 'Site URL' should be "${host}/wordpress" but the 'Blog URL' should simply say "${host}" to prevent the resulting URL from containing "/wordpress/...". This also requires that we move the .htaccess and index.php files from within our "/wordpress" folder and up one level to "/var/www/", in our case.

In the index.php file, make sure to change from

 
/** Loads the WordPress Environment and Template */
require('./wp-blog-header.php');
 

to

 
/** Loads the WordPress Environment and Template */
require('./wordpress/wp-blog-header.php');
 

If or when specifying the permalinks, WordPress must have write access to the .htaccess file, so 'chmod' that file.

 
# chmod 755 .htaccess
 

Otherwise it will complain and simply print out the .htaccess modification it tried to modify with and you are left to change it manually, instead.

Sending Mail from WordPress

Mailing is used in WordPress when generating new passwords to users, among others. We also use it to send data backups to the administator of the blog.

Setting up a mail server isn't a trivial task. I gave it a thought but quickly put it aside in favour of the simple WordPress plugin WP Mail SMTP as I realized I could make use of our existing corporate mail server. It was a breeze to configure the plugin with the necessary mail user credentials.

So, here we are. Sailing these fluffy clouds.

Prerequisites

In order to keep this reasonably brief I'll have to refer to the getting started guide for information on how to get going with Amazon EC2.

A FactoryBean to Launch EC2 Instances

What we want to achieve here is to launch an EC2 instance transparently and independently of the actual test code. Ideally, the test code should be oblivious of the target server and we want to externalize those details to external configuration. We will use Spring's excellent JUnit test support to automatically wire the integration test setup and make sure that the target server is up and running before the actual test code is running.

A powerful way to transparently perform complex initialization logic when using Spring is the FactoryBean concept. We will create a FactoryBean implementation to launch the EC2 image and set up our target resource:

AbstractEc2InstanceLaunchingFactoryBean.java
public abstract class AbstractEc2InstanceLaunchingFactoryBean extends AbstractFactoryBean {
  private static final int INSTANCE_START_SLEEP_TIME = 1000;
  private static final long DEFAULT_PREPARATION_SLEEP_TIME = 30000;
  private static final Log log = LogFactory.getLog(AbstractEc2InstanceLaunchingFactoryBean.class);
  private String imageName;
  private String awsKey;
  private String awsSecretKey;
  private String keypairName;
  private String groupName;
  private Instance instance;
  private long preparationSleepTime = DEFAULT_PREPARATION_SLEEP_TIME;
  public void setImageName(String imageName) {
      this.imageName = imageName;
  }
  public void setAwsKey(String awsKey) {
      this.awsKey = awsKey;
  }
  public void setAwsSecretKey(String awsSecretKey) {
      this.awsSecretKey = awsSecretKey;
  }
  public void setKeypairName(String keypairName) {
      this.keypairName = keypairName;
  }
  public void setGroupName(String groupName) {
      this.groupName = groupName;
  }
  public void setPreparationSleepTime(long preparationSleepTime) {
    this.preparationSleepTime = preparationSleepTime;
  }
  @Override
  protected final Object createInstance() throws Exception {
      Assert.hasLength(imageName, "ImageName must be set");
      Assert.hasLength(awsKey, "AwsKey must be set");
      Assert.hasLength(awsSecretKey, "AwsSecretKey must be set");
      Assert.hasLength(keypairName, "KeyName must be set");
      Assert.hasLength(groupName, "GroupName must be set");
 
      log.info("Launching EC2 instance for image: " + imageName);
 
      Jec2 jec2 = new Jec2(awsKey, awsSecretKey);
      LaunchConfiguration launchConfiguration = new LaunchConfiguration(imageName);
      launchConfiguration.setKeyName(keypairName);
      launchConfiguration.setSecurityGroup(Collections.singletonList(groupName));
 
      ReservationDescription reservationDescription = jec2.runInstances(launchConfiguration);
      instance = reservationDescription.getInstances().get(0);
      while (!instance.isRunning() && !instance.isTerminated()) {
          log.info("Instance still starting up; sleeping " + INSTANCE_START_SLEEP_TIME + "ms");
          Thread.sleep(INSTANCE_START_SLEEP_TIME);
          reservationDescription = jec2.describeInstances(Collections.singletonList(instance.getInstanceId())).get(0);
          instance = reservationDescription.getInstances().get(0);
      }
 
      if (instance.isRunning()) {
          log.info("EC2 instance is now running");
          if (preparationSleepTime > 0) {
              log.info("Sleeping " + preparationSleepTime + "ms allowing instance services to start up properly.");
              Thread.sleep(preparationSleepTime);
              log.info("Instance prepared - proceeding");
          }
          return doCreateInstance(instance.getDnsName());
      } else {
          throw new IllegalStateException("Failed to start a new instance");
      }
   }
 
  protected abstract Object doCreateInstance(String ip) throws Exception;
 
  @Override
  protected void destroyInstance(Object ignored) throws Exception {
    if (this.instance != null) {
      log.info("Shutting down instance");
      Jec2 jec2 = new Jec2(awsKey, awsSecretKey);
      jec2.terminateInstances(Collections.singletonList(this.instance.getInstanceId()));
    }
  }
}
 

This superclass makes use of the typica library (also available in maven) to launch the EC2 instance and delegates to doCreateInstance for creating the target resource that we will use in our test case. In our setup we want to create a ContextSource, which is the Spring LDAP equivalent of a DataSource:

ContextSourceEc2InstanceLaunchingFactoryBean.java
public class ContextSourceEc2InstanceLaunchingFactoryBean extends AbstractEc2InstanceLaunchingFactoryBean {
  private String base;
  private String userDn;
  private String password;
  @Override
  public final Class getObjectType() {
    return ContextSource.class;
  }
  @Override
  protected final Object doCreateInstance(final String dnsName) throws Exception {
    Assert.hasText(userDn);
    LdapContextSource instance = new LdapContextSource();
    instance.setUrl("ldap://" + dnsName);
    instance.setUserDn(userDn);
    instance.setPassword(password);
    instance.setBase(base);
 
    instance.afterPropertiesSet();
    return instance;
  }
  public void setBase(String base) {
    this.base = base;
  }
  public void setUserDn(String userDn) {
    this.userDn = userDn;
  }
  public void setPassword(String password) {
    this.password = password;
  }
}
 

Note that the AbstractEc2InstanceLaunchingFactoryBean will wait for the instance to start up properly. It will also wait an additional period of time once the instance is up and running to allow for all relevant services to start up properly. Finally it will automatically close down the launched instance once it is properly shut down (which it will be when executing using the Spring automated test support).

Test Case Configuration

We now have the infrastructure to have the EC2 instance launched transparently from a Spring Application Context. All we need to do is configure it:

testContext.xml
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="ldap.properties" />
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
</bean>
 
<bean id="contextSource" class="org.springframework.ldap.ContextSourceEc2InstanceLaunchingFactoryBean">
<property name="awsKey" value="${aws.key}" />
<property name="awsSecretKey" value="${aws.secret.key}" />
<property name="imageName" value="${aws.ami}" />
<property name="groupName" value="${aws.security.group}" />
<property name="keypairName" value="${aws.keypair}" />
<property name="base" value="dc=jayway,dc=se" />
<property name="userDn" value="${userDn}" />
<property name="password" value="${password}" />
</bean>
 
<bean id="ldapTemplate"
  class="org.springframework.ldap.core.LdapTemplate">
  <constructor-arg ref="contextSource" />
</bean>
 

We are referring to ldap.properties for the actual settings:

userDn=cn=admin,dc=jayway,dc=se
password=secret
base=dc=jayway,dc=se
aws.ami=ami-56ec083f
aws.keypair=spring-ldap-keypair
aws.security.group=spring-ldap

The referenced AMI is a public image specifically set up for our test cases. It has OpenLDAP installed and has been pre-populated with the test data that our test cases expect.

Note that we are not specifying any property value for aws.key and aws.secret.key, as this is the Amazon account access information. That will be the account settings of the individual developer and should be supplied at runtime using system properties (e.g. mvn -Daws.key=xxxxxxx -Daws.secret.key=xxxxxx test).

Performing the Test

Now all we need to do is use Spring's automated test support to start our test:

SomeLdapITest.java
@ContextConfiguration(locations = { "testContext.xml" })
public class SomeLdapITest extends AbstractJUnit4SpringContextTests {
  @Autowired
  private LdapTemplate tested;
 
  @Test
  public void testSomething() {
    // Use the automatically injected LdapTemplate instance to perform a test.
  }
}
 

Set to Go

With the above you should be all set to go to start using this powerful approach to integration testing. The actual code is available with the 1.3.0.RC1 release of Spring LDAP; links to downloads and documentation available here.

In the perfect world, each developer would at all times have access to a server park with all of these different server versions installed. Reality is however almost always a different question: Being an Open Source project with limited funding and developers located throughout the world this has been impossible for us, up until now. You're probably already guessing: The answer is in the Clouds.

Having used Amazon EC2 in a couple of customer projects, we suddenly realized this is the answer to our problems. For those of you not completely familiar with the concept, the general idea of Amazon EC2 is that you configure one or several machine images, installing whatever software and data you might need on each image. You will then start an instance from one of these pre-configured images, which will in effect start up a virtual computer somewhere in the amazon cloud, a computer that will then be yours to play around with in any way you might like until you're finished with it and just shut the instance down. All of this is available at a very reasonable pricing - you pay only (a very modest amount) for the time that your instances have been running.

The concept suits us perfectly: we want a number of isolated environments, each with a known start state where we can run our test suite against a particular server setup. So that's what we've started to do: we have now created an EC2 image with an Open LDAP installation, pre-loaded with the data that our test cases expect. We then devised support classes to have the JUnit test cases automatically start an appropriate instance (using the excellent typica library), run the tests against the started virtual machines, and shut down the instance after the tests are finished (regardless of the outcome - we don't want any stray instances running around costing us money ). We still have some more work to do with setting up more images for other server setups, but we're hoping to get that work going as we move on...

I'm planning to dig a little bit deeper into the details of the setup and the code involved in a later post. In the meantime: if you're really interested, feel free to check the code out from the svn repo and have a look at what we've done so far. The relevant code for this is in the test/integration-tests-openldap directory.