How to backup WordPress

There are numerous plugins for WordPress when it comes to backup. I have learnt to appreciate the plugin BackUpWordPress as it not only backs up your database but also all the files within the installation.

http://hmn.md/backupwordpress/

There are a few number (though quite complete) of configuration options like

- manual or daily scheduled backups
- number of backups to keep
- whether to include files or database or both

The backups are stored on disk on the path of your choice and could also be attempted to be emailed. Having this kind of backup data makes it very comfortable to restore your WordPress installation.

But what if your complete machine/instance is failing? On, for example Amazon EC2 one should take snapshots of instances when the setup of these change but this does not apply to content changes. Restoring from a snapshot does not bring back newly created or updated content from, for example WordPress. That is, moving those WordPress backup files to a safer place becomes vital. So, s3cmd to the rescue.

What is s3cmd?

Extracted from its' homepage:

"S3cmd is a command line tool for uploading, retrieving and managing data in Amazon S3. It is best suited for power users who don't fear command line. It is also ideal for scripts, automated backups triggered from cron, etc."

http://s3tools.org/s3cmd

At the time of writing s3cmd requires Python 2.4 or newer and some pretty common Python modules. Installing it is merely a matter of running:

sudo apt-get install s3cmd

To configure it run

s3cmd --configure

You will be asked for the two keys of your Amazon account - copy and paste them from your confirmation email or from your Amazon account page.

To list all your buckets run

s3cmd ls

and to list the content of a bucket run

s3cmd ls s3://[bucket name]

Now, you could use 's3cmd put' to upload and 's3cmd remove' to remotely remove specific files but then you would have to do some scripting to select these specific files and to keep the bucket contents to an acceptable size. There is another option that simply syncs the content of your local directory to the remote bucket, doing the removal automatically, namely 's3cmd sync'.

s3cmd sync --delete-removed [source directory] s3://[bucket name]

The source directory is naturally replaced with the directory of where you put your WordPress backups.

Scheduling it

To make it a little easier to schedule via crontab you preferrably put the sync command in a shell script, for example 'sync-wp-backups.sh'

#!/bin/bash

/usr/bin/s3cmd sync --delete-removed [source directory] s3://[bucket name]

To schedule this script run

crontab -e

which brings you into editing the cron job with the editor of your choice. For example you could edit the crontab file to schedule it to run daily at 23:30 (11:30 PM)

30 23 * * * /home/ubuntu/sync-wp-backups.sh >> /home/ubuntu/s3-backup.log

which logs the output to file (s3-backup.log).

That's it! Now I have a number of days of backup automatically synced from the Amazon instance running WordPress to a more fail safe S3 bucket. At least I sleep a lot better with this in place.

That said, I must also give credit to DreamHost for always being very helpful and supporting when needed.

If you want to know what we did to gain the speed needed for a lift-of, keep on reading.

Exporting the Existing Database

I really like MySQL, but when it comes to encoding for character sets and collations there seems to be such a complexity in the details that users often get lost and configure it the wrong way. The UTF-8 configured database we used on DreamHost worked just fine together with our WordPress installation but it turned out to be impossible to export a correct UTF-8 version for importing into our new environment. And I tried a lot of things. Really. I'm not going to go into details but to sum it up I had to sit down and do kind of manual replacements of the swedish specific characters in the database export. Lucky me this blog is in english.

Anyway, I had ssh access to the Trafficante server and could use this connection to export the database from the DreamHost MySQL server by issueing:

 
mysqldump --default-character-set=UTF8 -u [user] -p -h [host].dreamhosters.com [database] -r [output].sql
 

This resulted in a database dump later used for importing into the new MySQL instance.

Setting up the Amazon EC2 Instance

To make your Amazon life easier start with installing the Firefox plugins Elasticfox and S3 Organizer. These tools give you a GUI view of the somewhat more complex underlying details of Amazon.

Amazons' cloud solution is based on a user account. There are no start-up fees, you only pay for what you use. So if you still haven't got an account, sign-up now. The rest of this article assumes you have signed up and set up accounts for Amazon EC2, Amazon S3 and Amazon EBS. Start by reading the EC2 Getting Started Guide and don't stop reading until you have created an ssh key pair, a security group and have a server instance running.

Now you should also have a private key and a certificate in your ${EC2_HOME} (preferrably ${HOME}/.ec2/) folder of your local computer. These two files should also be uploaded to your instance /mnt folder to make it possible to bundle and upload your images to your S3 "bucket(s)" (buckets are one level deep folders/directories). Make sure you put them in the /mnt folder to prevent them from being bundled with the image and thereby increasing the risk of having your identity revealed.

Bundling, by the way, should be done when you've made changes to your server instance that you don't want to loose. The result of a bundling and uploading is an image with a specific Amazon Image Id (AMI ID) reflecting the state of your server instance at the moment of bundling. You also have to register the image with the manifest file created for you. If you do that you could then safely close down your instance and have it up and running again by issuing the ec2-run-instances command with the AMI ID as parameter. A way to save money.

Bundling and uploading was made easier for me as I used two scripts created by my colleague Ulrik. I put these scripts in the /root folder.

bundle-myself.sh:
 
#!/bin/sh
prg=`basename $0`
dest=/mnt
image=blog-image
 
USAGE="Usage: $prg <datetime>
 
Example:
% $prg 20090415-1000
Bundling into $dest/$image-20090415-1000.manifest.xml"
 
if [ $# -ne 1 ]; then
  echo "$USAGE"
  exit 1
fi
date=$1
echo "Bundling into $dest/$image-$date.manifest.xml"
ec2-bundle-vol --debug -k /mnt/pk-PRIVATE_KEY.pem -c /mnt/cert-CERTIFICATE.pem -u USER_ACCOUNT_ID -d $dest -p $image-$dat
e -r i386
 

upload-myself.sh:
 
#!/bin/sh
prg=`basename $0`
dest=/mnt
image=blog-image
 
USAGE="Usage: $prg <date>
 
Example:
% $prg 20090415-1000
Uploading bundled AMI parts to https://s3.amazonaws.com:443/[S3_BUCKET] ...
Uploaded blog-image-20090415-1000.part.00 to https://s3.amazonaws.com:443/[S3_BUCKET]/blog-image-20090415-1000.part.00.
..."
 
if [ $# -ne 1 ]; then
  echo "$USAGE"
  exit 1
fi
date=$1
ec2-upload-bundle --retry -m $dest/$image-$date.manifest.xml -b blog-images -a [PUBLIC_KEY] -s [PRIVATE_KEY]
 

Another thing I needed was a static IP address for my instance. To solve this I used Amazons Elastic IP service which doesn't cost anything as long as the generated IP address is in use. So, I generated an Elastic IP address and assigned it to the instance.

Installing Necessary Software

The blog require Apache2, Php5, MySQL and the Php GD graphics package, to start with. This was easily installed using Advanced Packaging Tool, apt-get:

 
# sudo apt-get update
# sudo apt-get install apache2 php5 mysql-server php5-mysql php5-gd
 

Setting up the Database for EBS

EBS volumes are, as you might know, more persistent than EC2 instances and will survive if instances go down. Volumes are attached to the instance and could as well be detached and re-attached to new instances. As EBS volumes are not 100% reliable you should also take snapshots of your database data and have it uploaded to an S3 bucket. All this is nicely described in Eric Hammonds' EBS and MySQL tutorial, so I urge you to read it if you haven't yet.

The only thing I sidestepped in the tutorial was the MySQL configuration to point out the data and log directories. Instead I used symbolic links:

 
# ln -s /vol/lib/mysql/ /var/lib/mysql
# ln -s /vol/log/mysql/ /var/log/mysql
 

The tutorial also walks you through automatic snapshots which requires the EC2 command line tools. Here is a nice tutorial on setting these tools up on Ubuntu.

I had to tweak the environment variable settings in ec2-snapshot-xfs-mysql.pl script from

BEGIN {
  $ENV{PATH} = '/bin:/usr/bin:/usr/local/bin';
}

to

BEGIN {
  $ENV{PATH} = '/bin:/usr/bin:/usr/local/bin:/usr/sbin:/root/ec2-api-tools/bin';
  $ENV{EC2_CERT} = '/mnt/cert-CERTIFICATE.pem';
  $ENV{EC2_PRIVATE_KEY} = '/mnt/pk-PRIVATE_KEY.pem';
  $ENV{EC2_HOME} = '/root/ec2-api-tools';
  $ENV{HOME} = '/root';
  $ENV{JAVA_HOME} = '/usr/lib/jvm/java-1.5.0-sun';
}

to get the snapshotting to fully work. I also wrapped the ec2-snapshot-xfs-mysql.pl script for a more convenient way to handle the input:

/root/snapshot-myself.sh

#!/bin/sh
mountpoint=/vol
volume_id=[VOLUME_ID]

su -c "/root/ec2-snapshot-xfs-mysql.pl ${mountpoint} ${volume_id} > /vol/cron.log"

To automatize the snapshotting I set up a simple crontab job by issueing:

 
# crontab -e
 

and then edited it like this

05 */6 * * * sh /root/snapshot-myself.sh

which makes the job run five minutes past every six hours, every day.

I also had to set the timezone info to apply to our local time. The easiest way I've found out of setting this for a Linux server is like this:

 
# ln -sf /usr/share/zoneinfo/Europe/Stockholm /etc/localtime
 

Configuring MySQL

To configure MySQL for UTF-8 content make sure the /etc/mysql/my.cnf file contains the following:

[client]
default-character-set=utf8

[mysqld]
#
# * Basic Settings
#
init_connect='SET collation_connection = utf8_general_ci'
init_connect='SET NAMES utf8'
character-set-server=utf8
collation-server=utf8_general_ci
default-character-set=utf8
skip-character-set-client-handshake

Now, if you haven't yet, go on and start your MySQL server. You need to create your wordpress database. It's as simple as:

 
# mysql -u [user] -p
> CREATE DATABASE [database_name]
 

To create the user with which WordPress will access the database:

 
> GRANT ALL ON [database_name].* TO '[user]'@'localhost' IDENTIFIED BY '[password]';
 

And to import your database dump into the newly created database:

 
# mysql -u [user] -p [database_name] < [output].sql
 

You shouldn't need to specify any character set or collation when importing thanks to the configuration done in /etc/mysql/my.cnf.

In order for us to stick with our domain name and have no downtime we needed to use the instances IP address until we were finished testing. This means I had to replace two properties in the database to prevent the blog from redirecting to the original host; the 'siteurl' and 'blogurl' properties residing in the option_value column of the wp_options table. The commands

 
# mysql -u [user] -p
> UPDATE wp_options SET option_value='http://[ip_address]/wordpress/' WHERE option_name='siteurl'
> UPDATE wp_options SET option_value='http://[ip_address]/wordpress/' WHERE option_name='blogurl'
 

fixed this. As soon as the DNS info was propagated throughout the Internet this was reset to "http://blog.jayway.com/wordpress" from within the WordPress admin interface.

Apache Configuration

Our WordPress installation makes use of permalinks and redirects to FeedBurner for RSS statistics. This is based on .htaccess redirects (at least that's how we've solved it) which in turn is dependent on the Apache redirect module. So, in order to make this to work I had to enable that.

 
# sudo a2enmod rewrite
 

Easy as that! One more thing on redirections. I had to configure Apache to allow redirection directives specified in the .htaccess file by changing the /etc/apache2/sites-available/default configuration of the element from

 
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>
 

to

 
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride All <----------
		Order allow,deny
		allow from all
	</Directory>
 

Securing the WordPress Admin Interface

Again, I choose to refer to detailed and well written sources. Here are the two articles I followed to create a self-signed certificate and to configure Apache to use it.

• Creating a self-signed certificate
• Configuring Apache for SSL on Ubuntu

To enable https access from the "outside world" I also edited the group policy to open up port 443. This could easily be done via the Elasticfox Firefox plugin mentioned earlier.

Installing and Configuring WordPress

The default location for Apache httpd server for reading content is /var/www/. If you, as I did, have an existing WordPress installation you can simply copy this to the new server. I ended up having it located at /var/www/wordpress/ on the Amazon instance.

Once the database connection properties in the wp-config.php wordpress configuration file was adapted to the new environment I was able to access the WordPress installation via a browser and the Amazon instances' IP address. Sweet!

And as we have our WordPress installation in a separate "/wordpress" folder instead of in the document root of Apache we need to specify, in WordPress admin settings, that the 'Site URL' should be "${host}/wordpress" but the 'Blog URL' should simply say "${host}" to prevent the resulting URL from containing "/wordpress/...". This also requires that we move the .htaccess and index.php files from within our "/wordpress" folder and up one level to "/var/www/", in our case.

In the index.php file, make sure to change from

 
/** Loads the WordPress Environment and Template */
require('./wp-blog-header.php');
 

to

 
/** Loads the WordPress Environment and Template */
require('./wordpress/wp-blog-header.php');
 

If or when specifying the permalinks, WordPress must have write access to the .htaccess file, so 'chmod' that file.

 
# chmod 755 .htaccess
 

Otherwise it will complain and simply print out the .htaccess modification it tried to modify with and you are left to change it manually, instead.

Sending Mail from WordPress

Mailing is used in WordPress when generating new passwords to users, among others. We also use it to send data backups to the administator of the blog.

Setting up a mail server isn't a trivial task. I gave it a thought but quickly put it aside in favour of the simple WordPress plugin WP Mail SMTP as I realized I could make use of our existing corporate mail server. It was a breeze to configure the plugin with the necessary mail user credentials.

So, here we are. Sailing these fluffy clouds.

I used the Jungle Disk software to provide the mapping from the S3 web service interface to a network drive. You give Jungle Disk your S3 account information, and it provides a network disk which you can use as the backup device. I first tried my existing backup software (Synchronize! Pro X) and it happily began creating directories and copying files. I then tried the backup feature built into Jungle Disk, and it works fine too. It's not fast compared to a local FireWire disk. It's limited to your upload bandwidth, which often is substantially smaller than the download bandwidth.

Anyway, it has been tugging along today storing 5 GB of by digital photos. It feels pretty good to know that my photos are now stored encrypted and with 99.99% availability somewhere far far away, costing me only 75 cents per month after the initial $2 for the transfer.