Migrate Your WordPress Blog to a Bitnami EC2 Instance

by on January 20, 2011 · 35 comments

The cool thing about technical howtos like this is the ability to share with other folks who get just as excited. Looking to save costs on our blog hosting, Amazon’s announcement of a 12 month free usage tier put us in the short list for migration. As Matthias did the bulk of the joyent server setups & linode migrations, it was high-time I got my hands dirty. Without further ado, here’s how I migrated AWO to our first Amazon EC2 instance. Let’s get our geek on!

Choosing bitnami

My initial plan was to grab some debian based Amazon Machine Image (AMI) and follow the various “howto config wordpress on lamp” posts out there, but Matthias asked if I’d had a look at bitnami. I’d never heard of it before and, truth be told, it actually sounded like some kind of BitTorrent client to me. Digging in a little deeper, I soon understood what all the hoopla was about and was soon sold on a nicely contained, pre-configured bitnami WordPress AMI. With Ubuntu and RedHat distros to choose from, you can even use an AMI which launches via Elastic Block Storage (EBS) – how’s that for easy?

Just in case you don’t know what EBS is good for – an Instance Store EC2 server doesn’t come with a disk for storing your data. Meaning if you plan on making changes to your AMI at runtime that you’d like to see after a reboot, you need to attach a disk (EBS in Amazon AWS speak).

Setting up a new EC2 instance on AWS

One of the catches for a 12 month, free EC2 instance is a new Amazon account, so be sure to register a new email address or you might get a surprise bill next month! And, yes, you will also need a valid credit card to open your account, because if your instance exceeds the free usage tier limits, you will be charged.

For those of you running multiple WordPress blogs, bitnami even offers preconfigured MultiSite AMIs!

You should definitely spend a few minutes considering the region in which to launch the instance. If you’re using Google Analytics, checkout your “Audience -> Geo -> Location” to see the regions sending your blog the most visits. Our primary traffic comes from the U.S. West Coast, but there are significant visits throughout the Continental U.S. and Europe.

traffic sourcesWe ultimately went with the mid-point “U.S. East” region.

Once you decide on a region and an instance type, just click on the desired bitnami WordPress AMI. You’ll be automatically forwarded to the AWS Console for launching that instance in the desired region:

Screen Shot 2014-01-11 at 15.12.16

The only change you’ll need to make to the default settings, is to add HTTP port 80 to the default security group. The Wizard will even give you the option of locking this down to your current IP address which I certainly recommend.

Once you click “Launch”, you’ll be prompted to create and/or download a public/private keypair in a .pem file (hereinafter referred to this howto as ‘my_aws_ec2.pem’) – this is your SSH keypair to get in the backdoor of your new instance:

keypair

After your instance launches, click on it in the AWS Management Console. The description tab below will list a “Public DNS” (hereinafter referred to in this howto as ‘public_dns.amazonaws.com’):

publicDNS

Point your browser to this URL and you’ll see the bitnami Welcome page with a few links including your new WordPress install. Not too shabby – a running blog on a new virtual server all in the space of about 5 minutes!

Last, but not least, make an EBS snapshot of your running instance so you have a known fallback before entering the next phase of the migration. Don’t forget about this capability – at every step where you feel you’ve accomplished an important step, back it up!

Migrating Your WordPress Blog

On your existing blog, dump your database and tarball your WordPress htdocs path.

$ mysqldump -uwordpress -p awo > awo_wp.sql
$ tar -cjf awo_htdocs.bz2 /home/awo/public

Now, let’s get those dumps copied over to your new EC2 instance:

$ scp -i my_aws_ec2.pem awo_wp.sql awo_htdocs.bz2 bitnami@public_dns.amazonaws.com:/home/bitnami/originals/

And, don’t forget to copy over any releveant configurations and performance tweaks in your Apache (apache2.conf, httpd.conf) and MySQL (my.cnf, conf.d/*.cnf) setups.

Moving on to the IP address. Goto the Elastic IP’s entry in your AWS console. Allocate New Address and associate it with your newly created (and probably only) EC2 instance. While we’re waiting for the DNS to catch up, edit your local (meaning the computer you’re using to do the migration) /etc/hosts accordingly, overriding your blog domain with this new IP address.

$ sudo vi /etc/hosts
...
<NEW_ELASTIC_IP> www.agileweboperations.com

On the new instance do something like the following (remembering your EBS snapshot above ;)),

$ rm -Rf /opt/bitnami/apps/wordpress/htdocs/
$ tar -xjf ~/originals/awo_htdocs.bz2 /opt/bitnami/apps/wordpress
$ mysql -uroot -pbitnami bitnami_wordpress < ~/originals/awo.sql

Note the location of the bitnami wordpress, and the MySQL root password and wordpress database schema.

Now, let’s setup a virtual host to point to your new blog:

$ sudo vi /opt/bitnami/apache2/conf/httpd.conf
...
# Virtual hosts
Include conf/extra/httpd-vhosts.conf
...
$ sudo vi /opt/bitnami/apache2/conf/extra/httpd-vhosts.conf
...
<VirtualHost *:80>
    DocumentRoot "/opt/bitnami/apps/wordpress/htdocs"
    ServerName agileweboperations.com 
    ServerAlias www.agileweboperations.com
    ErrorLog "logs/awo-error_log"
    CustomLog "logs/awo-access_log" common
</VirtualHost>
...

Any special apache configuration (KeepAliveTimeout, ServerTokens, etc.) can be edited @ /opt/bitnami/apache2/conf/extra/httpd-default.conf. Be sure to include these edited files in the base apache conf:

$ sudo vi /opt/bitnami/apache2/conf/httpd.conf
...
# Server-pool management (MPM specific)
Include conf/extra/httpd-mpm.conf
...
# Virtual hosts
Include conf/extra/httpd-vhosts.conf
...
# Various default settings
Include conf/extra/httpd-default.conf

Finally, remember your canonical redirects! No duplicate content here:

$ vi /opt/bitnami/apps/wordpress/htdocs/.htaccess
RewriteEngine On
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.agileweboperations\.com
RewriteRule (.*) http://www.agileweboperations.com/$1 [R=301,L]

Restarting processes with bitnami is accomplished using the ctlscript.sh. To restart Apache, just run sudo /opt/bitnami/ctlscript.sh restart apache.

MySQL my.cnf may be edited @ /opt/bitnami/mysql/my.cnf and the restart is accomplished in much the same way : sudo /opt/bitnami/ctlscript.sh restart mysql.

You can also use ctlscript.sh to check out your apache & mysql status:

$ sudo /opt/bitnami/ctlscript.sh status
apache already running
mysql already running

Gotchas & Caveats

Installing ec2-api-tools is not possible on a t1.micro Ubuntu instance. However, if you do need AWS CLI tools, checkout … well, their AWS CLI tools which run on python right out of the box!

Remember that stopping an instance removes the elastic IP address! So if you’re wondering why your restarted instance is no longer available via SSH, go reattach the elastic IP.

Additionally, the “Public DNS” name will dynamically change upon starting an instance. Don’t go hardcoding this into your /etc/hosts file and bookmarking it for all eternity!

For those spend-thrifty types, just starting an instance immediately costs you 1 hour of computing time, so don’t be surprised to see a bit more usage ($$$) than you’d expect if you do this a few dozen times over a weekend of experimentation.

YMMV, but the bitnami:daemon user:group which owned /opt/bitnami/apps/wordpress/htdocs was not sufficient for performing any “inline” wordpress upgrades/installs – meaning trying to upgrade/install a plugin asked for FTP access information. Performing a sudo chown -Rf daemon:daemon /opt/bitnami/apps/wordpress/htdocs did the trick, however. Anyone more familiar with bitnami, please feel free to step in and clarify.

MySQL died the first night after relaunch. A strange “Killed” showed up in the /opt/bitnami/mysql/data/mysqld.log:

101229 15:41:33 [Note] /opt/bitnami/mysql/bin/mysqld.bin: ready for connections.
Version: '5.1.50'  socket: '/opt/bitnami/mysql/tmp/mysql.sock'  port: 3306  MySQL Community Server (GPL)
Killed
101230 00:15:51 mysqld_safe Number of processes running now: 0
101230 00:15:51 mysqld_safe mysqld restarted
101230  0:15:53 [Note] Plugin 'FEDERATED' is disabled.
Killed
101230 00:15:54 mysqld_safe mysqld from pid file /opt/bitnami/mysql/data/ip-10-112-91-108.pid ended
101230 07:32:02 mysqld_safe Starting mysqld.bin daemon with databases from /opt/bitnami/mysql/data
...

Turns out, the bitnami image makes use of Linux’s OOM Killer which takes a heavy-handed approach to potential Out of Memory issues before they over swap the server. The problem was I forgot to include the httpd-mpm.conf file I had altered so carefully above in the httpd.conf, so there were a bunch more simultaneous clients than this micro server could manage.

After you get an elastic IP setup, you may have to rerun the /opt/bitnami/scripts/init/updateip file. To manually set the wp_options values to your blog’s domain name:

$ mysql -uroot -p bitnami_wordpress
mysql> update wp_options set option_value = 'http://www.agileweboperations.com' where option_name in ('siteurl', 'home');
Query OK, 2 rows affected (0.00 sec)

Our “free” year worked out great for such little effort (~1 day) and was well worth the savings. Check out where the rubber meets the road – our Pingdom response times before and after the relaunch:

Professional Grade Virtual Computing for “Free”

Did you enjoy this article? Get new articles for free by email:

Comments

  1. says

    Thanks for taking the time to document the WordPress migration.

    Is the instance that you started a micro instance? The last I heard that was the only one that was free for one year. If it is…how do you know that from the name?

  2. says

    The free instance is called a “micro” instance. Thanks, Barry, for pointing out I’d missed this – I’ve updated the post with a more helpful image!

  3. says

    That is so annoying how WordPress resets the option back to the public url. Any idea why? or a way to avoid it? I know how to fix it, but I find I have to restart the instance a lot during development.

  4. Nicolas Echavarria says

    Dan,

    One probably “dumb” question…
    Does the EC2 AMI “maintains” itself updated in the latest, stable build of WP?
    Or do you have to fully account for all maintenance and updates after you’ve created the new instance under your AWS account?
    Thanks a lot for sharing!!

    Best,

    Nicolas E.

  5. says

    Unfortunately, Nicolas, the AMI does not maintain itself. But I did configure the directory permissions in such a way that you can upgrade/maintain WP by clicking the appropriate button in wp-admin. Additionally, you’re responsible for keeping your Ubuntu instance patched via apt-get from the cmdline.

  6. says

    Hi Dan, Thanks for your post. Very informative. I have an additional question. After you have your EC2 instance ready and running, what is the easiest way of keeping the instance backed up in case the instance goes silent?

    Thanks, Geo

  7. says

    Arrghh, I’m writing that post right now ;) … I’ll give you a hint though : ‘ec2addsnap’ from the ec2 api tools. As soon as I get that post published, I’ll add a link here!

  8. Mark says

    I ran into the issue you had where MySql died a couple of days after relaunch. I found the same log entries that you found in the /opt/bitnami/mysql/data/mysqld.log so my issue was also likely due to OOM Killer. I have also not included the httpd-mpm.conf in my httpd.conf. Did you use the httpd-mpm.conf file as-is from the Apache/config/extras directory or did you modify the settings in that file?

  9. says

    definitely modified the httpd-mpm.conf prefork module to :

    
        StartServers         10
        MinSpareServers      10
        MaxSpareServers      10
        MaxClients           20
        MaxRequestsPerChild   0
    
    

    But we are also using MaxCDN to serve all our static resources.

  10. shan says

    Got a different question, maybe you can help me out.
    I’ve got a mysql database set in amazon, and i’m trying to dump one table back to my local machine (it’s a log table and i want to test some log processing applications on my live data)
    I’ve looked and looked, but didn’t any way of doing this :-(
    Any ideas ?
    10x
    Shan

  11. says

    Aloha Dan-
    I’m thinking of transferring a large website from a mediatemple DV, where I’ve learned way more than I truly wanted to about managing a server, over to a bitnami stack. The site I have is a wordpress based site that really utilizes it as a CMS whereby I have a lot of database querying going on. I’ve had to spend days tuning my MT-DV and still run into server hogs (also have cleaned the code over there to only use wp functions so it limits my bad querying). The site only gets about 1000 uniques a day but still is a database hog. I’m going to be using W3TC with an S3/Cloudfront CDN to offload some of the work, and will most likely be on a small instance (unless you convince me I could get a micro to work :).)

    I’ve done a few installs of wordpress using bitnami stacks and plane jane Amazon AMI’s just to get familiar with things. But I’ve run into the same database kill issues you mentioned. I’m not going to claim to be a httpd.conf wizard by any means and was wondering if you could elaborate or point me in a direction as to why you list the values you do in httpd.mpm.conf and or if I run into issues, what method of adjusting I should follow to troubleshoot?

    Any help would be great, but this tutorial has already helped tremendously, it was my starter to dive in a while back.

    OK, back to hammering my test micro install with your httpd tweaks to see if I can get the database to shutdown.

  12. says

    Hi Erik,

    definitely would not recommend a micro instance with a 1000 uniques a day (and a lot of db hits). The values in the httpd.conf are due to memory constraints on the micro instance which only gives you about 700MB. Each of our apache threads (client) uses about 15MB – 15*20 = 300MB. So max clients leaves a “bit” of room for Linux and MySQL (but things do get tight).

    Another piece of advice, be careful how you cache your static resources at the CDN. Do it longer than 24 hrs (else, W3TC will recrawl your entire site everyday looking for new resources)!

    Good luck (and have fun)!

  13. says

    Hey thanks Dan!
    I’ve found a bunch of good resources on tuning apache at EC2, still getting the hang of that. Curious as to the base setting of 0 for MaxRequestsPerChild?

    Also, i’ve set a couple up without enabling httpd-vhosts.conf and configuring virtual hosts. What’s the advantage of using virtual host?

    Aloha,
    Erik

  14. says

    MaxRequestsPerChild 0 means we don’t care about killing the thread after x requests – allow it serve an infinite number of requests.

    Virtual hosts are helpful if you’re running more than one website (subdomain/domain name) from the same Apache server.

  15. says

    Awesome Dan, thanks a bunch again.
    That was where my mind was headed, just thought I’d bounce it off someone.
    If I ever write on my progress I’ll be sure to mention you and this great resource.
    Aloha

  16. Eric says

    Hey Dan,

    I’ve been going through the various on-line tutorials on this for a couple weeks and can’t get it to work. I keep having issues with the re-directs. When I change my A Name to point to the Elastic IP, it works fine. Then when I go into httpd.conf to change docroot and directory to to point from bitnami/apache2/htdocs to bitnami/apps/wordpress/htdocs I lose the ability to access my wp-admin. Also, all the links on the default WP blog are pointing to the amazonurl/wordpress instead of myurl.

    I tried using your virtual host method instead of httpd.conf and had the same issue.

    Any ideas?
    Thanks

  17. Gregor says

    Hi Dan – MySQL crashes continuously (in less than a couple of hours). This is a test site (a clone of a production site – with no users except me). It’s WP + Buddypress + about 20 plugins. The DB is less than 2MB.

    I did follow your instructions, I modified the httpd-mpm.conf and included
    Include conf/extra/httpd-mpm.conf

    My MySQL log shows:
    111210 09:43:37 mysqld_safe Number of processes running now: 0
    111210 09:43:37 mysqld_safe mysqld restarted
    111210 9:43:37 [Note] Plugin ‘FEDERATED’ is disabled.
    111210 9:43:37 InnoDB: The InnoDB memory heap is disabled
    111210 9:43:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins
    111210 9:43:37 InnoDB: Compressed tables use zlib 1.2.3
    111210 9:43:37 InnoDB: Using Linux native AIO
    111210 9:43:37 InnoDB: Initializing buffer pool, size = 128.0M
    InnoDB: mmap(137363456 bytes) failed; errno 12
    111210 9:43:37 InnoDB: Completed initialization of buffer pool
    111210 9:43:37 InnoDB: Fatal error: cannot allocate memory for the buffer pool
    111210 9:43:37 [ERROR] Plugin ‘InnoDB’ init function returned error.
    111210 9:43:37 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
    111210 9:43:37 [ERROR] Unknown/unsupported storage engine: InnoDB
    111210 9:43:37 [ERROR] Aborting

    111210 9:43:37 [Note] /opt/bitnami/mysql/bin/mysqld.bin: Shutdown complete

    111210 09:43:37 mysqld_safe mysqld from pid file /opt/bitnami/mysql/data/ip-10-48-201-206.pid ended

    Thanks!

  18. says

    First idea is that since this is a clone of your production site running on a test server, you might not have the same available memory on this machine. Double check the my.cnf file and how much innodb memory it’s trying to allocate – it’s probably way more than the available RAM.

  19. Gregor says

    Thanks Dan – there is no reference to innodb in my.conf. Do you know of a tutorial that shows how to add innodb to my.conf.

    There are a bunch of example my.conf files on Bitnami (/opt/bitnami/mysql/support-files). These include a reference to innodb, but I have no clue how to add them to the original my.conf . Also, isn’t it strange that there is no innodb setting on my.conf?

    The cloned site in question is running on micro ec2. The corresponding site runs on a shared hosting account, together with 4 other WP sites. I thought that a micro ec2 would have much more memory and run faster. And, it actually runs much faster – that is, till MySQL crashes.

    Thanks for your help!

  20. Gregor says

    Dan – do you have a sample my.cnf file for an ec2 micro instance or can you recommend a configuration?

    Thanks!

  21. James Fishwick says

    How did you calculate these numbers based on a micro instance? Tips for how I would modify this for a small instance?

  22. says

    If you run top and look for your Apache processes, you get an idea about how much memory each thread consumes. Multiply this times your MaxClients to get an idea of the max memory Apache could consume. Add this value to the max memory of your MySQL db (checkout out http://mysqltuner.pl) for an estimate of the total possible memory used by your LAMP stack.

  23. says

    This is the best tutorial I’ve found on using the Bitnami WordPress stack on EC2. Thanks a million.

    One question: Have you actually tried running more than one site on a micro instance, either with separate vhost entries, or with a WordPress Multi-Site install? (I’d like to transfer a few low-traffic sites from a shared hosting account when my current subscription period runs out.)

Trackbacks