How to install Scalr on Ubuntu 8.10 EC2 Instance

Presentation:

If Amazon EC2 doesn’t ring a bell to you, chances are that you are looking at the wrong page to find solutions for your problems. EC2 stands for Elastic Compute Cloud and it’s a service offered by Amazon. I will not enter in details about the advantages of using it, since this is not the scope of this post. You can read more about it here: http://aws.amazon.com/ec2/

Scalr is a fully redundant, self-curing and self-scaling hosting environment utilizing Amazon’s EC2. You basically can build farms of Amazon’s instances that can do load balancing using nginx, serve web pages using Apache 2, use MySQL master-slave servers or maybe you may want to define your own roles.

The beauty of this is that you don’t have to monitor the health of your server infrastructure, scalr will do it for you. If a node type gets overloaded scalr will launch another instance to spread the load and the cluster will be reconfigured.

Preparing:

Generate a new key for scalr instance:

ec2-add-keypair scalr-keypair > id_rsa-scalr-keypair

Edit id_rsa-scalr-keypair so it begins with

-----BEGIN RSA PRIVATE KEY-----

and is terminated with

-----END RSA PRIVATE KEY-----

Make sure you have the correct permissions for this key:

chmod 600 id_rsa-scalr-keypair

If everything went ok you should see your new key when executing

ec2-describe-keypairs

Choosing the right AMI:

For the instance we will be using ami-7806e211 which is an AMI containing a base install of Ubuntu 8.10 Intrepid Ibex Release. More details here.

Start the instance:

ec2-run-instances -z us-east-1a -k scalr-keypair ami-7806e211

You will get some output, look for the line that begins with INSTANCE and write down somewhere the id of the instance (i-XXXXXXXX) and the address of the instance(ec2-XX-XX-XX-XX.compute-1.amazonaws.com). The status of your instance should be pending.

Check in a couple of minutes the status of your instance:

ec2-describe-instances i-XXXXXXXX

When the status is running it means that your instance is ready for work. You should have at least ssh and web ports open(22 and 80). If you are not sure execute the following commands:

ec2-authorize default -p 22
ec2-authorize default -p 80

Now connect to your instance using ssh:

ssh -i id_rsa-scalr-keypair -v ec2-XX-XX-XX-XX.compute-1.amazonaws.com

First time you connect you will be asked if

Are you sure you want to continue connecting (yes/no)?

Type yes and you should be the happy owner of a fresh Ubuntu Intrepid Ibex instance.

Update your system now:

apt-get update
apt-get upgrade

After the update is completed, logout and reboot your instance:

ec2-reboot-instances i-XXXXXXXX

Installing required software:

Reconnect to your instance and install MySQL server and php extensions:

apt-get install bind9 mysql-server mysql-client apache2 php5-cli libapache2-mod-php5 php5-mysql php5-mcrypt php5-mhash

When you install MySQL server you will be prompted to setup a password for the root account. Don’t forget it, you will need it. Also you will have to restart Apache2 server after you finish installing everything, like this:

/etc/init.d/apache2 restart

You could also download their php script that checks if your system has all the prerequisites

wget http://scalr.googlecode.com/files/checkenvironment.php
mv checkenvironment.php /var/www/
chmod a+r /var/www/checkenvironment.php

Now point your browser to http://ec2-XX-XX-XX-XX.compute-1.amazonaws.com/checkenvironment.php and see if everything it’s ok.

Most likely you will get only these 2 errors:

Errors:
• Cannot find SSH2 functions. Make sure that SSH2 Functions enabled.
• Cannot find SNMP functions. Make sure that SNMP Functions enabled.

Here is how to quick fix it:
Adding SSH2 support to PHP5, better known as: why don’t we have php5-ssh2?

apt-get install php5-dev php-pear libssh2-1 libssh2-1-dev

Thought it will be easy? Not so quick. Try to install it with:

pecl install ssh2 "channel://pecl.php.net/ssh2-0.10"

I got an error saying:

ERROR: `make' failed

Great! Let’s fix that stupid error. Edit the file /tmp/pear/download/ssh2-0.10/ssh2.c and replace the line containing:

#if LIBSSH2_APINO < 200412301450

with

#if false

Go to directory /tmp/pear/download/ssh2-0.10/ and compile the stuff manually:

make && make install
echo extension=ssh2.so >> /etc/php5/apache2/php.ini

I don’t get it why they don’t fix this thing. A lot of people are having this problem and are complaining!

Luckily for you and me snmp is a breeze, it is already in repositories:

apt-get install php5-snmp

Restart apache server and check now if you have all the required extensions for scalr. You should have them.

Getting the latest version of Scalr:

At the time of writing this article latest version was 1.0 RC2
Go to Scalr download page and copy the link to the latest release. Download it using wget:

wget http://scalr.googlecode.com/files/scalr-1.0RC2.tar.gz

Extract it:

tar zxvf scalr-1.0RC2.tar.gz

Create database for scalr and import the sql:

mysqladmin -p create scalr
mysql -p scalr < scalr/sql/database.sql

Put the scalr application in /var/scalr and change permissions as suggested in the spartan documentation of scalr:

mkdir /var/scalr && cp -R scalr/* /var/scalr/
chmod 777 -R  /var/scalr/app/cache /var/scalr/app/cron/cron.pid /var/scalr/app/etc/.passwd

ATTENTION: I’m not planing on using this instance for anything else except scalr. Also this is a clean install so I don’t have anything of interest in /var/www. Read carefully the following first line:

rm -rf /var/www/
ln -sf /var/scalr/app/www /var/
chmod a+rX -R /var/www

Edit the file /var/scalr/app/etc/config.ini and update it to your values:

[db]
driver=mysql ;Actually mysql is the only option here - mysqli doesn't support nconnect(), which is essential for PCNTL (which is essential for crobjobs)
host = "localhost"
name = "scalr"
user = "root"
pass = "*YOUR PASS HERE*"

TO DO: make a mysql user for scalr.

Put your EC2 access certificate into /var/scalr/app/etc/cert-XXXXXXXXXXXX.pem
Put your EC2 private key into /var/scalr/app/etc/pk-XXXXXXXXXXXX.pem
ATTENTION: This part is a bit tricky. If you don’t put the right settings you will not be able to start instances. I warned you!
Login to Amazon AWS and go to Home->Your Account->Access Identifiers
Go to http://ec2-XX-XX-XX-XX.compute-1.amazonaws.com/ and login with admin/admin
Go to Settings->Core Settings. Modify the following fields:
Password: duh!!! change it!
Email: your email address here
Account ID: AWS Account Number, it’s called Account Number in AWS and it’s top right. Remove the ‘-‘ from the number
Key Name:Scroll down in AWS untill you see Your X.509 Certificate: Copy everything between ‘cert-‘ and ‘.pem’. Also XXXXXXXXXXXXXX is the string in the filenames of cert-XXXXXXXXXXXXX.pem and pk-XXXXXXXXXXX.pem. If they don’t match you will have problems.
Access Key:Look for Your Secret Access Key: in AWS and click on Show. Copy the string.
Access key ID: is Your Access Key ID: in AWS.

Hit save.

setting crontab:

Type crontab -e and add the following lines to cron:

* * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --Poller
1 1 * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --RotateLogs
*/15 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --MySQLMaintenance
*/6 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --DNSMaintenance
*/3 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --DNSZoneListUpdate
*/2 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --DBQueueEvent
*/11 * * * * /usr/bin/php -q /var/scalr/app/cron/cron.php --Cleaner

You are done. I hope.

How to terminate the instance:

I thought to write down instructions on how to terminate an instance. You should know how, but just in case, here it is how to stop the instance forever and not pay for it anymore. ATTENTION: terminate will really delete the instance so there is no way you can reconnect to it or recover it. Double check what instance you terminate!

ec2-terminate-instances i-XXXXXXXX
Hermes Borse

Hello just wanted to give you a quick heads up. The text
in your article seem to be running off the screen in Internet
explorer. I’m not sure if this is a format issue or something to do with browser compatibility but I thought I’d post to let
you know. The layout look great though! Hope you
get the problem resolved soon. Thanks

Octavian Neacsu

Thank you! The article is now over 1 year old so probably many other things have changed. If I have some free time I will write an article about scalr latest version and how to install it on ubuntu 9.10 or centos.

Anonymous

Very good post. The 'checkenvironment.php' is now changed to 'testenvironment.php', also the 0.11.0 version 'pecl install ssh2 "channel://pecl.php.net/ssh2-0.11.0" will now install without error.

Octavian Neacsu

I didn’t have much time lately to focus more on scalr, but I’ll take my time this week to see what is causing the problems you are experiencing.

akira_atl: great thing, did you write about setting snmp and configuration steps to work with scalr somewhere? I would greatly appreciate any bits of informations as I try to write a complete scalr tutorial.

Mat, James: there is some documentation about nameservers here http://code.google.com/p/scalr/wiki/ConfiguringNameservers
I didn’t test it yet, I will try it out and post the results here.

akira_atl

Octavian, I figured the restarting issue – I had to install snmp, and then setup the event handler URL under core setting so that it has the EC2 instance address to reach for updating status.

Matt

Hi – Great article – many thanks! Just one question though about dns and nameservers. How does this work for pointing domains at various applications? I guess the primary nameserver is the Instance domain name that is running scalr? –

Would be great if you could shed some light on what to enter in the Settings>>General – DNN Settings and what to enter in the Nameservers settings section.

Would really appreciate the help on this –

Many thanks
Matt

eventi

Octavian – That was indeed the problem – thanks – now I’m getting an XML error when I try to run the Application Wizard (AIM/Yahoo/irc.freenode.net: eventi)

Octavian Neacsu

Thank you all for the comments!

akira_atl – from what I've read on scalr discussion's list on google it seems that there is a problem with timeout (the time that scalr waits to receive hostInit from an instance it started). I will look more into it when I have some spare time

eventi – did you add a new client? I added a client after scalr install, then went to Clients->View All, click on Login into user CP. From there follow the Add application wizard.

eventi

Great article Octavian, thanks – Once I’m finished, I can’t do anything though – can’t create farms, or figure out how to launch an instance… Did I do something wrong? TIA

akira_atl

Thanks for the writeup! Not sure if I could’ve figured it out with this as I’m not a PHP guy..

I had slight issue making the ssh2 library, but then tried this and it worked:

phpize
./configure –with-ssh2
make
make install

I seem to be having an issue with instances keep restarting, not sure if it is due to snmp not running, but haven’t gotten chance to look into it in depth..

Your email address will not be published. Required fields are marked *