Activities

June 2012
M T W T F S S
« May   Jul »
 123
45678910
11121314151617
18192021222324
252627282930  

Amazon EC2 : Configure Auto Scaling with Loadbalencer

Auto scaling is the best solution for the application those are expecting huge traffic some hours of a day or for a week long like some content publishing new cause huge traffic . This solutions has both merits and demerits as well. Pls have a look at my earlier post before you implement this mechanism since you may need to re-tune your server/application to work in multiple server in loadbalencer. Otherwise it will be a disaster for you. Be aware of.

There are major 6 steps involved. Each procedures are connected together.

1. Prepare your own EBS AMI and identify it’s AMI ID.
2. Create a Launch Config.
3. Create Scaling Group.
4. Set loadbalancer and Route53.
5. Define scaleUp/ScaleDown policy.
6. Set Alarm to trigger the scaling activities. [ You need to activateCloudWatch service for this]

Let’s start,

1. Prepare new EBS Ami using AWS management console.
This is pretty simple by choosing running instance and select the “instance actions –> Create EBS Ami”. Pls note you do not need to launch this AMI after the creation. “Scaling Croup” will do that. Identify the “AMI ID” once after you create AMI which will be like “ami-e461e06d”

Next you may need to setup AWS API command line environment. You may need JDK 1.6 version installed to use it.
Download Autoscaling and CloudWatch command line tools.

[root@rc-026 bin]#wget http://ec2-downloads.s3.amazonaws.com/AutoScaling-2011-01-01.zip
[root@rc-026 bin]#unzip AutoScaling-2011-01-01.zip
[root@rc-026 bin]# mv AutoScaling-1.0.49.1/ /usr/local/autoscaling
[root@rc-026 bin]# cd /usr/local/autoscaling/
[root@rc-026 bin]# chmod 600 /usr/local/autoscaling/credential-file-path.template
root@rc-026 bin]# java -version
java version "1.6.0"
OpenJDK  Runtime Environment (build 1.6.0-b09)
OpenJDK Server VM (build 1.6.0-b09, mixed mode)
[root@rc-026 bin]# wget http://ec2-downloads.s3.amazonaws.com/CloudWatch-2010-08-01.zip
[root@rc-026 bin]# unzip CloudWatch-2010-08-01.zip
[root@rc-026 bin]# mv CloudWatch-2010-08-01 /usr/local/CloudWatch

The following lines does all for environmental setup for AWS.

JAVA_HOME=/usr/java/jdk1.6.0_14; export JAVA_HOME
PATH=$PATH:/usr/local/autoscaling/bin; export PATH
AWS_AUTO_SCALING_HOME=/usr/local/autoscaling; export AWS_AUTO_SCALING_HOME
AWS_CREDENTIAL_FILE=/usr/local/autoscaling/credential-file-path.template; export AWS_CREDENTIAL_FILE
AWS_CLOUDWATCH_HOME=/usr/local/CloudWatch; export AWS_CLOUDWATCH_HOME
EC2_REGION="us-east-1";export EC2_REGION
PATH=$PATH:/usr/local/CloudWatch/bin; export PATH

Pls note you may need to fill out the Access ID and Secret key inside “/usr/local/autoscaling/credential-file-path.template” file. Also you need to define your Amazon region in which instances and LB are configured. Suppose if you launch an instance at “us-east-1c” , then your EC2_REGION should be use “us-east-1”. Note the sub network name (c) is removed.

2. Create a launch Config
This command choose the instance which is autoscaling to be used.

[root@rc-026 bin]# as-create-launch-config BLC --image-id ami-8e7c147  --instance-type c1.medium --key botch --group Web-secure

Here :
BLC : My LaunchConfig name.
ami-8e7c147 : Private AMI I created.
c1.medium : type of Instance.
Boch : my private key name.
Web-secure : My security group name.

Listing Lauch config

[root@rc-026 autoscaling]# as-describe-launch-configs
LAUNCH-CONFIG  BLC  ami-8e7c147  c1.medium

3. Create Loadbalencer.

You can simply use AWS management console to create a load balencer where your EBS instance are located. Add the required ports and set a criteria for health check ( I uses a html file to check the server availability). Pls note down the load balancer name.

4. Creating Scaling group
Scaling group will define how many numbers of instance to add/remove when scaling process starts.

root@rc-026 autoscaling]#as-create-auto-scaling-group BSG --launch-configuration BLC --availability-zones us-east-1d  --min-size 1 --max-size 3 --load-balancers BochLB

BLC: Lauch config name
BSG: Scaling group name
us-east-1d: Region where EBS AMI kept.
–min-size: Minimal no. of instance to be run.
–max-size: Maximum no. of instance will be run when load is getting over threshold limit.
BochLB: Loadbalancer name.

Listing the Scaling group activity

[root@rc-026 autoscaling]# as-describe-auto-scaling-groups --headers
AUTO-SCALING-GROUP  GROUP-NAME  LAUNCH-CONFIG  AVAILABILITY-ZONES  LOAD-BALANCERS  MIN-SIZE  MAX-SIZE  DESIRED-CAPACITY
AUTO-SCALING-GROUP  BSG         BLC            us-east-1d          BochLB       1         3         1
INSTANCE  INSTANCE-ID  AVAILABILITY-ZONE  STATE      STATUS   LAUNCH-CONFIG
INSTANCE  i-8ba983ed   us-east-1d         InService  Healthy  BLC
[root@rc-026 autoscaling]#

5. Setting up Scaling Policy

Here we may need to set two type policy for ScaleUp and ScaleDown. Here I uses 300 seconds used for each instance addition and termination.

a. ScaleUp Policy

[root@rc-026 autoscaling]# as-put-scaling-policy BScaleUpPolicy --auto-scaling-group BSG  --adjustment=1 --type ChangeInCapacity  --cooldown 300

b. ScaleDown Policy

[root@rc-026 autoscaling]#as-put-scaling-policy MyScaleDownPolicy --auto-scaling-group BSG  --adjustment=-1 --type ChangeInCapacity  --cooldown 300

Note : if you execute the above command from Windows desktop , you need to change the adjustment=-1 to “adjustment=-1” [ put double quotes in between the values. Otherwise you always gets syntax error while executing this command. Also you can bind scaling policy /change the threshold limit etc.. to an Cloutwatch Alarm using the web interface.

You will get a similar output as shown below.

[root@rc-026 bin]# as-put-scaling-policy BScaleUpPolicy --auto-scaling-group BSG  --adjustment=1 --type ChangeInCapacity  --cooldown 300
arn:aws:autoscaling:us-east-1:39679798:scalingPolicy:61b239-c496-444c-bbb6-b6d58:autoScalingGroupName/BSG:policyName/BScaleUpPolicy

Listing Policies

[root@rc-026 bin]# as-describe-policies

6. Set Alarm for trigger the Policy

We need to create two alarms for executing whenever a event meet the alarm condition. This is just like a service which is monitoring the instance status on every seconds and trigger/fire an alarm to execute the defined policy at right time.

Here I have created “MyHighCPUAlarm” and “MyLowCPUAlarm” alarm for this scaling activity. You may also create the same over AWS management console easily.

MyHighCPUAlarm

[root@rc-026 bin]# mon-put-metric-alarm MyHighCPUAlarm  --comparison-operator  GreaterThanThreshold  --evaluation-periods  3 --metric-name  CPUUtilization  --namespace  "AWS/EC2"  --period  60  --statistic Average --threshold  70 --alarm-actions arn:aws:autoscaling:us-east-1:39e323498:scalingPolicy:b9fdf3fs2-327a-4e2f-ab26-b3dff029ce:autoScalingGroupName/BSG:policyName/BScaleUpPolicy --dimensions "AutoScalingGroupName=BSG"

–evaluation-periods: is the time limit when the threshold state continue to run.
–period 60: Is a Alarm check interval
–threshold: Here I set CPU usage alarm will fire ScaleUp Policy when CPU usage is reached 70% for more than 3 minutes.
–metric-name : This is a Resource type and value can be anything related to instance. Like can monitor disk I/O ratio, memory usage, disk usage etc…
alarm-actions : This is a lenghty combination of string which is re-directing to a policy name.

MyLowCPUAlarm

This is used for when scale down policy execute, It’s uses newly created instance will be deleted automatically when CPU load is moving down to 40% in 3 minutes of interval.

[root@rc-026 bin]# mon-put-metric-alarm MyLowCPUAlarm  --comparison-operator  LessThanThreshold --evaluation-periods  3 --metric-name  CPUUtilization --namespace  "AWS/EC2"  --period  60  --statistic Average --threshold  40  --alarm-actions arn:aws:autoscaling:us-east-1:39e323498:scalingPolicy:b9fdf3fs2-327a-4e2f-ab26-b3dff029ce:autoScalingGroupName/BSG:policyName/MyScaleDownPolicy --dimensions "AutoScalingGroupName=BSG"

Listing out Alarms created.

[root@rc-026 bin]# mon-describe-alarms
MyHighCPUAlarm  OK     arn:aws:autoscalin...Name/BScaleUpPolicy  AWS/EC2  CPUUtilization  60  Average  3  GreaterThanThreshold  70.0
MyLowCPUAlarm   ALARM  arn:aws:autoscalin...e/MyScaleDownPolicy  AWS/EC2  CPUUtilization  60  Average  3  LessThanThreshold     40.0
[root@rc-026 bin]#

We all set now 🙂 Unfortunately you will not get any feeling until you test it out first. I would recommend to verify it before implementing.

Creating Scaling group launches a new instance from the mentioned AMI. Don’t delete the instance directly from AWS management console. Those instances will be deleted when you update the scaling group. ( –min-size to 0)

How to disable/de-active Autoscaling.

Just update the Scaling group minimal instance size to Zero.

Terminating auto-scale instances.

[root@rc-026 bin]# as-update-auto-scaling-group BSG --min-size 0 --max-size 0

Deploying new AMI on existing Scaling group.

You can’t do it without delete everythig you created since the scaling group is same. There is few work around,
Create a new Lauch Config and update the scaling group using “as-update-auto-scaling-group” (recommended).
Because each steps of autscaling is connected together. Deletion of scaling group also will delete everything assosiate with it like terminate policies/alarams that has configured. This is the auto scaling process from the bottom Alarms –> Policies –> Scaling Group —> Launch Config

a. Delete the Auto scaling group

[root@rc-026 bin]# as-delete-auto-scaling-group BSG --force

b. Deleting launcg Config

 [root@rc-026 bin]# as-delete-launch-config BLC

Running a test to verify whether it’s working or not ?

Here I’m going to run some CPU intensive command to raise the load. Let’s see what are happening
I use echo the following commands

[root@ip-10-194-203-183 ~]# echo "9999999^999999" | bc &
[1] 3432
[root@ip-10-194-203-183 ~]# echo "9999999^999999" | bc &
[2] 3434
[root@ip-10-194-203-183 ~]# echo "9999999^999999" | bc &
[3] 3436
[root@ip-10-194-203-183 ~]# echo "9999999^999999" | bc &
[4] 3438

See the current no. of instances,

See the CPU load after 2 minute over putty,

A new instance is initializing and running after one minute,

So Autoscaling worked like a charm Hurrah !!! 🙂 Now I’m going to reduce the CPU load by killing the command.

[root@ip-10-194-203-183 ~] killall -9 bc

Now the cpu load is getting down…. after 3 minutes later, I found that the one aged instance which had created earlier when scaling group was created is now terminating. See the picture shown below

That’s all completed.

Now you can deploy this in to production mode. Be aware of the data loss and other load balancing session issues. I would higly recommending to use S3 for keeping user uploads/processing data like images/videos etc.

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>