Launch AWS instance and attach volume using AWS CLI & script
Goals or steps:
- Create a key pair
- Create a security group
- Launch an instance using the above created key pair and security group.
- Create an EBS volume of 1 GB.
- The final step is to attach the above created EBS volume to the instance you created in the previous steps.
All the above steps must be performed using AWS CLI
Create a script to automate most of the things.
Let’s see some prerequisites
For doing this task, you need to know about AWS public Cloud, AWS CLI program, AWS ec2 service, etc. So, let’s take the above things first:
What is the AWS CLI?
The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts. For more info: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
What is AWS EC2 Service?
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud. For more info: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html
What is Amazon EBS (Elastic Block Storage)?
Amazon Elastic Block Store (Amazon EBS) provides block-level storage volumes for use with EC2 instances. For more info: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html
Now, Let’s jump to the project:
Note: Github repo where the code is available: https://github.com/theadarshsaxena/AWS-CLI-Scripting/
Part I of the project: Doing it manually
Step 1: Creating the key pairs from the command line
To create the key-pair from the AWS CLI, we will be using the AWS command. Key-pair comes under the umbrella of EC2 services. You can also check all the commands and services under AWS EC2 using the following command:
aws ec2 helpFor creating the key-pair, use the following command:
aws ec2 create-key-pair --key-name myclikey --tag-specification ResourceType=key-pair,Tags=[{Key=Name,Value=myclikey}] --query "KeyMaterial" --output text > myclikey.pemIn the above command, we have used the — query option with “KeyMaterial”, this will give only the private key content output, which we are redirecting to a file named myclikey.pem. This will create a private key and save that in that file.
For confirming, either you can check from the AWS Web Console or Command line using the following command:
aws ec2 describe-key-pairs
Step 2: Creating the security group
Security groups in the AWS act as the virtual firewall and help secure the instance or other resources that you have created in the AWS.
For creating the security group, you can use the following command:
aws ec2 create-security-group --group-name mycligrp --description "created from CLI" --vpc-id vpc-50f0e456The security group also comes under the umbrella of ec2, hence, using the AWS ec2 here as well. There is a number of other options available with this command while creating the security group. To see other options, you can use the following command:
aws ec2 create-security-group helpFor confirming that your security group is created successfully or not, you can use Web Console of AWS or the following command from CLI
aws ec2 describe-security-groups
Step 3: Add the inbound rules to the security group
For allowing the specific IP or allowing the specific port connection from the outside world, you need to add the inbound rules in the security group. For instance, for allowing everyone from the outside world to be able to access the resource, but only in port 22 (which is used for ssh), you can run the following command:
aws ec2 authorize-security-group-ingress --group-id sg-0186840bb87671260 --protocol tcp --port 22 --cidr 0.0.0.0/0Note: ingress rules are the ones that control the inbound traffic (which is coming from the outside world to your instance).
You can check, whether the above command worked properly or not by describing the security groups.
aws ec2 describe-security-groupsStep 4: Launching the EC2 instance
For launching the EC2 instance, you can use the following command:
aws ec2 run-instances --image-id ami-0e306788ff2473ccb --count 1 --instance-type t2.micro --key-name myclikey --security-group-ids sg-0186840bb87671260In the above command, while launching the ec2 instance, you have to specify some of the things, like the number of instances, image-id, instance-type, and the key which you want to use, and the security group that you want to attach.
Step 5: Creating the EBS volume
To create the EBS volume, you can make use of the create-volume command available in AWS CLI. Along with this command, you need to provide the availability zone that you want to use. So, here, you must need to specify the same availability zone as that of your instance’s AZ because only then, you will be able to attach the volume to the instance.
aws ec2 create-volume --availability-zone ap-south-1a --size 1 --tag-specification ResourceType=volume,Tags=[{Key=Name,Value=clivolume}]
Step 6: Attaching that EBS Volume to the EC2 instance that we have already created
For attaching the volume, you need the device name and instance-id to which you need to attach the volume and obviously the volume-id of the volume that is to be attached to the instance.
Now, to get the volume-id, you can make use of the following command:
aws ec2 describe-volumes --query "Volumes[*].{ID:VolumeId,AZ:AvailabilityZone,Size:Size}" --filters "Name=tag:Name,Values=clivolume"
For getting the instance-id, you can create a command for getting the instance id using the query and filters. Though you can also use the simple command, but that won’t help us in automation, that we’ll do in the latter part of this article.
aws ec2 attach-volume --device /dev/xvdb --instance-id i-0159851f2556c1a04 --volume-id vol-0536d4b73a5937242Part II — Automating the things using scripting
Tip: To read the full code at once, go to the end of the article
For automating the above things, means for creating those resources with a faster pace and no manual involvement, you just need to do a few more things.
First thing, you need to notice that the above commands that we have discussed above are for the Windows Command or PowerShell, and if you try to run those commands in Linux, some of those will give some error. For automating, we are going to write the bash script, and for that, you need a bash shell, which you can get either from any Linux Distro or even Git Bash.
So, Let’s start the scripting part
The first thing, you need to do is to declare the variables for the tags or the name of resources.
#!/bin/bash#Declaring Variableskeyname=testingclikey
instancename=testinginstance
sgname=testingsg
ebsname=testingebs
Now, let’s say you want to create the key-pair from the bash. But, here, you need to note that if you are going to give tag to the key-pair, as we have done in the command at the start, it will give an error. This is because we have used the curly braces in that command, {}, which are intended for other things in Bash. So, to tackle this error, you can use double quote “” as done in command below:
#creating the key pairaws ec2 create-key-pair --key-name $keyname --tag-specification "ResourceType=key-pair,Tags=[{Key=Name,Value=$keyname}]" --query "KeyMaterial" --output text > $keyname.pem
Similarly, you can create the security group:
#Creating the security groupaws ec2 create-security-group --group-name $sgname --description "created from CLI" --vpc-id vpc-50f0e456
Now, the heart of Scripting
Till now, everything was really fine, but when you are going to create the inbound rule for the security group, you need to retrieve the group-id of the security group that you have created and then pass that group-id to another command.
So, for getting the security group info, we can use the aws ec2 describe-security-groups command. But that will show you the info about all the SGs and in json format. So, we need to format that output, then, pass to other commands. For that you can use the following command:
aws ec2 describe-security-groups --query "SecurityGroups[].GroupId" --filters Name=group-name,Values=$sgname
If you have noticed, that the above command is in windows and the output is not what we require, so, now switch to Linux, and let’s do some change in the output.
Now, the output is in the shape of what we require, so, instead of printing that in Console, let’s store that in variable sgid. Here, for formatting the output we are using the sed and tr. See command below:
sgid=`aws ec2 describe-security-groups --query "SecurityGroups[].GroupId" --filters "Name=group-name,Values=$sgname" | sed -n 2p | tr -d \"`Now, we have the security group id stored in sgid variable. So, let’s use this and create inbound rules.
#Adding the inbound rules to the security group createdaws ec2 authorize-security-group-ingress --group-id $sgid --protocol tcp --port 22 --cidr 0.0.0.0/0
For launching the instance, the command is the same as discussed previously except that we are now using the variables inside the command:
#Launching the instanceaws ec2 run-instances --image-id ami-0e306788ff2473ccb --count 1 --instance-type t2.micro --key-name $keyname --security-group-ids $sgid --tag-specification "ResourceType=instance,Tags=[{Key=Name,Value=$instancename}]"
For Creating the volume, not much difference except using double quotes and variable in place of the name of the volume.
#Creating the Volume
aws ec2 create-volume --availability-zone ap-south-1a --size 1 --tag-specification "ResourceType=volume,Tags=[{Key=Name,Value=$ebsname}]"Now, for attaching the instance volume, you need the volume id and the instance id and then, pass the values to another command. To achieve, we have used the following commands:
#storing the value of Volume ID in variable volumeID
volumeID=`aws ec2 describe-volumes --query "Volumes[*].VolumeId" --filters "Name=tag:Name,Values=$ebsname" | sed -n 2p | tr -d \"`#Storing the value of instance ID in the Variable instanceid
instanceID=`aws ec2 describe-instances --query "Reservations[*].Instances[].InstanceId" --filter "Name=key-name,Values=$keyname" | sed -n 2p | tr -d \"`#Attaching the volume to the instance
aws ec2 attach-volume --device /dev/xvdb --instance-id $instanceID --volume-id $volumeID
So, Let’s sum up everything into one script file:
GitHub Repo: https://github.com/theadarshsaxena/AWS-CLI-Scripting/
