Category: Scripts and codes

Deploy AWS EC2 instances using Ansible

We have seen in the past how to use Terraform to deploy AWS EC2 instance. But, this is also possible using Ansible. In this blog post, we will focus on the deployment of AWS EC2 instance using Ansible. I assume that you have already been to the basics installation of Ansible and basic playbook creation. Here are some links on tunnelix.com on Ansible. Please consider visiting them if you have any doubt. I assume that you have already install Ansible on your machine and know the basics of Ansible Playbook creation.

Setup the AWS IAM Account

1. Start by creating an AWS user account through the AWS IAM. Go to IAM, then click on USER, then click on ADD USER:

2. Once you have click on the ADD USER, Enter a name and tick on PROGRAMMATIC ACCESS, click on NEXT: PERMISSIONS

3. On the following page, Create a group, I have created one called ‘Ansible’ then attached the user to the group. After that click on ATTACH EXISTING POLICIES DIRECTLY’, then search for AMAZONEC2FULLACCESS and tick it and click on NEXT: TAGS.

4. Click on Next, add the tags and click on ‘CREATE USER’.

5. Consider downloading the credential.csv file by clicking on Download .csv

6. Consider also creating a key pair

Some installations and configurations on the Ansible controller

7. Now, on your Linux controller, we will need some Python modules to interact with AWS. Assuming you have already installed Ansible, consider installing python-pip:

yum install python-pip

8.  Let’s now install the AWS CLI

yum install awscli

9. Sync the clock of the VM to prevent any error

hwclock -s

10. Configure your AWS CLI

aws configure

It will prompt you to enter the AWS Access Key ID, secret key, etc. Just enter the information. Example:

[[email protected] ~]# aws configure

AWS Access Key ID [****************GYGY]: AKIA5xxxxxxxxxx
AWS Secret Access Key [****************458q]: EvEd55xxxxxxxxxx
Default region name [us-east-1]:
Default output format [json]: 

11. Create the following file: /home/.boto

[Credentials]
aws_access_key_id = AKIAxxxxxxxxxxxxx
aws_secret_access_key = xc3xxxxxxxxxxx

12. The following command should test your AWS credentials

aws get sts-caller-identity

13. Install the boto Python module. The boto python module will talk with the AWS CLI to authenticate on aws.

pip install boto 

Creating the Playbook and Deploying the AWS EC2 Instance

14. Now, let’s create a playbook as follows in /home/AWSTask.yml

- name: EC2 Instance creation
  hosts: localhost
  connection: local
  tasks:
  - name: Launching the EC2 instance
    ec2: 
      instance_type: t2.nano
      key_name: ansible
      image: ami-0b69ea66ff7391e80
      region: us-east-1
      group: default
      count: 1
      vpc_subnet_id: subnet-ef9179a4
      wait: yes
      assign_public_ip: yes

You can also access it on my Ansible Github repository.

15. Simply launch the Playbook

ansible-playbook AWSTask.yml

16. As you can see below, the EC2 instance has been created.


Getting started with Terraform

Terraform is an open-source tool created by HashiCorp and it is written in Go programming language. Using Terraform allows us to define our infrastructure as a Code by using declarative language. It’s important to understand that Terraform language is declarative, which describes an intended goal rather than the steps to reach the goal. Once you define your infrastructure, Terraform will figure out how to create it. Terraform also supports a variety of cloud providers and virtualization platforms such as AWS, Azure, VMware, OpenStack, etc.. This is pretty cool as it eliminates several tasks, for example, to create several AWS instances.

Photo credits: terraform.io
Photo credits: terraform.io

Installation of Terraform

1. This is pretty simple. You just have to go on the official website and download the package. In my case, I am on a Linux machine, and I am choosing a Linux 64 bit package.

To download and unzip it, use the following command:

wget https://releases.hashicorp.com/terraform/0.12.10/terraform_0.12.10_linux_amd64.zip && unzip terraform*.zip

2. I moved the binary to /usr/local/bin. Make sure it is in the path environment variable.

mv terraform /usr/local/bin

3. By this time, you should get your binary and be able to check the version.

terraform version

Setting up API call for Terraform on AWS

4. We also need to allow terraform to make an API call on our behalf. I will be calling the API on AWS. For that, you will need to create a user on the AWS IAM and assign the rights and policies. Assuming that you have already created the user and you have the credentials to move ahead. Use the following commands:

export AWS_ACCESS_KEY_ID="AKIA***************"
export AWS_SECRET_ACCESS_KEY="mVTwU6JtC***************"
export AWS_DEFAULT_REGION="us-east-1"

Writing the codes

5. Once you are done exporting the credentials, you can start building your Terraform code. The whole code is in my Github and you can download it for free.

The first thing is to configure the provider and the region.

provider "aws" {

 region = "us-east-1"

}

6. Each provider supports different kinds of resources such as load balancers, servers, databases, etc.. In this example, we are trying to create a single EC2 instance. I have chosen the AWS Linux OS and the smallest nano server. The tags are just the identifier in AWS.

resource "aws_instance" "web" {

  ami           = "ami-0b69ea66ff7391e80"

  instance_type = "t2.nano"

} 

7. Then launch a terraform init to initialized the Terraform working directory.  By that, I mean that it will download the AWS plugin. You should found a similar type of output from your screen.

8. Before performing the actual change, you can use the terraform plan to understand what change has been established. The plus sign means what is going to be added and the minus sign means those that are going to be removed.

9. To create the instance use the terraform apply to create the instance. It will prompt you to type ‘yes’ to continue on with the creation.

10. If you go on the AWS EC2 console, you will notice that the resource has been created successfully.

11. Hey, it’s not over yet! There are more things that need to be added for example the name of the instance. Let’s called it Nginx-Server. Let’s add the tags. Also, launch a terraform apply.

tags = {

    Name = "Nginx-Web"

 }

Adding User Data and Security groups

12. At this stage, I believed you must understand what is Terraform and how it works? To make the installation of Nginx add the following block of lines:

user_data = <<-EOF

  #!/bin/bash

  yum install nginx -y

  systemctl start nginx

  systemctl enable nginx

  EOF

13. To add the security groups, enter these codes:

resource "aws_security_group" "allow_http" {

  name        = "allow_http"

  description = "Allow HTTP inbound traffic"

  ingress {

    from_port   = 80

    to_port     = 80

    protocol    = "tcp"

    cidr_blocks = ["0.0.0.0/0"]

  }

14. In part 6 under instance_type, I have added this line. What it means? “aws_security_group” is a resource, “allow_http” is a variable that has been called from the security group in part 13, and lastly “id” is the attribute.

  vpc_security_group_ids = ["${aws_security_group.allow_http.id}"]

15. Note that when launching terraform apply, you will notice that Terraform will destroy the old machine and build a new one which implies that there will be a downtime.

16. You can also view your code through a graph. Launch the command terraform graph. The output can also be viewed as more human-readable through Graphviz which you have to install. You can also go to webgraphviz.com to view it online.

It is very interesting to understand the dependency when using declarative language in Terraform. The full code can be viewed here on my Github Repository.



Chef workstation and a basic cookbook

Since the main jobs of system administrator is to maintain systems, keep repeating ourselves which is kind boring as well as to dig into our memory of previous configurations that we have set up on a machine. No wonder, manual consistency configurations need to be checked on server configurations. It can be thousands of machines. Chef, is just another tool to get rid of these situations. It is a configuration management tool which is written in Ruby and Erlang for IT professional. Compared to Puppet which has only the Workstation and the Derver whilst Chef has three components that are the Chef Server, Chef workstation and Chef Node.

Photo credits: Linode.com
Photo credits: Linode.com

The cookbooks are written on the Workstation, and its then uploaded to the Chef server (service) which will be executed on the nodes. Chef nodes can be physical, virtual or directly on the cloud. Normally, chef nodes cannot communicate directly to the workstation. Let’s not focus on the installation.

Let’s first get into the workstation.

1.On the workstation download and install the Chef client from the client download page. In my case, i am on a Centos7 virtual machine.

[[email protected] ~]# wget https://packages.chef.io/stable/el/7/chef-12.12.15-1.el7.x86_64.rpm

2.After installation, you should notice the four utils already available: chef-apply chef-client chef-shell chef-solo

3. Now, we are going to create a cookbook. Since chef use the DSL – Domain specific language, the file created should end with the extension .rb Here is an example called file.rb. The first line means file resource which means a file is being created. The file resource will manage a file on the machine. The content of the file will be created with the line ‘Hello Tunnelix’

file 'file.txt' do
            content 'Hello Tunnelix'
 end

4. The tool chef-apply can be used to run it as follows:

Screenshot from 2016-08-07 21-49-07

5. You will also noticed that the file.txt has been created in the current directory as the path has not been specified.

Screenshot from 2016-08-07 21-50-24

Tips:

  • If the content of file.rb (refer to point 3) has not been modified and you fire a chef-apply again, you would notice a prompt that its already ‘up to date’ which means that it reduce the disk IO as well as the bandwidth. 
  • A string must be enclosed  in double quotes when using variables. You cannot use a single quote into another single quote. It won’t work!

Chef always check and refer to the resource and attributes in the cookbook to execute an order ; ie to cook a food. The thing is that Chef focus on the DSL with the aim to what the modifications need to be. Chef allows servers to be in a consistent state.


Running a server using Django

Running a server using Django is pretty easy. Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of Web development, so you can focus on writing your app without needing to reinvent the wheel. It’s free and open source. – Django.

Screenshot from 2016-04-23 18-59-12

1.To start with you will need to have Python which is already there by default on your linux machine. You will need to install Pip with the command. As I am on a Ubuntu machine im using the following commands.

apt-get install python-pip 

2. I am also running komodo-edit as editor. You easily install it with the following commands

add-apt-repository ppa:mystic-mirage/komodo-edit
apt-get update
apt-get install komodo-edit

3. Then, you will need to install django.

pip install django

4. Point yourself to a directory where you want to create your Django projects with the following commands. You would need to create a project. Here my project name is linuxproject. You would notice that a directory called linuxproject is created.

django-admin.py linuxproject

5. Get into the project directory. Here in my case its linuxproject. You should notice a manage.py file and a directory containing the project.

6. Now, open Komodo-edit. Create a new project in the same directory where your project has been created. Save the Django project with Komodo. It should looks something like this:

Screenshot from 2016-04-23 22-01-27

Save the project. launch the following command from terminal.

python manage.py migrate

Now, you can start your server with the following command

python manage.py runserver

You can now navigate to http://127.0.0.1:8000 In future articles, I will get into details of Django.

 

Tips:

  • You can also perform offline installation by downloading the Django Package and launch the following command for installation: python setup.py install
  • To create a password for the admin interface use the following command: python manage.py createsuperuser
  • In case you have encountered the “Invalid HTTP_HOST header error, you will need to add your IP in the settings.py file. Example is ALLOWED_HOSTS = [‘172.10.10.1’, ‘localhost’, ‘127.0.0.1’]

Starting up with Python – Part 16 – Last Part

Here we are with the last article “Starting up with Python”. This one will shed some ideas on Working with files, Reading and Writing and Writing lines in Python. You can view all the past Python articles here.

python_logo_by_bluex_pl

39. Working with Files

To write to a file is pretty simple. I have the file toto which is blank at /python/toto. After using the close function the phrase hey hackers mauritius is written to toto.

>> fileop=open('/python/toto','w')
 >>> fileop.write('hey hackers mauritius')
 >>> fileop.close()

To read from the file do the following. The number 3 here means 3 byte

>> fileop=open('/python/toto','r')
 >>> fileop.read(3)
 'hey'

If you want to read the whole file, just put nothing inside the read function. Its important to close the function to prevent memory leaks.

>> fileop.read()
 ' hackers mauritius

40. Reading and Writing

Let’s not read line by line. Here is how you read a whole line

>> fileop=open('/python/toto','r')
>>> print fileop.readline()
hey hackers mauritius

You can also read line by line and put it into a list

>> fileop=open('/python/toto','r')
>>> print fileop.readlines()
['hey hackers mauritius\n', 'how are you doing\n', 'what are the new projects?\n']

To create a new file or overwrite a file with some lines do this

>> fileop=open('/python/toto','w')
>>> fileop.write('This is a new LINE\n')
>>> fileop.close()

41. Writing Lines

Let’s now create a list and store it into a temporary variable called listtest. listtest is now a list of all lines in toto.

>> fileop=open('/python/toto','r')
>>> listtest=fileop.readlines()
>>> listtest
['This is a new LINE\n']
>>> fileop.close()

We can also modify the same list. Here [1] is the second line as it starts with 0

>> filelist=open('/python/toto','r')
>>> listtest=filelist.readlines()
>>> listtest
['This is a new LINE\n', 'this is another line\n']
>>> filelist.close()
>>> listtest[1]="this is second line"
>>> listtest
['This is a new LINE\n', 'this is second line']

However, this has not been saved to the file. Let’s see how to save it.

>> filelist=open('/python/toto','w')
>>> filelist.writelines(listtest)
>>> filelist.close()