Category: Scripts and codes

Getting started with Terraform

Terraform is an open-source tool created by HashiCorp and it is written in Go programming language. Using Terraform allows us to define our infrastructure as a Code by using declarative language. It’s important to understand that Terraform language is declarative, which describes an intended goal rather than the steps to reach the goal. Once you define your infrastructure, Terraform will figure out how to create it. Terraform also supports a variety of cloud providers and virtualization platforms such as AWS, Azure, VMware, OpenStack, etc.. This is pretty cool as it eliminates several tasks, for example, to create several AWS instances.

Photo credits: terraform.io
Photo credits: terraform.io

Installation of Terraform

1. This is pretty simple. You just have to go on the official website and download the package. In my case, I am on a Linux machine, and I am choosing a Linux 64 bit package.

To download and unzip it, use the following command:

wget https://releases.hashicorp.com/terraform/0.12.10/terraform_0.12.10_linux_amd64.zip && unzip terraform*.zip

2. I moved the binary to /usr/local/bin. Make sure it is in the path environment variable.

mv terraform /usr/local/bin

3. By this time, you should get your binary and be able to check the version.

terraform version

Setting up API call for Terraform on AWS

4. We also need to allow terraform to make an API call on our behalf. I will be calling the API on AWS. For that, you will need to create a user on the AWS IAM and assign the rights and policies. Assuming that you have already created the user and you have the credentials to move ahead. Use the following commands:

export AWS_ACCESS_KEY_ID="AKIA***************"
export AWS_SECRET_ACCESS_KEY="mVTwU6JtC***************"
export AWS_DEFAULT_REGION="us-east-1"

Writing the codes

5. Once you are done exporting the credentials, you can start building your Terraform code. The whole code is in my Github and you can download it for free.

The first thing is to configure the provider and the region.

provider "aws" {

 region = "us-east-1"

}

6. Each provider supports different kinds of resources such as load balancers, servers, databases, etc.. In this example, we are trying to create a single EC2 instance. I have chosen the AWS Linux OS and the smallest nano server. The tags are just the identifier in AWS.

resource "aws_instance" "web" {

  ami           = "ami-0b69ea66ff7391e80"

  instance_type = "t2.nano"

} 

7. Then launch a terraform init to initialized the Terraform working directory.  By that, I mean that it will download the AWS plugin. You should found a similar type of output from your screen.

Getting started with Terraform 1

8. Before performing the actual change, you can use the terraform plan to understand what change has been established. The plus sign means what is going to be added and the minus sign means those that are going to be removed.

9. To create the instance use the terraform apply to create the instance. It will prompt you to type ‘yes’ to continue on with the creation.

Getting started with Terraform 2

10. If you go on the AWS EC2 console, you will notice that the resource has been created successfully.

Getting started with Terraform 3

11. Hey, it’s not over yet! There are more things that need to be added for example the name of the instance. Let’s called it Nginx-Server. Let’s add the tags. Also, launch a terraform apply.

tags = {

    Name = "Nginx-Web"

 }

Adding User Data and Security groups

12. At this stage, I believed you must understand what is Terraform and how it works? To make the installation of Nginx add the following block of lines:

user_data = <<-EOF

  #!/bin/bash

  yum install nginx -y

  systemctl start nginx

  systemctl enable nginx

  EOF

13. To add the security groups, enter these codes:

resource "aws_security_group" "allow_http" {

  name        = "allow_http"

  description = "Allow HTTP inbound traffic"

  ingress {

    from_port   = 80

    to_port     = 80

    protocol    = "tcp"

    cidr_blocks = ["0.0.0.0/0"]

  }

14. In part 6 under instance_type, I have added this line. What it means? “aws_security_group” is a resource, “allow_http” is a variable that has been called from the security group in part 13, and lastly “id” is the attribute.

  vpc_security_group_ids = ["${aws_security_group.allow_http.id}"]

15. Note that when launching terraform apply, you will notice that Terraform will destroy the old machine and build a new one which implies that there will be a downtime.

16. You can also view your code through a graph. Launch the command terraform graph. The output can also be viewed as more human-readable through Graphviz which you have to install. You can also go to webgraphviz.com to view it online.

Getting started with Terraform 4

It is very interesting to understand the dependency when using declarative language in Terraform. The full code can be viewed here on my Github Repository.


Getting started with Terraform 5


Chef workstation and a basic cookbook

Since the main jobs of system administrator is to maintain systems, keep repeating ourselves which is kind boring as well as to dig into our memory of previous configurations that we have set up on a machine. No wonder, manual consistency configurations need to be checked on server configurations. It can be thousands of machines. Chef, is just another tool to get rid of these situations. It is a configuration management tool which is written in Ruby and Erlang for IT professional. Compared to Puppet which has only the Workstation and the Derver whilst Chef has three components that are the Chef Server, Chef workstation and Chef Node.

Photo credits: Linode.com
Photo credits: Linode.com

The cookbooks are written on the Workstation, and its then uploaded to the Chef server (service) which will be executed on the nodes. Chef nodes can be physical, virtual or directly on the cloud. Normally, chef nodes cannot communicate directly to the workstation. Let’s not focus on the installation.

Let’s first get into the workstation.

1.On the workstation download and install the Chef client from the client download page. In my case, i am on a Centos7 virtual machine.

[[email protected] ~]# wget https://packages.chef.io/stable/el/7/chef-12.12.15-1.el7.x86_64.rpm

2.After installation, you should notice the four utils already available: chef-apply chef-client chef-shell chef-solo

3. Now, we are going to create a cookbook. Since chef use the DSL – Domain specific language, the file created should end with the extension .rb Here is an example called file.rb. The first line means file resource which means a file is being created. The file resource will manage a file on the machine. The content of the file will be created with the line ‘Hello Tunnelix’

file 'file.txt' do
            content 'Hello Tunnelix'
 end

4. The tool chef-apply can be used to run it as follows:

Screenshot from 2016-08-07 21-49-07

5. You will also noticed that the file.txt has been created in the current directory as the path has not been specified.

Screenshot from 2016-08-07 21-50-24

Tips:

  • If the content of file.rb (refer to point 3) has not been modified and you fire a chef-apply again, you would notice a prompt that its already ‘up to date’ which means that it reduce the disk IO as well as the bandwidth. 
  • A string must be enclosed  in double quotes when using variables. You cannot use a single quote into another single quote. It won’t work!

Chef always check and refer to the resource and attributes in the cookbook to execute an order ; ie to cook a food. The thing is that Chef focus on the DSL with the aim to what the modifications need to be. Chef allows servers to be in a consistent state.


Running a server using Django

Running a server using Django is pretty easy. Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of Web development, so you can focus on writing your app without needing to reinvent the wheel. It’s free and open source. – Django.

Screenshot from 2016-04-23 18-59-12

1.To start with you will need to have Python which is already there by default on your linux machine. You will need to install Pip with the command. As I am on a Ubuntu machine im using the following commands.

apt-get install python-pip 

2. I am also running komodo-edit as editor. You easily install it with the following commands

add-apt-repository ppa:mystic-mirage/komodo-edit
apt-get update
apt-get install komodo-edit

3. Then, you will need to install django.

pip install django

4. Point yourself to a directory where you want to create your Django projects with the following commands. You would need to create a project. Here my project name is linuxproject. You would notice that a directory called linuxproject is created.

django-admin.py linuxproject

5. Get into the project directory. Here in my case its linuxproject. You should notice a manage.py file and a directory containing the project.

6. Now, open Komodo-edit. Create a new project in the same directory where your project has been created. Save the Django project with Komodo. It should looks something like this:

Screenshot from 2016-04-23 22-01-27

Save the project. launch the following command from terminal.

python manage.py migrate

Now, you can start your server with the following command

python manage.py runserver

You can now navigate to http://127.0.0.1:8000 In future articles, I will get into details of Django.

 

Tips:

  • You can also perform offline installation by downloading the Django Package and launch the following command for installation: python setup.py install
  • To create a password for the admin interface use the following command: python manage.py createsuperuser
  • In case you have encountered the “Invalid HTTP_HOST header error, you will need to add your IP in the settings.py file. Example is ALLOWED_HOSTS = [‘172.10.10.1’, ‘localhost’, ‘127.0.0.1’]

Starting up with Python – Part 16 – Last Part

Here we are with the last article “Starting up with Python”. This one will shed some ideas on Working with files, Reading and Writing and Writing lines in Python. You can view all the past Python articles here.

python_logo_by_bluex_pl

39. Working with Files

To write to a file is pretty simple. I have the file toto which is blank at /python/toto. After using the close function the phrase hey hackers mauritius is written to toto.

>> fileop=open('/python/toto','w')
 >>> fileop.write('hey hackers mauritius')
 >>> fileop.close()

To read from the file do the following. The number 3 here means 3 byte

>> fileop=open('/python/toto','r')
 >>> fileop.read(3)
 'hey'

If you want to read the whole file, just put nothing inside the read function. Its important to close the function to prevent memory leaks.

>> fileop.read()
 ' hackers mauritius

40. Reading and Writing

Let’s not read line by line. Here is how you read a whole line

>> fileop=open('/python/toto','r')
>>> print fileop.readline()
hey hackers mauritius

You can also read line by line and put it into a list

>> fileop=open('/python/toto','r')
>>> print fileop.readlines()
['hey hackers mauritius\n', 'how are you doing\n', 'what are the new projects?\n']

To create a new file or overwrite a file with some lines do this

>> fileop=open('/python/toto','w')
>>> fileop.write('This is a new LINE\n')
>>> fileop.close()

41. Writing Lines

Let’s now create a list and store it into a temporary variable called listtest. listtest is now a list of all lines in toto.

>> fileop=open('/python/toto','r')
>>> listtest=fileop.readlines()
>>> listtest
['This is a new LINE\n']
>>> fileop.close()

We can also modify the same list. Here [1] is the second line as it starts with 0

>> filelist=open('/python/toto','r')
>>> listtest=filelist.readlines()
>>> listtest
['This is a new LINE\n', 'this is another line\n']
>>> filelist.close()
>>> listtest[1]="this is second line"
>>> listtest
['This is a new LINE\n', 'this is second line']

However, this has not been saved to the file. Let’s see how to save it.

>> filelist=open('/python/toto','w')
>>> filelist.writelines(listtest)
>>> filelist.close()

 

 


Starting up with Python – Part15

If you have been following the Python articles since some days, you would have noticed that things are getting more interesting. In case, you have missed the past articles here is a recap. In this article, i will get into Constructors, Import modules, reload modules and Getting modules info.

python_logo_by_bluex_pl

35. Constructors

When an object is created, the methods need to be called but when a contructor is when the first object is created, the objects are automatically called.

Lets see a basic method and object analogy

>> class hackers:
... def mauritius(self):
... print "hackers mauritius"
... 


>> obj=hackers()
>>> obj.mauritius()
hackers mauritius

However, in constructors we want to automatically call the object.

In this example, you would notice a class called new and a method called __init__

>> class new:
... def __init__(self):
... print "hackers mauritius"
... print "this is a constructor"
... 
>>> newobject=new()
hackers mauritius
this is a constructor

36. Import Modules

In python, you can write modules and each time you can call it from anywhere in your code. So let’s create a module.

I created a file called hackers.py with the following simple data.

# cat hackers.py 
def testmod():
 print "this is hackers Mauritius"

This is how you can import different modules imported from a file. Modules can also be imported once per file.

>> import hackers
>>> hackers.testmod()
this is hackers Mauritius

37. Reload modules

However, if the source file is edited and you would import the same module, the result would be the same. To get result from the source file, use this parameter

>> import hackers
>>> hackers.testmod()
this is hackers Mauritius
>>> reload(hackers)
<module 'hackers' from 'hackers.py'>
>>> hackers.testmod()
this is hackers Worldwide

38. Getting modules info

To get modules info, there are in build module. For example, there is an inbuilt module called SQRT

>> import math
>>> math.sqrt(25)
5.0

To know what the module math contains, you can use the dir function

>> dir(math)
['__doc__', '__name__', '__package__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'hypot', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']

There is also the help function. You just need to type this

>> help(math)

You would noticed  a list of what each modules does. However, before using a module you can use this 

>> math.__doc__
'This module is always available. It provides access to the\nmathematical functions defined by the C standard.'

It gives a quick summary of what the module does.