Category: Virtualisation

Getting started with Terraform

Terraform is an open-source tool created by HashiCorp and it is written in Go programming language. Using Terraform allows us to define our infrastructure as a Code by using declarative language. It’s important to understand that Terraform language is declarative, which describes an intended goal rather than the steps to reach the goal. Once you define your infrastructure, Terraform will figure out how to create it. Terraform also supports a variety of cloud providers and virtualization platforms such as AWS, Azure, VMware, OpenStack, etc.. This is pretty cool as it eliminates several tasks, for example, to create several AWS instances.

Photo credits: terraform.io
Photo credits: terraform.io

Installation of Terraform

1. This is pretty simple. You just have to go on the official website and download the package. In my case, I am on a Linux machine, and I am choosing a Linux 64 bit package.

To download and unzip it, use the following command:

wget https://releases.hashicorp.com/terraform/0.12.10/terraform_0.12.10_linux_amd64.zip && unzip terraform*.zip

2. I moved the binary to /usr/local/bin. Make sure it is in the path environment variable.

mv terraform /usr/local/bin

3. By this time, you should get your binary and be able to check the version.

terraform version

Setting up API call for Terraform on AWS

4. We also need to allow terraform to make an API call on our behalf. I will be calling the API on AWS. For that, you will need to create a user on the AWS IAM and assign the rights and policies. Assuming that you have already created the user and you have the credentials to move ahead. Use the following commands:

export AWS_ACCESS_KEY_ID="AKIA***************"
export AWS_SECRET_ACCESS_KEY="mVTwU6JtC***************"
export AWS_DEFAULT_REGION="us-east-1"

Writing the codes

5. Once you are done exporting the credentials, you can start building your Terraform code. The whole code is in my Github and you can download it for free.

The first thing is to configure the provider and the region.

provider "aws" {

 region = "us-east-1"

}

6. Each provider supports different kinds of resources such as load balancers, servers, databases, etc.. In this example, we are trying to create a single EC2 instance. I have chosen the AWS Linux OS and the smallest nano server. The tags are just the identifier in AWS.

resource "aws_instance" "web" {

  ami           = "ami-0b69ea66ff7391e80"

  instance_type = "t2.nano"

} 

7. Then launch a terraform init to initialized the Terraform working directory.  By that, I mean that it will download the AWS plugin. You should found a similar type of output from your screen.

Getting started with Terraform 1

8. Before performing the actual change, you can use the terraform plan to understand what change has been established. The plus sign means what is going to be added and the minus sign means those that are going to be removed.

9. To create the instance use the terraform apply to create the instance. It will prompt you to type ‘yes’ to continue on with the creation.

Getting started with Terraform 2

10. If you go on the AWS EC2 console, you will notice that the resource has been created successfully.

Getting started with Terraform 3

11. Hey, it’s not over yet! There are more things that need to be added for example the name of the instance. Let’s called it Nginx-Server. Let’s add the tags. Also, launch a terraform apply.

tags = {

    Name = "Nginx-Web"

 }

Adding User Data and Security groups

12. At this stage, I believed you must understand what is Terraform and how it works? To make the installation of Nginx add the following block of lines:

user_data = <<-EOF

  #!/bin/bash

  yum install nginx -y

  systemctl start nginx

  systemctl enable nginx

  EOF

13. To add the security groups, enter these codes:

resource "aws_security_group" "allow_http" {

  name        = "allow_http"

  description = "Allow HTTP inbound traffic"

  ingress {

    from_port   = 80

    to_port     = 80

    protocol    = "tcp"

    cidr_blocks = ["0.0.0.0/0"]

  }

14. In part 6 under instance_type, I have added this line. What it means? “aws_security_group” is a resource, “allow_http” is a variable that has been called from the security group in part 13, and lastly “id” is the attribute.

  vpc_security_group_ids = ["${aws_security_group.allow_http.id}"]

15. Note that when launching terraform apply, you will notice that Terraform will destroy the old machine and build a new one which implies that there will be a downtime.

16. You can also view your code through a graph. Launch the command terraform graph. The output can also be viewed as more human-readable through Graphviz which you have to install. You can also go to webgraphviz.com to view it online.

Getting started with Terraform 4

It is very interesting to understand the dependency when using declarative language in Terraform. The full code can be viewed here on my Github Repository.


Getting started with Terraform 5


Upgrading firmware of HP ProLiant Gen9 on VMware infrastructure

Some days back, I was having a conversation with a friend about the recent CVEs that impact the firmware of several physical machines. HP usually will release several vulnerability alerts and it is very important to patch your firmware. You might also notice that the mitigation happens at CPU levels such as AMD or Intel. In this blog post, I will focus on one of the very basic ways to update your firmware. Prior before upgrading, its important to make a checklist. My HP Proliant Gen9  is actually an ESXi on VMware infrastructure. You can view the step by step actions below and pause the video at any time. I have blurred some information for security purpose such as the name of servers, IPs, Logins, etc..

Upgrading firmware of HP ProLiant Gen9 on VMware infrastructure 6

A basic checklist can be considered as follows:

  • Monitoring consideration.
  • How many and size of the VMs on production.
  • The consistency of the Firmware provided by HP.
  • Logins and Passwords for the HP ILO, vCenter, Virtual Machines running, etc..
  • Java or Dot Net framework for accessing the ILO.
  • Where is the storage node of the virtual machines?
  • Load on the cluster or the ESX itself.
  • The expected amount of time during migration.
  • The output of the update (Correction of bugs, New feature, etc..).

1. Prior before upgrading the firmware, you need to make a survey about the oversized VMs. Consider performing a manual migration before activating the maintenance mode. Then, enter maintenance mode, all the virtual machines in the cluster should migrate to other physical machines in the cluster. We assume that the datastore of each machine is not on the physical machine itself which is not recommended.

2. Once all virtual machines migrated to other ESXi hosts, connect to the HP ILO onboard administrator interface. Consider checking the health status of other ESXi hosts on your chassis. 

3. Also consider, verifying the system information of your ESXi host (HP physical machine). 

4. On the ‘information’ tab, click on ‘system information’, you will notice the field ‘Integrated remote console’. You can choose any framework whether Java or .Net to open the console.

5. Once connected, you should be able to see the following screen.

ESXI Console
ESXI Console

6. From vCenter, upload the image file which constitutes of the patch for the new firmware.

7. On vCenter, right click on the physical machine, then ‘reboot’. Consider checking the grey bar that is now blinking on the left just below the ‘<F2> tag ‘at the bottom on the ESXi console.

ESXI-Console-blinking
ESXi-Console-blinking

8. By the time, you should also notice that you have been logout on vCenter.

9. Normally, after a few minutes, the server will reboot showing the HP Enterprise logo followed by other system information, then you will notice a screen that with four key options below: F9 (System Utilities), F10 (Intelligent Provisioning), F11 (Boot Menu), and F12 (Network Boot). Hit the F9 button to enter ‘System Utilities’.

10. Choose the ‘one-time boot menu’ option. Then go to the USB virtual disk that you have mounted at step 6 and hit ‘Enter’.

11. By now you should notice the installation of the firmware in progress. This might take a considerate amount of time. Monitor the installation.

12. After extracting the iso file, it will go through three steps: Inventory, Review, and Deployment.

13. You can also monitor for the ‘blink’ message on the HP Onboard Administrator interface which means that the upgrade has not completed yet.

14. Once, the UID state is off, you can remove the server from maintenance. On vCenter, right click on the server, and click on ‘Exit maintenance mode’.

15. Several machines will now join the ESXi host which has been added back to the cluster through an election process.

If you are interested more on the election process and how High Availability works, please check the article ‘VMware vSphere High Availability‘ which I published several months back.

All steps from 1 to 15 have been described in the video below. If you like the article please click on the like button and share.


Attending AWSome day online conference 2019

The AWSome day was a free online Conference and a training event sponsor by Intel that will provide a step-by-step introduction to the core AWS (Amazon Web Services) services. Its free and everyone can attend. It was scheduled on 26 March 2019 online. The agenda covered broad topics such as AWS Cloud Concepts, AWS Core Services, AWS Security, AWS Architecting and AWS Pricing and Support. It’s pretty interesting for IT manager, system engineers, system administrators, and architects who are eager to learn more about cloud computing and how to get started on the AWS cloud. I do have some experience in managing AWS servers and even host my own server. However, I registered for the free training to refresh my knowledge and get more exposure such as the AWS pricing which I am not aware at all. Another interesting thing is that you will receive a certificate of attendance and you received 25 USD of AWS credits. Pretty cool right?

Attending AWSome day online conference 2019 7

Right from the beginning, I knew this was something interesting. I encountered a minor problem whilst signing in. I had to send a mail to support and it was resolved immediately. Once connected to the lobby, it was pretty easy to attend and follow the online conference. After some minutes, Steven Bryen, head in the AWS Cloud delivered the keynote speech.

Attending AWSome day online conference 2019 8

There was also an online challenge and I score 25,821 on the Trivia Leaderboard.

Attending AWSome day online conference 2019 9

On the “Ask an Expert” tab, I was mostly interested in Man on the Side Attack – MOTS attack. They referred me to the WAF section on AWS. Another interesting link is the whitepaper of the AWS Overview of Security guidelines. AWS also offers comprehensive security across all the layers, SSL, DDoS, Firewall, HSM and Networking. I also shoot some question on Metric and Monitoring on application level such as on MariaDB. I discovered about the RDS performance insight. For applications on EC2, Containers, and Lamda, X-Ray looks very promising. Apart from virtualization, its good to note that AWS also provides containerization services.

The event was pretty enriching. The panel on the question area knows well their subject. I discovered a lot by participating in the AWSomeDay. I’m looking forward to AWS certifications in the near future.


VMware vSphere High Availability Basics

VMware vSphere HA is one of the core feature in a cluster. So let’s bring some more precision about it. High Availability – HA enables a cluster of ESXi hosts to work together so that they can provide high levels of High Availability for virtual machines rather than just an ESXi host by itself. In brief, the High Availability feature is provided by pooling virtual machines and the ESXi hosts in the cluster for protection. Some examples could be host failures, host isolations and application crashes. The requirements for HA is a minimum of two hosts, vCenter Server and Shared Storage.

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

Photo Credits: VMware.com
Photo Credits: VMware.com

One ESXi goes down

By default, HA uses management network (Service Console/Management Network VMkernel connections). Let’s take a scenario where there are three ESXi hosts in a cluster. In the event where a physical server (ESXi hosts) goes down, the VM machines will be restarted on the other ESXi hosts. We can also set up applications to be started on the other physical server. From the three physical servers in the cluster one is going to be elected as master. The master server is going to keep track of other ESXi hosts through the heartbeat of other servers. This is done at the management network level. The master server will always expect to have heartbeat responses from other ESXi hosts.

Only the management network went down

If at any moment, the master server detects that a host is down, it will report that to the vCenter server and all servers will be powered on the other ESXi hosts. What is more interesting is that if only the management network goes down, and other network such the datastore network is still working, that would be referred as an Isolation incident. In that case, the vSphere will communicate to the master server and will claim that the ESXi host is still active is through the datastore heartbeat. In that case, the VMs will not be powered onto other ESXi host because it is an Isolation incident.

Only the Datastore network went down

Now, what if only the Datastore network went down and not the Management network? The master server will still receive heartbeat messages from other ESXi hosts, but no data communication is being sent to the datastore. Another element that is included in HA is VMCP – VM Component Protection which is a component that detects that if a VM is having access to the datastore. In the event of failure messages from the datastore heartbeat, the VMs will be powered onto other ESXi hosts where the datastore is sending alive heartbeat messages.

In all three scenarios, HA implies downtime as servers will be restarted in other ESXi hosts, but same is usually done within minutes. Another point to keep in mind is that HA applies only to physical host. For example, if a particular VM encounter a BSOD or Kernel Panic, HA will not know about it because the Physical server (ESXi host) is still communicating with the master server.

How the election process takes place to become the master?

When HA gets activated in the vSphere, the election process takes around 10-15 seconds. In that process (Enabling HA) an agent gets installed to activate HA which is called FDM – Fault Domain manager. Logs can be checked at /var/log/fdm.log. The election process is defined by an algorithm with two rules. For the first, the host with access to the greatest number of datastores wins.

Now, what if all ESXi hosts see the same number of datastores ? There will be a clash. This is where the second rule kicks in i.e; the host with the lexically-highest Managed Object ID (MOID) is chosen. Note that in vCenter Server each object will have a MOID. For example, objects are ESXI servers, folders, VMs etc.. So the lexical analyzer is a first component where it takes a character stream as input, outputs a token which goes into a syntax analyzer and the lexical analysis is performed. Care must be taken when attempting to rig this election because lexically here means, for example, that host-99 is in fact higher than host-100.

What IF …. ?

 

So what if vCenter Server goes down after setting up HA? 

The answer is HA will still work as it now the capacity to power on the vCenter Server. FDMs are self sufficient to carry on the election process as well as to start the vCenter Server. FDMs are inside the VMs but not inside the vCenter Server.

Enable and Configure vSphere HA
 
I will be using the free labs provided by VMware to set up HA.
 
1.The first action is to choose the Cluster then click on ‘Actions‘  then ‘Settings‘.
 
Photo Credits: VMware.com
Photo Credits: VMware.com

2. Choose ‘vSphere Availability‘ on the left -> then click on ‘Edit‘.

Photo Credits: VMware.com
Photo Credits: VMware.com

3. Click on ‘Turn ON vSphere HA’.

Photo Credits: VMware.com
Photo Credits: VMware.com

4. Choose ‘Failures and Responses‘ option and click on -> and enable ‘VM and Application monitoring‘.

Photo Credits: VMware.com
Photo Credits: VMware.com

5. On the ‘Admission control‘ -> check the ‘Cluster resource percentage‘ option.

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

Photo Credits: VMware.com
Photo Credits: VMware.com

6. Click on ‘Heartbeat Datastores’ and select ‘Automatically select datastores accessible form the host‘.

Photo Credits: VMware.com
Photo Credits: VMware.com
7. From the ‘Summary’ tab click on ‘vSphere Availability‘, it should mentioned vSphere HA: Protected.
 
Photo Credits: VMware.com
Photo Credits: VMware.com
 
 
REFERENCES and CREDITS:
1.VMware Tech Plus:
2.VMware White paper:
3.VMware Labs:
4.Other Links:
 

ESXi installation on my Dell Laptop and hands on VMware Labs

If you are thinking why i should install a bare metal hypervisor on a laptop, i assure you its just for educational and testing purpose only. I noticed that it was quite difficult for me to get this done. However, after some research it looks that my Dell Inspiron n5110 motherboard will not authorised me to install ESXi 6.x. Probably, it looks like there are some drivers missing or the motherboard does not support it.

Here is what my processors looks like from the configuration menu on VMware vSphere Center

ESXi installation on my Dell Laptop and hands on VMware Labs 10

Anyway, i have been able to inject some network drivers – VIB files into the ESXi5.0 which allowed me to install the ESXi 5.0 on the laptop. You can follow the instructions at the link how to make your unsupported NIC work with ESXi. Once installed, VMware will provide you with a two months free trial before you purchase the license.

Another way of messing around VMware Vsphere is to deploy a lab from labs.hol.vmware.com That’s so easy to deploy labs and access the VMware vSphere web client. All credentials will be available on the readme.txt file found on the desktop. Also a lab manual will be shown alongside whilst working on the environement labs.

ESXi installation on my Dell Laptop and hands on VMware Labs 11

I am sure this would help anyone to get into hands on lab quickly and it would be a nice start for beginners.