Tag: Python

An agentless servers inventory with Ansible & Ansible-CMDB

Building from scratch an agentless inventory system for Linux servers is a very time-consuming task. To have precise information about your server’s inventory, Ansible comes to be very handy, especially if you are restricted to install an agent on the servers. However, there are some pieces of information that the Ansible’s inventory mechanism cannot retrieve from the default inventory. In this case, a Playbook needs to be created to retrieve those pieces of information. Examples are VMware tool and other application versions which you might want to include in your inventory system. Since Ansible makes it easy to create JSON files, this can be easily manipulated for other interesting tasks, say an HTML static page. I would recommend Ansible-CMDB which is very handy for such conversion. The Ansible-CMDB allows you to create a pure HTML file based on the JSON file that was generated by Ansible. Ansible-CMDB is another amazing tool created by Ferry Boender.


Photo credits: Ansible.com
Photo credits: Ansible.com

Let’s have a look how the agentless servers inventory with Ansible and Ansible-CMDB works. It’s important to understand the prerequisites needed before installing Ansible. There are other articles which I published on Ansible:

Ansible Basics and Pre-requisites

1. In this article, you will get an overview of what Ansible inventory is capable of. Start by gathering the information that you will need for your inventory system. The goal is to make a plan first.

2. As explained in the article Getting started with Ansible deployment, you have to define a group and record the name of your servers(which can be resolved through the host file or DNS server) or IP’s. Let’s assume that the name of the group is “test“.

3. Launch the following command to see a JSON output which will describe the inventory of the machine. As you may notice that Ansible had fetched all the data.


Ansible -m setup test

4. You can also append the output to a specific directory for future use with Ansible-cmdb. I would advise creating a specific directory (I created /home/Ansible-Workdesk) to prevent confusion where the file is appended.

Ansible-m setup --tree out/ test

5. At this point, you will have several files created in a tree format, i.e; specific file with the name of the server containing JSON information about the servers inventory.

Getting Hands-on with Ansible-cmdb

6. Now, you will have to install Ansible-cmdb which is pretty fast and easy. Do make sure that you follow all the requirements before installation:

git clone https://github.com/fboender/ansible-cmdb
cd ansible-cmdb && make install

7. To convert the JSON files into HTML, use the following command:

ansible-cmdb -t html_fancy_split out/

8. You should notice a directory called “cmdb” which contain some HTML files. Open the index.html and view your server inventory system.

Tweaking the default template

9. As mentioned previously, there is some information which is not available by default on the index.html template. You can tweak the /usr/local/lib/ansible-cmdb/ansiblecmdb/data/tpl/html_fancy_defs.html page and add more content, for example, ‘uptime‘ of the servers. To make the “Uptime” column visible, add the following line in the “Column definitions” section:


{"title": "Uptime",        "id": "uptime",        "func": col_uptime,         "sType": "string", "visible": True},

Also, add the following lines in the “Column functions” section :

<%def name="col_uptime(host, **kwargs)">
${jsonxs(host, 'ansible_facts.uptime', default='')}
</%def>

Whatever comes after the dot just after ansible_fact.<xxx> is the parent value in the JSON file. Repeat step 7. Here is how the end result looks like.

Photo credits: Ferry Boender
Photo credits: Ferry Boender

Getting beyond Ansible-cmdb

Now, imagine that you want to include a specific application version (Example VMware tool version ) in the HTML inventory file. As I mentioned in part 4, I created the directory /home/Ansible-Workdesk. This where the “out” and “cmdb” directories have been created.

10. Create another directory called /home/Ansible-Workdesk/other_info/vmwaretool. I use this directory to deposit another JSON file for the VMware tool version after launching a playbook. Here is an extract from my InventoryUsingAnsibleCMDB.yml Playbook.

- setup:
  register: setup_res

- command: vmware-toolbox-cmd -v
  register: vmwareversion

- set_fact:
  vmwareversion: '{ "vmwareversion": {{ vmwareversion.stdout_lines }} }'

You can view the whole Ansible Playbook here on my Github.

11. Once the playbook has been executed, you will have identical files name in /home/Ansible-Workdesk/out and /home/Ansible-Workdesk/out/other_info/vmwaretool.

12. However, the content will be different. The one in the “out” directory will contain JSON files about the default Ansible inventory, whilst, the one in the “vmwaretool” directory will contain a JSON file about the VMware tool version having its parent as “vmwareversion“. I change the parent from “stdout_lines” to “vmwareversion” using the set_fact module in Ansible.

13. By now, you are ready to tweak the html_fancy_defs.html again as described in part 9. Both the Column definitions and Column functions need to be appended. Here is the line to be added in the Column definitions section:

{“title”: “VMware Tool”,        “id”: “vmwareversion”,        “func”: col_vmwareversion,         “sType”: “string”, “visible”: True},

And that of the Column functions section:

<%def name=“col_vmwareversion(host, **kwargs)”>
${jsonxs(host, ‘vmwareversion’, default=”)}
</%def>

14. Repeat steps at part 7 with the “vmwaretool” directory.


ansible-cmdb -t html_fancy_split out/ out/other_info/vmwaretool/

In case, you are able to create an Ansible Playbook to create valid JSON files by merging those in the vmwaretool directory to that of the out directory, please comment below. I would like to hear more about it.

Tips:

  • More Playbooks can be found on my Ansible-Playbooks Github repository.
  • With regards to part 3, if direct root access has been disabled on the destination servers, you can use -u <username> which will permit you to connect on the server.
  • The ansible-cmdb command also allows you to generate CSV file.
  • Part 10 lays emphasis on a separate JSON file. If you have been able to merge both outputs on the same JSON file that has been created by ansible default inventory please comment below.
  • The group in the ansible host file can also be added to the server inventory html file. Please see the ansible-cmdb doc for more information.

Running a server using Django

Running a server using Django is pretty easy. Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of Web development, so you can focus on writing your app without needing to reinvent the wheel. It’s free and open source. – Django.

Screenshot from 2016-04-23 18-59-12

1.To start with you will need to have Python which is already there by default on your linux machine. You will need to install Pip with the command. As I am on a Ubuntu machine im using the following commands.

apt-get install python-pip 

2. I am also running komodo-edit as editor. You easily install it with the following commands

add-apt-repository ppa:mystic-mirage/komodo-edit
apt-get update
apt-get install komodo-edit

3. Then, you will need to install django.

pip install django

4. Point yourself to a directory where you want to create your Django projects with the following commands. You would need to create a project. Here my project name is linuxproject. You would notice that a directory called linuxproject is created.

django-admin.py linuxproject

5. Get into the project directory. Here in my case its linuxproject. You should notice a manage.py file and a directory containing the project.

6. Now, open Komodo-edit. Create a new project in the same directory where your project has been created. Save the Django project with Komodo. It should looks something like this:

Screenshot from 2016-04-23 22-01-27

Save the project. launch the following command from terminal.

python manage.py migrate

Now, you can start your server with the following command

python manage.py runserver

You can now navigate to http://127.0.0.1:8000 In future articles, I will get into details of Django.

 

Tips:

  • You can also perform offline installation by downloading the Django Package and launch the following command for installation: python setup.py install
  • To create a password for the admin interface use the following command: python manage.py createsuperuser
  • In case you have encountered the “Invalid HTTP_HOST header error, you will need to add your IP in the settings.py file. Example is ALLOWED_HOSTS = [‘172.10.10.1’, ‘localhost’, ‘127.0.0.1’]

Starting up with Python – Part 16 – Last Part

Here we are with the last article “Starting up with Python”. This one will shed some ideas on Working with files, Reading and Writing and Writing lines in Python. You can view all the past Python articles here.

python_logo_by_bluex_pl

39. Working with Files

To write to a file is pretty simple. I have the file toto which is blank at /python/toto. After using the close function the phrase hey hackers mauritius is written to toto.

>> fileop=open('/python/toto','w')
 >>> fileop.write('hey hackers mauritius')
 >>> fileop.close()

To read from the file do the following. The number 3 here means 3 byte

>> fileop=open('/python/toto','r')
 >>> fileop.read(3)
 'hey'

If you want to read the whole file, just put nothing inside the read function. Its important to close the function to prevent memory leaks.

>> fileop.read()
 ' hackers mauritius

40. Reading and Writing

Let’s not read line by line. Here is how you read a whole line

>> fileop=open('/python/toto','r')
>>> print fileop.readline()
hey hackers mauritius

You can also read line by line and put it into a list

>> fileop=open('/python/toto','r')
>>> print fileop.readlines()
['hey hackers mauritius\n', 'how are you doing\n', 'what are the new projects?\n']

To create a new file or overwrite a file with some lines do this

>> fileop=open('/python/toto','w')
>>> fileop.write('This is a new LINE\n')
>>> fileop.close()

41. Writing Lines

Let’s now create a list and store it into a temporary variable called listtest. listtest is now a list of all lines in toto.

>> fileop=open('/python/toto','r')
>>> listtest=fileop.readlines()
>>> listtest
['This is a new LINE\n']
>>> fileop.close()

We can also modify the same list. Here [1] is the second line as it starts with 0

>> filelist=open('/python/toto','r')
>>> listtest=filelist.readlines()
>>> listtest
['This is a new LINE\n', 'this is another line\n']
>>> filelist.close()
>>> listtest[1]="this is second line"
>>> listtest
['This is a new LINE\n', 'this is second line']

However, this has not been saved to the file. Let’s see how to save it.

>> filelist=open('/python/toto','w')
>>> filelist.writelines(listtest)
>>> filelist.close()

 

 

Starting up with Python – Part15

If you have been following the Python articles since some days, you would have noticed that things are getting more interesting. In case, you have missed the past articles here is a recap. In this article, i will get into Constructors, Import modules, reload modules and Getting modules info.

python_logo_by_bluex_pl

35. Constructors

When an object is created, the methods need to be called but when a contructor is when the first object is created, the objects are automatically called.

Lets see a basic method and object analogy

>> class hackers:
... def mauritius(self):
... print "hackers mauritius"
... 


>> obj=hackers()
>>> obj.mauritius()
hackers mauritius

However, in constructors we want to automatically call the object.

In this example, you would notice a class called new and a method called __init__

>> class new:
... def __init__(self):
... print "hackers mauritius"
... print "this is a constructor"
... 
>>> newobject=new()
hackers mauritius
this is a constructor

36. Import Modules

In python, you can write modules and each time you can call it from anywhere in your code. So let’s create a module.

I created a file called hackers.py with the following simple data.

# cat hackers.py 
def testmod():
 print "this is hackers Mauritius"

This is how you can import different modules imported from a file. Modules can also be imported once per file.

>> import hackers
>>> hackers.testmod()
this is hackers Mauritius

37. Reload modules

However, if the source file is edited and you would import the same module, the result would be the same. To get result from the source file, use this parameter

>> import hackers
>>> hackers.testmod()
this is hackers Mauritius
>>> reload(hackers)
<module 'hackers' from 'hackers.py'>
>>> hackers.testmod()
this is hackers Worldwide

38. Getting modules info

To get modules info, there are in build module. For example, there is an inbuilt module called SQRT

>> import math
>>> math.sqrt(25)
5.0

To know what the module math contains, you can use the dir function

>> dir(math)
['__doc__', '__name__', '__package__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'hypot', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']

There is also the help function. You just need to type this

>> help(math)

You would noticed  a list of what each modules does. However, before using a module you can use this 

>> math.__doc__
'This module is always available. It provides access to the\nmathematical functions defined by the C standard.'

It gives a quick summary of what the module does.

 

 

Starting up with Python – Part14

Welcome to another Python article on Subclasses / Superclasses. I will also get into Ovewrite Variable on Sub as well as Multiple Parent Classes. Here is a mind map of all articles related to Python.

python_logo_by_bluex_pl

32.Subclasses

This is interesting when you can inherit a single parent class from a class.

>> class parentClass:
… var1=”country”
… var2=”region”

>>> class childClass(parentClass):
… pass

>>> parentObject=parentClass()
>>> parentObject.var1
‘country’
>>> childObject=childClass()
>>> childObject.var1
‘country’
>>> childObject.var2
‘region’

33. Overwrite Variable on Sub

We will now try to overwrite variable on subclass. I will create a parent class called hackers and the child class called project.

>> class hackers:
… var1=”country”
… var2=”age”

>>> class project(hackers):
… var2=”skills”

Let’s access var2 from the parent and the child.

>> parentobject=hackers()
>>> childobject=project()
>>> parentobject.var1
‘country’
>>> childobject.var2
‘skills

As we can see now if we try to overwrite its just by calling the variable in the class

>> childobject.var1
‘country’

This is cool if you have one huge class and if you want to change some few things in it.

34. Multiple Parent Classes

This is straight forward to use different items just by adding the name of your classes in your parameter.

>> class hackers:
… var1=”Mauritius”

>>> class skills:
… var2=”devops”

>>> class best(hackers,skills):
… var3=”we are hackers”

>>> childObject=best()
>>> childObject.var1
‘Mauritius’
>>> childObject.var2
‘devops’
>>> childObject.var3
‘we are hackers’