Through out the tunnel

Installing OpenSSH on Windows 2012 R2 through PowerShell

10/15/2018 tunnel1234 Add Comment

This is probably my first article on Microsoft Windows. Some days back, I was asked to perform some tasks on Windows, though I’m not really a big fan of Windows, I managed to do it. Prior to the tasks, I wanted to have my usual SSH capabilities to log on the server, so I decided to install OpenSSH on the Windows 2012 R2 server. Microsoft has a repository for OpenSSH on Github. An interesting thing about Windows is that SSH has now been brought to Windows 2016. Well, I decided to add a new category on tunnelix.com about ‘Windows‘. Comment below if you find this weird!

Please credit tunnelix.com for using the picture

1. Point yourself into the directory where you want the file to be downloaded. In my case, it is the directory: C:\Users\Administrator\Desktop :

PS C:\Users\Administrator> cd C:\Users\Administrator\Desktop

2. The installation is pretty simple. You will need to download the .zip file from the Github repository using the Invoke-WebRequest command. By default, Invoke-WebRequest command supports TLS 1.1 and same has been deprecated. So you might need to change the security protocol to TLS1.2 or TLS1.3 using the following command:

PS C:\Users\Administrator\Desktop> [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

3. Then download the binary using the Invoke-WebRequest:

PS C:\Users\Administrator\Desktop> Invoke-WebRequest -Uri "https://github.com/PowerShell/Win32-OpenSSH/releases/download/v7.7.2.0p1-Beta/OpenSSH-Win64.zip" -OutFile "powershell.zip"

4. On a fresh installation, Windows 2012 R2 does not have the Expand-Archive command, so we will use .NET directly. Add-Type loads a .dll with the necessary .net functions in your current session. then [io.compression.zipfile] is a reference to that loaded .dll and ::ExtractToDirectory is the way to call a function from that dll :

PS C:\Users\Administrator\Desktop> Add-Type -assembly "system.io.compression.filesystem"

5. Now, we can unzip the file:

PS C:\Users\Administrator\Desktop> [io.compression.zipfile]::ExtractToDirectory( 'C:\Users\Administrator\Desktop\powershell.zip','C:\Users\Administrator\Desktop' )

6. After unzipping the file, get into the directory that has been unzipped and launch the installation:

PS C:\Users\Administrator\Desktop> cd .\OpenSSH-Win64

PS C:\Users\Administrator\Desktop\OpenSSH-Win64> .\install-sshd.ps1

7. The output should look as follows:

**** Warning: Publisher OpenSSH resources are not accessible.

[SC] SetServiceObjectSecurity SUCCESS
[SC] ChangeServiceConfig2 SUCCESS
[SC] ChangeServiceConfig2 SUCCESS
sshd and ssh-agent services successfully installed

8. The following command will show the status of the SSHD service:

PS C:\Users\Administrator\Desktop\OpenSSH-Win64> get-service | findstr ssh
Stopped ssh-agent OpenSSH Authentication Agent
Stopped sshd OpenSSH SSH Server

9. Launch the service with the following command:

PS C:\Users\Administrator> Start-Service sshd

10. You might need to add firewall rules to allow port 22 on the machine

PS C:\Users\Administrator> netsh advfirewall firewall add rule name=SSHPort dir=in action=allow protocol=TCP localport=22

11. You can also configure OpenSSH server to start automatically after the server reboot.

PS C:\Users\Administrator> Set-Service -Name sshd -StartupType "Automatic"

OpenSSH must be ready by now. You can SSH on your Windows server now. In future articles, I will blog more about Windows system administration, LDAP on Windows and more about Windows 2016 server. Enjoy 🙂

Windows openssh, PowerShell, Windows, Windows 2016

Project Tabulogs: Linux Last logs on HTML table with Ansible

10/10/2018 tunnel1234 Add Comment

Project Tabulogs: It will give you the ability to display logs on an HTML tabular format page using Ansible Playbook. Some days back, I shed some ideas, how to use Ansible to create an agentless server inventory with an interesting HTML format. In this post, we will see how to present information on an HTML tabular format logging information from lastlog. You can apply the same principle for any logs and present it to an HTML template. One of the advantage with Ansible is that you can also launch shell commands which allows you to automate several tasks whether remote or local. Many times, I came across non-technical personnel who want to see logins and logouts of staffs on specific servers. The goal of this project is to present accurate and reliable information to non-technical persons in an IT organization. It’s also a way to perused logins and logouts of users more easily.

All right reserved : tunnelix.com — All right reserved: tunnelix.com

Limitations and Solutions

However, there is some limitation of presenting bulk information such as logs on an HTML page. Imagine having hundreds of servers which could amount more than 20,000 lines. This would make the page impossible to load and may crash on the browser. To remediate this situation, a database such as SQLite could be interesting. Otherwise, AJAX can be used to fetch page by page. Since I’m keeping all information in JSON format in a JavaScript page, I decided to make use of Pagination. A search button will also come handy for all columns and rows.

Now, the thing is how to report that information which keeps on changing? let’s say every month you want to have a report of users connected on your organization’s servers in JSON format that’s too in a JavaScript page. Here is where Ansible comes useful. I created a Playbook to build the JSON array itself and the JavaScript page using the Shell module based on Linux Awk commands. Of course, there are other tasks Ansible will perform such as fetching of the information remotely, updating the HTML page: Example the date the report was created. The directory can then be compressed and send easily to anyone. So, no database needed!!. Cool isn’t it? However, if you want to adopt this system for really huge logs of information, it might work but could be really slow, hence, the need of a database.

You can clone the repo from my GitHub repository.

HTML, AngularJS and JSON

I created a simple HTML page which will call the AngularJS to keep the information in a JSON array. The HTML page will be static. However, for your own use, if you want to add some more columns, feel free to edit the section below. It is located at TabuLogs/master/perimeter1/index.html

 <tr class="tableheader">
                            <th>Hostname</th>
                            <th>Username</th>
                            <th>Login</th>
                            <th>TimeTaken</th>
                            <th>IPAddress</th>
                            <th>Perimeter</th>
                            <th>Application</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr ng-repeat="rec in SampleRecords | filter:q | startFrom:currentPage*pageSize | limitTo:pageSize">
                            <td>{{ rec.Hostname }}</td>
                            <td>{{ rec.Username }}</td>
                            <td>{{ rec.Login }}</td>
                            <td>{{ rec.TimeTaken }}</td>
                            <td>{{ rec.IPAddress }}</td>
                            <td>{{ rec.Perimeter }}</td>
                            <td>{{ rec.Application }}</td>
                        </tr>

Since I have used the lastlog from Linux, I called the page “Linux Login Management”. Otherwise, you can also filter any logs such as Apache or secure log. Some modifications will have to be carried out with the awk command or using Ansible Jinja filters.

You can also point a logo on the HTML page and kept in the images folder

<body ng-app="AngTable" ng-controller="Table">
    <p></p>
      <img src="images/logo1.png" align="right" alt=" " width="380" height="180"> 
      <img src="images/logo2.png" alt="cyberstorm" align="top-left" width="380" height="180">
    <p><h2>Login Management for my servers</p></h2>

The most interesting thing is about the data which is stored in a JSON array. In that particular example, we have information about the hostname, username, login, the time taken, IP Address, Application, and the Perimeter.

[ 
{"Hostname":"Apacheserver","Username":"tunnelix","Login":"Fri-Sep-28-15:11","TimeTaken":"15:11(00:00)","IPAddress":"192.168.0.1","Application":"Apache","Perimeter":"Production"},
 ];

The JSON array will be fetched by AngularJS to render the HTML table.

As mentioned previously, if you had tried to load all the JS page into your browser, same would crash, hence, to overcome this situation, pagination comes handy.

Messing around with Linux AWK command and the Ansible Playbook

The Linux AWK command is pretty powerful to concatenate all logs together. The more interesting is the conversion of the logs into JSON format. The playbook is located at TabuLogs/AnsiblePlaybook/AuditLoginLinux.yml

When the Playbook is launched we can see the first AWK will create a file with the hostname of the machine in /tmp and inside that file, the data is comprised of the Hostname, Username, Login, Time-Taken, and IP Address. To get back the logs from the previous month, I used Date=`date +%b -d ‘last month’`

Assuming you have rotated logs for /var/log/wtmp either weekly or monthly or any range, the loop is supposed to find logs for only last month. Now, in case you have kept wtmp logs for years, another method needs to be used to fetch log for the actual year.

Date=`date +%b -d 'last month'`; t=`hostname -s` ; for i in `ls /var/log/wtmp*` ; do last -f $i ; done | grep $Date | awk -v t="$t" '{print t, $1, $4 "-" $5 "-" $6 "-" $7, $9$10, $3 }' | egrep -v 'wtmp|reboot' > /tmp/$t

Once the logs have been created on the remote hosts, it is fetched by ansible and kept on the local server. Consequently, the remote files are destroyed.

We also need to merge all the data into one single file and also removing all blank lines, which is done using the following command:

awk 'NF' /Tabulogs/AnsibleWorkDesk/Servers/* | sed '/^\s*$/d' > /Tabulogs/AnsibleWorkDesk/AllInOne

Assuming that the file located at /Tabulogs/AnsibleWorkDesk/rhel7_application_perimeter contain information about the server, application, and perimeter, the following awk command will append the information to the table created remotely on the hosts.

awk 'FNR == NR {a[$3] = $1 " " $2; next} { if ($1 in a) { NF++; $NF = a[$1] }; print}' /Tabulogs/AnsibleWorkDesk/rhel7_application_perimeter /Tabulogs/AnsibleWorkDesk/AllInOne > /Tabulogs/AnsibleWorkDesk/AllInOneWithPerimeterAndApplication

Example of the format of the table is:

Nginx dev server2

Apache prod server1

Nginx dev server4

After adding all the data together on a simple table, the following AWK will convert each row into JSON format

awk '{print "{" "\"" "Hostname" "\"" ":" "\"" $1"\"" "," "\"" "Username" "\"" ":" "\"" $2"\"" "," "\"" "Login" "\"" ":" "\"" $3"\"" "," "\"" "TimeTaken" "\"" ":" "\"" $4"\"" ",""\"" "IPAddress" "\"" ":" "\"" $5"\"" "," "\"" "Application" "\"" ":" "\"" $6"\"" "," "\"" "Perimeter" "\"" ":" "\""$7"\"""}" "," }' /Tabulogs/AnsibleWorkDesk/AllInOneWithPerimeterAndApplication > /Tabulogs/AnsibleWorkDesk/perimeter1/table.js

It does not end here, we also need to add the JS codes to it. Using Lineinfile module of Ansible, writing at the beginning of the file is pretty easy.

lineinfile: dest=/Tabulogs/AnsibleWorkDesk/perimeter1/table.js line={{ item.javascript }} insertbefore=BOF
     with_items:
       - { javascript: "$scope.SampleRecords=[ " }
       - { javascript: "$scope.q = ''; " }
       - { javascript: "$scope.pageSize = 10; " }
       - { javascript: "$scope.currentPage = 0; " }
       - { javascript: "app.controller('Table', ['$scope', '$filter', function ($scope, $filter) { "}
       - { javascript: "var app = angular.module(\"AngTable\", []);  "}

Same method to write at the end of the file to create the table.js file.

lineinfile: dest=/Tabulogs/AnsibleWorkDesk/Perimeter1/table.js line={{ item.javascript }} insertafter=EOF
     with_items:
       - { javascript: " ]; " }
       - { javascript: "$scope.getData = function () { " }
       - { javascript: "return $filter('filter')($scope.SampleRecords, $scope.q) " }
       - { javascript: " }; "}
       - { javascript: "$scope.numberOfPages=function(){ "}
       - { javascript: "return Math.ceil($scope.getData().length/$scope.pageSize); "}
       - { javascript: "}; "}
       - { javascript: "}]); "}
       - { javascript: "app.filter('startFrom', function() { "}
       - { javascript: "    return function(input, start) { "}
       - { javascript: "start = +start; "}
       - { javascript: "return input.slice(start); "}
       - { javascript: " } "}
       - { javascript: "}); "}

After writing at the end of the file, the table.js is now completed.

Everything is assembled using the Ansible playbook. The Playbook has been tested for RedHat 7 servers. I believe it can also be adapted for other environments.

Final result

Here is an idea how will be the final result

Some information blurred for security reasons — Some pieces of information are blurred for security reasons

Future Enhancements

I’m so excited to share this article though I know there are much more improvements that can be done such as:

The awk command can be combined.
Removal of backticks on the shell commands.
shell command like rm -rf will be removed as well using the file module.
Some deprecated HTML tags were used.
Code sanitization.

My future goal is to improve the Ansible Playbook and the source code itself. Maybe someone else has a much better way of doing it. I would be glad to hear more. Please do comment below for more ideas. In case, you believed that some improvements can be done on the Playbook, feel free to send some commits on my GitHub repository or comment below.

As usual, a plan is needed. Consider reading the “tips” section below which might give you some hints.

TIPS:

There is, however, some other limitations with last command. Some arguments will not be present if you are using old utils-linux packages. Consider updating the package to be able to filter the last command easily.
If you can group your servers by some category or environment, it would be helpful.
There will be other versions of the project Tabulogs to create better and fast Ansible playbook. Lets called this one version 1.0

IT Management, Linux Application, Linux System AngularJS, Ansible, Automation, awk, DevOps, HTML, JavaScipt, JSON, linux, RedHat, rhel7, tabulogs

cyberstorm.mu meetup on OpenSource Licensing with Dr Till Jaeger

10/07/2018 tunnel1234 Add Comment

It was a great opportunity to meet with Dr. Till Jaeger, Attorney at Law on cybersecurity, laws, trademarks and opensource licensing on Tuesday the 02nd of October 2018 at Flying Dodo, Bagatelle Mall. The event was announced on the official cyberstorm.mu community Facebook group by Logan days back. Those present for the meetup was Loganaden Velvindron, Jagveer Loky, Rahul Golam, Kifah Meeran, Veegish Ramdani, Muzaffar Auhammud, Jeremie Daniel and myself from the cyberstorm.mu team.

Who is Dr. Till Jaeger? – Till Jaeger has been a partner at JBB Rechtsanwälte since 2001. He advises large and medium-sized IT businesses as well as government authorities and software developers on matters involving contracts, licensing and online use. Till Jaeger also covers conventional areas of copyright law and entertainment law, advising corporate clients on matters relating to open content, web design and photography. – Source: JBB.DE

Some pictures during the informal event:

OpenSource licensing with Dr. Till Jaeger and cyberstorm.mu

42935540_10212323570864049_8368692107799429120_n

42968234_10212323572504090_2567889336288673792_n

42950830_10212323576784197_4654246000414687232_n

43006824_10212323574624143_3826122953662136320_n

42972026_10212323735508165_2228714038051733504_n

42985512_10212323573544116_1829255669769830400_n

43021478_10212323736348186_8197301708435488768_n

43070514_249074402474539_4146444367671853056_n

43097410_10212323575984177_8134476138010378240_n

43119656_10212323575344161_97068219394686976_n

I should admit that it was really an informal meeting over some beers, juice, Dame-Blanche, and Pizzas with Dr. Till Jaeger. It was very fruitful and amazing to the team. We received lots of advice for ourselves as well as for the cyberstorm.mu team. Some days back, the National Computer Board conducted a workshop on Opensource licensing which where Dr. Till Jaeger was the main resource person to deliver this workshop. Thanks to the National Computer Board for making this event a success. Cyberstorm.mu members were also invited to the event.

Prev1 of 1Next

We have also welcomed Rahul Golam for joining cyberstorm.mu team and looking forward to work together. I seized this opportunity to announce that cyberstorm.mu is proud to announce it’s official logo. Thanks to the hard work of the team.

MeetUps and Presentations cybersecurity, cyberstorm.mu, licensing, OpenSource, Privacy, security

cyberstorm.mu – Going further and beyond from Mauritius to Africa and Worldwide

09/26/2018 tunnel1234 Add Comment

We are for free and opensource software. We are presenting Mauritius worldwide in the IT sector. We are a Linux and Open source group which is going further and beyond. We believe in collaborative work and team spirit. We believed in running codes. We strived for excellence. We have contributed codes in several popular applications. We have worked on Internet-Drafts. We participate heavily in Hackathons.

Proudly designed by: Chromiko, member of cyberstorm.mu — Proudly designed by Chromiko, member of cyberstorm.mu

It is with this vision and a strong will for a great change that cyberstorm.mu was born days back. After participating in several groups, we are now aiming for a better Mauritius. Cyberstorm.mu is just like the wrestling ring to fight for a better Internet. The world is going to benefit from it. The members who started running the cyberstorm.mu group are Loganaden Velvindron, Codarren Velvindron, Kifah Meeran, Veegish Ramdani, Muzaffar Auhammud, Nigel Yong, Jagveer Loky, Nathan Sunil Mangar, Jeremie Daniel, Heervesh Lallbahadur and myself (Nitin J Mutkawoa).

The team met at Flying Dodo, Bagatelle with a sip of Jack Daniel

The next step of cyberstorm.mu is championing several tracks for the IETF 103 hackathon remotely at Bangkok, Thailand. We have been welcomed by many people both from Mauritius and overseas, even from Silicon Valley, USA. No wonder, our past achievements is now a new step to reach today’s objectives – To focus heavily on research and development. Loganaden Velvindron from cyberstorm.mu mentioned on his Medium blog about the change cyberstorm.mu want to achieve: “The name cyberstorm.mu is an interesting one. It’s about change coming. Don’t get me wrong: I still love hacking. For me, hacking is about finding clever solutions to problems.” The youngest guy from the team is also going to participate in the IETF 103 hackathon.

The roadmap of cyberstorm.mu has already been plotted. We believe about us as a team moving in the right direction and showcase Mauritius worldwide. Feel free to join our cyberstorm.mu community Facebook group with a real profile’s name and you would be welcome. We are also present on Twitter and GitHub. Kindly note that all members of cyberstorm.mu have their official @cyberstorm.mu E-mail address. Otherwise, you can also contact us at [email protected]

What other’s are saying about cyberstorm.mu?

Charles Eckel, Chairman of the IETF Hackathon :

Peter Loshin from TechTarget:

MeetUps and Presentations Africa, cyberstorm.mu, hackathon, ietf, linux, mauritius

An agentless servers inventory with Ansible & Ansible-CMDB

09/16/2018 tunnel1234 Add Comment

Building from scratch an agentless inventory system for Linux servers is a very time-consuming task. To have precise information about your server’s inventory, Ansible comes to be very handy, especially if you are restricted to install an agent on the servers. However, there are some pieces of information that the Ansible’s inventory mechanism cannot retrieve from the default inventory. In this case, a Playbook needs to be created to retrieve those pieces of information. Examples are VMware tool and other application versions which you might want to include in your inventory system. Since Ansible makes it easy to create JSON files, this can be easily manipulated for other interesting tasks, say an HTML static page. I would recommend Ansible-CMDB which is very handy for such conversion. The Ansible-CMDB allows you to create a pure HTML file based on the JSON file that was generated by Ansible. Ansible-CMDB is another amazing tool created by Ferry Boender.

Let’s have a look how the agentless servers inventory with Ansible and Ansible-CMDB works. It’s important to understand the prerequisites needed before installing Ansible. There are other articles which I published on Ansible:

Ansible Basics and Pre-requisites

1. In this article, you will get an overview of what Ansible inventory is capable of. Start by gathering the information that you will need for your inventory system. The goal is to make a plan first.

2. As explained in the article Getting started with Ansible deployment, you have to define a group and record the name of your servers(which can be resolved through the host file or DNS server) or IP’s. Let’s assume that the name of the group is “test“.

3. Launch the following command to see a JSON output which will describe the inventory of the machine. As you may notice that Ansible had fetched all the data.

Ansible -m setup test

4. You can also append the output to a specific directory for future use with Ansible-cmdb. I would advise creating a specific directory (I created /home/Ansible-Workdesk) to prevent confusion where the file is appended.

Ansible-m setup --tree out/ test

5. At this point, you will have several files created in a tree format, i.e; specific file with the name of the server containing JSON information about the servers inventory.

Getting Hands-on with Ansible-cmdb

6. Now, you will have to install Ansible-cmdb which is pretty fast and easy. Do make sure that you follow all the requirements before installation:

git clone https://github.com/fboender/ansible-cmdb
cd ansible-cmdb && make install

7. To convert the JSON files into HTML, use the following command:

ansible-cmdb -t html_fancy_split out/

8. You should notice a directory called “cmdb” which contain some HTML files. Open the index.html and view your server inventory system.

Tweaking the default template

9. As mentioned previously, there is some information which is not available by default on the index.html template. You can tweak the /usr/local/lib/ansible-cmdb/ansiblecmdb/data/tpl/html_fancy_defs.html page and add more content, for example, ‘uptime‘ of the servers. To make the “Uptime” column visible, add the following line in the “Column definitions” section:

{"title": "Uptime",        "id": "uptime",        "func": col_uptime,         "sType": "string", "visible": True},

Also, add the following lines in the “Column functions” section :

<%def name="col_uptime(host, **kwargs)">
${jsonxs(host, 'ansible_facts.uptime', default='')}
</%def>

Whatever comes after the dot just after ansible_fact.<xxx> is the parent value in the JSON file. Repeat step 7. Here is how the end result looks like.

Getting beyond Ansible-cmdb

Now, imagine that you want to include a specific application version (Example VMware tool version ) in the HTML inventory file. As I mentioned in part 4, I created the directory /home/Ansible-Workdesk. This where the “out” and “cmdb” directories have been created.

10. Create another directory called /home/Ansible-Workdesk/other_info/vmwaretool. I use this directory to deposit another JSON file for the VMware tool version after launching a playbook. Here is an extract from my InventoryUsingAnsibleCMDB.yml Playbook.

- setup:
  register: setup_res

- command: vmware-toolbox-cmd -v
  register: vmwareversion

- set_fact:
  vmwareversion: '{ "vmwareversion": {{ vmwareversion.stdout_lines }} }'

You can view the whole Ansible Playbook here on my Github.

11. Once the playbook has been executed, you will have identical files name in /home/Ansible-Workdesk/out and /home/Ansible-Workdesk/out/other_info/vmwaretool.

12. However, the content will be different. The one in the “out” directory will contain JSON files about the default Ansible inventory, whilst, the one in the “vmwaretool” directory will contain a JSON file about the VMware tool version having its parent as “vmwareversion“. I change the parent from “stdout_lines” to “vmwareversion” using the set_fact module in Ansible.

13. By now, you are ready to tweak the html_fancy_defs.html again as described in part 9. Both the Column definitions and Column functions need to be appended. Here is the line to be added in the Column definitions section:

{“title”: “VMware Tool”, “id”: “vmwareversion”, “func”: col_vmwareversion, “sType”: “string”, “visible”: True},

And that of the Column functions section:

<%def name=“col_vmwareversion(host, **kwargs)”>
${jsonxs(host, ‘vmwareversion’, default=”)}
</%def>

14. Repeat steps at part 7 with the “vmwaretool” directory.

ansible-cmdb -t html_fancy_split out/ out/other_info/vmwaretool/

In case, you are able to create an Ansible Playbook to create valid JSON files by merging those in the vmwaretool directory to that of the out directory, please comment below. I would like to hear more about it.

Tips:

More Playbooks can be found on my Ansible-Playbooks Github repository.
With regards to part 3, if direct root access has been disabled on the destination servers, you can use -u <username> which will permit you to connect on the server.
The ansible-cmdb command also allows you to generate CSV file.
Part 10 lays emphasis on a separate JSON file. If you have been able to merge both outputs on the same JSON file that has been created by ansible default inventory please comment below.
The group in the ansible host file can also be added to the server inventory html file. Please see the ansible-cmdb doc for more information.

Linux Application, Linux System Ansible, Automation, HTML, linux, Linux Performance, Python, RedHat, ubuntu

Through out the tunnel

cat /tunnel/unix > tunnelix – Blog by Nitin J Mutkawoa

Installing OpenSSH on Windows 2012 R2 through PowerShell

Project Tabulogs: Linux Last logs on HTML table with Ansible

cyberstorm.mu meetup on OpenSource Licensing with Dr Till Jaeger

OpenSource licensing with Dr. Till Jaeger and cyberstorm.mu

Cheers with Dr Till Jaeger

OpenSource Licensing with cyberstorm.mu team and Dr Till Jaeger

OpenSource licensing meetup with Dr Till Jaeger

OpenSource Licensing with cyberstorm.mu team and Dr Till Jaeger

OpenSource Licensing with cyberstorm.mu team and Dr Till Jaeger

cyberstorm.mu – Going further and beyond from Mauritius to Africa and Worldwide

An agentless servers inventory with Ansible & Ansible-CMDB