Through out the tunnel

Managing and Analysing disk usage on Linux

09/28/2015 tunnel1234 2 Comments

Disk usage management and analysis on servers can sometimes be extremely annoying even disastrous if proper management and analysis are not carried out regularly. However, you can also brewed some scripts to sort out the mess. In large companies, monitoring systems are usually set up to handle the situation especially where there are several servers with consequent sized partitions allocated for lets say /var/log directory. An inventory tool such as the OCS Inventory tool can also be used to monitor large number of servers.

This blog post will be updated as my own notebook to remember the commands used during management of disk usage. You can update me some tricks and tips. I will update the article here 🙂

Managing disk space with ‘find’ command

1.To find the total size of a particular directory of more than 1000 days

find . -mtime +1000 -exec du -csh {} + | grep total$

2.Find in the partition / files of more than 50 M and do a ls which is long listed and human readable.

find / -xdev -type f -size +50M -exec ls -lh '{}' ';'

3.Find in the /tmp partition every file or directory with mtime of more than 1 day and delete same.

find /tmp -name "develop*" -mtime +1 -exec rm -rf {} \;

4.Count from the file /tmp/uniqDirectory in the /home directory (uniqDirectory), every directory having the same unique name.

find /home > /tmp/uniqDirectory && for i in $(cat /tmp/uniqDirectory);do echo $i; ls -l /home/test/$i |wc -l;done

5. Find from /tmp all files having the extension .sh or .jar and calculate the total size.

find . -type f \( -iname "*.sh" -or -iname "*.jar" \) -exec du -csh {} + | grep total$

6. Find all files in /tmp by checking which files are not being used by any process and delete same if not being used.

find /tmp -type f | while read files ; do fuser -s $files || rm -rf  $files ; done

Another interesting issue that you might encounter is a sudden increase of the log size which might be caused by an application due to some failure issues Example a sudden increase of binary logs generated by MySQL or a core dump generated!

Lets say we have crash package installed on a server. The crash package will generate a core dump for analysis as to why the application was crashed. This is sometimes annoying as you cannot expect as to when an application is going to fail especially if you have many developers, system administrators working on the same platform. I would suggest a script which would send a mail to a particular user if ever the core dump has been generated. I placed a script here on GitHub to handle such situation.

Naturally, a log rotate is of great help as well as crons to purge a certain temporary logs. The “du” command is helpful but when it comes to choose and pick for a particular reason, you will need to handle the situation with the find command.

Tips:

You should be extremely careful when deleting files from find command. Imagine some replication log file is being used by an Oracle Database Server which has been deleted. This would be disastrous.
Also make sure that, you see the content of the log as any file can be named as *log or *_log

Deploying WordPress labs on Virtual Box

09/26/2015 tunnel1234 4 Comments

Building miniature virtual labs on Virtualbox are most of time fascinating especially when you have to troubleshoot between the virtual servers within a network environment, however there are usual bugs that i have to deal with. The difference between NATNETWORK and that of NAT on VirtualBox differs differently to what i have noticed, this can be seen on the official website documentation.

However, i have noticed that in both situation, you are provided with a virtual router within virtualbox. In the case of a NAT network, you are NOT allowed to ping between two VMs on NAT network unless you have established a tunnel whereas in the option of the NATNETWORK, this allows you to choose to dynamically range of IPs through the DHCP functionality on VirtualBox and you are also allowed to ping the outside world as well as other VMs on NATNETWORK.I have noticed that this work only on the new version compared to old ones where the NAT and NATNETWORK works almost the same way. There are still many discrepancies if ‘NatNetwork’ is the real name that should have been set!!

I have install Centos [minimum install] on my first lab. Here are the procedures for building the webserver.

yum install httpd wget mysql-server php php-mysql php-gd nmap traceroute w3m vim
wget https://wordpress.com/latest.tar.gz
tar -xzf latest.tar.gz && cp -r wordpress /var/www
chown -R apache:apache /var/www/wordpress
vi /etc/httpd/conf.d/myweb.conf

create the vhost with the following values

<VirtualHost *:80>
DocumentRoot /var/www/wordpress
ServerName www.myweb.com
ServerAlias myweb.com
<DIrectory /var/www/wordpress>
Options FollowSymlinks
Allow from all
</Directory>
ErrorLog /var/log/httpd/wordpress-error-log
CustomLog /var/log/httpd/wordpress-access-log common
</VirtualHost>

Time to create the Database

mysql -u root -p [mysqld service should be started first]
CREATE DATABASE mydb;
CREATE USER [email protected];
SET PASSWORD FOR [email protected]= PASSWORD (“mypassword”);
GRANT ALL PRIVILEGES ON mydb.* TO [email protected] IDENTIFIED BY ‘mypassword’;
FLUSH PRIVILEGES;

Exit MySQL and proceed with the following instructions.

mv /var/www/wordpress/wp-sample-config.php wp-config.php
Vi wp-config.php and modify username, dbname, password and hostname
vi /etc/hosts and enter myweb.com to run as localhost
Service httpd start // service httpd graceful // service mysqld start
w3m www.myweb.com register on wordpress. Website up

Setting up the SSL

For ssl activation [https] do this yum install openssl mod_ssl
openssl genrsa -out ca.key 2048 [to generate a signed certificate]
openssl req -new -key ca.key -out ca.csr [to generate the .csr]
openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt [generate a self-signed key]
cp ca.crt /etc/pki/tls/certs
cp ca.key /etc/pki/tls/private/ca.key
cp ca.csr /etc/pki/tls/private/ca.csr
vi /etc/httpd/conf.d/myweb.conf and add another vhost with the following values

<VirtualHost *:443>
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/ca.crt
SSLCertificateKeyFile /etc/pki/tls/private/ca.key
DocumentRoot /var/www/wordpress
ServerName www.myweb.com
ServerAlias myweb.com
<DIrectory /var/www/wordpress>
Options FollowSymlinks
Allow from all
</Directory>
ErrorLog /var/log/httpd/wordpress-error-log
CustomLog /var/log/httpd/wordpress-access-log common
</VirtualHost>

Service httpd graceful and website up on https

To make the website accessible on any hosts on same natnetwork, edit /etc/resolv.conf with ipaddress 10.0.2.4 myweb.com

Now that two servers are configured the same way, you can add another server as load Balancing to access the servers behind the load balancer. What is most interesting is that end users (hosts) will know only the load balancing server. I have achieve this by installing Pound on the server use as Load Balancing. This means that end users [hosts] will access the load balancing server which will in turn decides upon master/slave priorities. Pound converts server3 to a reverse proxy load balancing server. The aim is to take http/s request from the hosts and request server 1/2 according to the configuration.

Based on this article a new Bash project is actually being brewed on Github to automate the installation of WordPress, Apache, MySQL and all the application specified. This project should enable anyone to deploy a website through the script.

Linux Application, Linux System, Networking VirtualBox, Wordpress

Getting started with the basics of Github

09/26/2015 tunnel1234 Add Comment

“GitHub is a Web-based Git repository hosting service. It offers all of the distributed revision control and source code management (SCM) functionality of Git as well as adding its own features. Unlike Git, which is strictly a command-line tool, GitHub provides a Web-based graphical interface and desktop as well as mobile integration. It also provides access control and several collaboration features such as bug tracking, feature requests, task management, and wikis for every project.” – Wikipedia

To get started with Github, you have to create an account on Github.com. Once the account has been created, download the Git tool which is use to manipulate your repository.

Here are the steps which you need to follow :

Create a repository on the Github website. I am actually working on a project using this repository – TheTunnelix/Deploy

Now, you can start setting up git locally. Here are some commands which you need to remember:

git config –global user.name “TheTunnelix” – Setting up a username

git config –global user.email “[email protected]” – Configuring your mail address

git clone https://github.com:TheTunnelix/Deploy.git – Cloning your local machine with the Github Repo that you have created.

git status – You can use this command to understand the status of the repo. For example, the number of files which have not been commited yet.

git add README.md – When you add a file, you are now in between your local and the repository to proceed forward with the commit.

git commit -m “updated read me file for better description” – This is a way to document the stage area to be ready to finalize the repo. It is not actually a commit. Use git commit -help. Remember, each commit has an ID so that you can roll back in time.

git remote add origin https://github.com:TheTunnelix/Deploy.git – If you did not clone a repo, you may need to use this command because when a repo is cloned, you have already add the origin.

git push -u origin master – the “u” option means saving for the origin master (the branch you are pushing to). It will prompt you to enter your username and password. You will notice the following message “Branch master set up to track remote branch master from origin” which means that things went well.

From the Github account, you will notice that the change has been carried out. Changes are done using the three principles ADD-COMMIT-PUSH. You cannot push without committing. If a new file is created locally and you use the git status command, it will prompt you the test file which you have created.

git add . – Another interesting command is the git add . where the dot means to add untrack files following to a git commit -m “test file is added” The “-m” means message. A git status will show you that there is nothing to commit and everything is clean. Each git push will prompt you to enter your username and password.

git pull -u origin master – This will pull all the content of the current repo.

There also interesting options to explore on the Github website such as the branch/master. New branches can also be created. The graphs can be useful to view contributors on the project.

Scripts and codes

Hello Tunnelers

09/25/2015 tunnel1234 Add Comment

Hello Tunnelers across the globe. I made this blog to share my experience and knowledge as a System and Application Administrator. Most articles are based on real life experience in the field of Linux, FreeBSD and Open source technologies. However, additional tests are usually made to support my blog posts and i welcome constructive comments from you to enlighten me if needed.

Fellow Tunnelers, the Tunnelix is a concept that have inspired me to bridge Linux and Unix Operating systems tunneling through the hacking world. Do follow me on Twitter and join the adventure through out the Tunnel.

My website has been made using technologies like Nginx, HHVM, WordPress, CentOS, PHP, JQuery, MariaDB and others. I made some penetration testing using Kali Linux tools, Apache Benchmark and other online testing tools such as GTmetrix. You can follow my tweets to keep in touch with me. Your comments are welcome and i am also reachable on Facebook. Most blog posts will be based on the technical aspects of IT though sometimes i will blog about my own IT Management skills that i have encountered. Sharing is the key to success. Technology always keep on evolving and just as other blogs, old posts are sometimes void. I will try my best to keep all my blog posts up-to-date.

Linux System, Networking MariaDB, Nginx, PHP, Wordpress