Tag: centos

SAR command daily tips and tricks

As promised on Twitter days back, I would post some interesting tips and tricks using the SAR (System Activity Report) linux command. The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the count and interval parameters, writes information the specified number of times spaced at the specified intervals in seconds. If the interval parameter is set to zero, the sar command displays the average statistics for the time since the system was started.die.net 

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

Understanding SAR and its main configuration files

The SAR command is part of the sysstat package which is a multi-purpose analysis tool and it is useful to pin point specific issue related to CPU, Memory, I/O and Network. The command is really useful especially to plot the output on a graph for visual analysis and reporting. One example of such tool is GNUplot. To install GNUplot and SAR use the command yum install sysstat gnuplot. The configuration file of SAR is located at /etc/cron.d/sysstat and /etc/sysconfig/sysstat . If you would perform a rpm -ql sysstat | less , you would noticed that there are other binaries such as iostat,mpstat,pidstat etc.. that comes along with the package sysstat.

SAR command daily tips and tricks 1

Difference between SA and SAR logs

In the directory /etc/cron.d/sysstat you would noticed that there is a cron which have been set up by default to run every ten minutes. The purpose is to write a log in the directory /var/log/sa . In this directory there are two type of files starting with sa and sar. SAR is the text file while SA is a binary. The sa file – Binary is updated every 10 minutes whereas the sar file is written at the end of the day. This parameter have been configured in the cron itself. By using the file command you would know which one is a binary or a text file.

SAR command daily tips and tricks 2

To open the sa file, you need to use the command sar -f . Here is an example:

SAR command daily tips and tricks 3

The /etc/sysconfig/sysstat file allows you to configure how long you want to keep the log, compression etc..

SAR command daily tips and tricks 4

Some ways to use the SAR command

You can also visualize sar logs live using the sar command with the start and ending second. Let’s say you have run a command on the background or simply want to track the resource status for some seconds, you can use the command sar 1 3 Here is an example with the command

sar 1 3

SAR command daily tips and tricks 5

Checking the load average, we apply the same principle but with the following command. The load average will also include load on each processor.

sar -q 1 3

To check for memory being consumed per seconds, use the following command

sar -r 1 3

To check number of memory coming in and out of the swap space, use the following command

sar -W 1 3

For the Disk I/O read write per seconds use the following command. Read/Write on disk also depend on the hardware

sar -b 1 3

For info about the CPU use the following

sar -u 1 3

To monitor the network activity in terms of packets in and out per interface received and compressed, use the following command

sar -n DEV 1 3

If the sar file has not been generated yet from the binary, you can use the following command, let’s say to convert it to a text in the /tmp directory.

sar -A -f  sa25 > /tmp/sar25

KSAR Graph with SAR logs

Now, in production environment, you need to to analyze for example at a specific time where memory or CPU was high. This can be done by means of a graph. I use the Ksar program. Ksar is a BSD-Licensed JAVA based application which create graph based on sar logs. You will need to install JAVA Runtime and launch the run.sh script to install the Ksar program. Once downloaded, just click on ‘Data’ and ‘Load from text file’. This is an example of swap usage

SAR command daily tips and tricks 6

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

TIPS:

  • The SAR output by default is in 12 hour clock format. To make it become a 24 hour clock format edit the .bashrc file and insert the parameter alias sar=”LANG=C sar
  • GNUPlot is one another application to plot your information on a graph for better analysis.

A simple and fast Memcache implementation on WordPress

Memcache is an in-memory key-value pairs data store. The data store lives in the memory of the machine. That is why it is much faster compared to other data stores. The only thing that you can put in memcache are key-value pairs. You can also put anything that is serializable into memcache either as a key or a value. In general, there are two major use cases such as Caching and Sharing Data Cross App Instances. In caching, datastore query results are kept or a user authentication token and session data. There are also APIs call that can also be cached. URL fetching can also be kept in memcache. Content of a whole page can also be kept in memcache. 

Results can be ten times much faster as explained by this benchmark test carried out by Google Developers team some times back.

Photo credits: Google Developers Team
Photo credits: Google Developers Team

1.For installation on a wordpress website, thats simple straight forward. On CLI, simply do a

yum install memcached

2. By default, memcache configurations on Centos7 is found at /etc/sysconfig/memcached as follows

[[email protected] ~]# cat /etc/sysconfig/memcached 
PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS=" "

3. To check if your memcache is running correctly, use the following command

echo stats | nc localhost 11211

4. Now, login to the plugin area and download the WP-FFPC plugin and activate it.

Screenshot from 2016-09-04 13-35-27

5. In the wp-config file of the wordpress CMS, define the following parameters

define('WP_CACHE', TRUE);

6. Save changes in the WP-FFPC plugins settings. On your server, you can now see the caching being done

[[email protected]]# echo stats | nc localhost 11211 | grep total_items
STAT total_items 32

And, here you are. A simple and fast memcache implementation on your WordPress website.

Note: By activating a wordpress plugin, you should make sure that proper auditing have been carried out as these days there are several vulnerabilities present on WordPress plugins.

Backup, Restore and work on PostgreSQL cmd line

Last time, we have a look how to install and set up PostgreSQL on a CentOS machine as well as to create a database. Let’s now see some more details about backups and database restore as well as some commands to master the basic of psql on command line.

Screenshot from 2016-03-31 05-31-15

1.We have created a database called test. Here is an example  how you create a table on Postgres

test=# create table hackers( id int primary key not null, country char(20), number int not null, project varchar(50) );

Screenshot from 2016-03-31 05-41-13

2. You can view your table with the command \d or \d <table name>

Screenshot from 2016-03-31 05-42-38

3. To perform a dump of a database log in as the user which have access to the database and launch the following command.

pg_dump test > db_backup.sql

To restore the same backup on a postgres machine, you need to have the database created first

psql test < db_backup.sql

4. A little work around of psql around the database for example accessing directly the database test with the command psql test and to list all the database can be done with \l

Screenshot from 2016-03-31 06-04-24

5. Let’s say you want to switch between database test and toto. This can be done with \c test and \c toto

Screenshot from 2016-03-31 06-07-05

6. To show all tables inside the database can be done with the command \dt

Screenshot from 2016-03-31 06-08-30

7. The schema of the table can also be seen with the command \d+ <table name> Here, it will be \d+ hackers

Screenshot from 2016-03-31 06-11-19

8. All users and permissions can be viewed with \du

Screenshot from 2016-03-31 06-15-18

9. Databases are create and drop inside the psql command line ending with a semi column

Screenshot from 2016-03-31 06-19-19

10. You can also alter user role with superuser and non superuser

Screenshot from 2016-03-31 06-21-51

11. You can also output the result of a select statement from the psql prompt directly using the following (For example the to output the result of a select statement to ~/SelectStatement.out)  :

postgres=# \o ~/SelectStatement.out - Start logging the output
postgres=# Select * from users;     - The command
postgres=# \o                       - Stop the logging

 

Install, Setup and Create DB on PostgreSQL

PostgreSQL is yet another open source object relational database system. Its compatible with almost all operating system including BSD, Windows and Linux. I will be using a Centos7 machine to install a PostgreSQL and set up some basics of PostgreSQL. In a next article, i will give some idea of something more robust you can do with PostgreSQL.

Photo Credits: postgresql.org
Photo Credits: postgresql.org

1.You can install PostgreSQL from the repository with the following command

yum install postgresql-server postgresql-contrib

2. Now the first action you need to perform after PostgreSQL server installation is to initialize the database by creating a new database cluster

postgresql-setup initdb

3. You can now start the postgresql service. Postgres will be listening on port 5432

systemctl start postgresql

4. Now that we have PostgreSQL installed on our machine, we can create the first super user with following command. Let’s called the user test. Its a practice to use the username having sudo privilege on the machine itself.

sudo -u postgres createuser --superuser  test

5. To connect on the Postgres command line use the following. postgres is the default user and psql is what u what to run

sudo -u postgres psql

6. You should have something similar to this.

Screenshot from 2016-03-30 21-21-11

7. Now you can set the password for the superuser that you have just created. In my case it is user ‘test’ at step 4

\password test

8. Once you have been prompted to enter the password twice means that you have already set up PostgreSQL. You can exit with the command

\q

9. To connect to the default postgres database you simply need to use the command. To quit follow step6

psql postgres

10. Lets create a database with the superuser. You might need to add your user to the group postgres with the command usermod -g test postgres as postgres will need permission to access your home directory to drop the .psql_history file

sudo -u postgres createdb test

11. To get on the command line you just need to type psql which should show you something similar to this

Screenshot from 2016-03-30 21-25-49 

MariaDB-10.1 Galera Cluster on CentOS 7

Some times back, i posted two articles on MariaDB Master-Master replication and MariaDB Master-Slave replication. Well, after several requests from friends, i was asked to blog on MariaDB Galera Cluster. MariaDB Galera Cluster is a synchronous multi-master cluster for MariaDB. It is available on Linux only, and only supports the XtraDB/InnoDB storage engines (although there is experimental support for MyISAM. – Mariadb.com. You can easily download Galera Cluster using the Repository Configuration tool. There is also a slight difference between oldest version of MariaDB compared to that of the 10.1 series. Recently, a question was asked on ServerFault to which i clarified same. Check it out for more information. Lets now have some funs with MariaDB Galera Clustering.

Photo credits:Mariadb.org
Photo credits:Mariadb.org

I have created 3 CentOS7 virtual machines on Virtual Box. Lets called each Server as Galera1 [192.168.1.9], Galera2 [192.168.1.10] and Galera3[192.168.1.11]. You need to edit the /etc/hosts file to allow each machine to ping each others via hostname. For testing purposes port 3306 which is by default listening on MySQL should be open. I have also disable SELinux and Firewalld as same are for testing purposes. Lets now install MariaDB Galera Cluster.

1.Edit the repository by using configuration as mentioned on the official MariaDB website.

# MariaDB 10.1 CentOS repository list - created 2015-11-08 17:34 UTC
# http://mariadb.org/mariadb/repositories/
[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.1/centos7-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

2. Lets install some Pre-requisites and other interesting tools.

yum install rsync nmap lsof perl-DBI nc

3. Install MariaDB-server and start the Mariadb service

yum install MariaDB-server MariaDB-client MariaDB-compat galera socat jemalloc
service mysql start

4. Launch the mysql_secure_installation command to set up the username and password. At this point you can start MariaDB normally by using the mysql command and create a database.

5. After the normal procedure to set up the first database on galera1, you can create a database and dump it, then import the database on galera2 and galera3. You have now 3 independant databases on 3 different servers. I would adivce you to dump the whole database with the command

mysqldump --all-databases > db.sql

6. We will now start setting up the Galera clustering. On galera1, edit the /etc/my.cnf.d/server.cnf file on galera1 and configure it as follows. The server galera1 is the being setup as the first primary cluster. So every other cluster is going to be set up in the network that is galera2 and galera3 is going to replicate itself from galera1. Add the following parameter under the [galera] option.

[galera]
# Mandatory settings
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address='gcomm://'
wsrep_cluster_name='galera'
wsrep_node_address='192.168.1.9'
wsrep_node_name='galera1'
wsrep_sst_method=rsync
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0

7. Once this is added, you can now start the first Galera cluster with the command. Make sure that the mysql service is stopped first. as at step5 we have started mysql to create a test database

service mysql start

8. You would noticed that the ports 3306 and 4567 should be listening

Screenshot from 2016-03-27 11-23-59

9. Now you would like to make galera2 joined the primary cluster i.e galera1. Simple set up the galera2 /etc/my.cnf.d/server.cnf file and under [galera] enter the following parameter. The only difference is the wsrep_node_name, wsrep_node_address and the gcomm value where you need to add the first cluster.

[galera]
# Mandatory settings
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address='gcomm://192.168.1.9'
wsrep_cluster_name='galera'
wsrep_node_address='192.168.1.10'
wsrep_node_name='galera2'
wsrep_sst_method=rsync
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0

10. Once mysql is started with the command service mysql start, you would noticed that galera2 is now on replicating from cluster1. You should noticed something similar to this on your log.

Screenshot from 2016-03-27 12-33-56

11. Repeat the same procedure for galera3 by tweaking the gcomm value and modifying the wsrep_node_address etc..

If you interested in having some more ideas about the MariaDB improved security features, do check out the following link where I attended a conference on MariaDB organized by the MSCC.

Tips:

  • If  you are using the 10.0, you would need MariaDB-Galera-server package compared to 10.1 where you just need MariaDB-server.
  • There are no longer separate MariaDB Galera Cluster releases for MariaDB 10.1 and above. Simply download MariaDB (10.1 or above) and configure your cluster as normal. MariaDB-10.1 is just ‘Galera ready’.
  • If you are migrating from MariaDB-server to Galera make sure you have clean up all the residues belonging to the old versions. See http://serverfault.com/questions/646973/installing-mariadb-galera-via-yum-fails-with-no-package-mariadb-galera-server.
  • The galera package will provide you the /usr/lib64/galera/libgalera_smm.so  and several binaries for the cluster configuration. There are several cluster configuration under ws_rep see the documentation.
  • The socat package establishes two birectional byte streams and transfers data between them.
  • I would also recommend to disable SELinux on the machines.
  • When you start the mysql service, you can tail the log at /var/log/messages otherwise you can set up mysql log.
  • Use this command show status like ‘wsrep_%’; on your mysql prompt to understand the replication.
  • To know how much node in the cluster use the following command show status like ‘wsrep_cluster_size’;