Category: Security

Operation Prison Break by cyberstorm.mu – Sandboxing and Firejail

This is yet another successful hackathon carried out under the umbrella of cyberstorm.mu. Branded by the theme “Operation PB – Prison Break”, members of cyberstorm.mu shows skills of security innovations. We have also Rahul who is our proud newest member has created Sandboxing on  Strings

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

Photo credits: skycure.com
Photo credits: skycure.com

Our task was to find out vulnerabilities in a linux application and create a Firejail environment. Firejailing is the art of using a SUID program that reduces the risk of security breaches by restricting the running environment of untrusted applications using Linux namespaces and seccomp-bpf. It allows a process and all its descendants to have their own private view of the globally shared kernel resources, such as the network stack, process table, mount tableFirejail can sandbox any type of processes: servers, graphical applications, and even user login sessions. The software includes security profiles for a large number of Linux programs: Mozilla Firefox, Chromium, VLC, Transmission etc. To start the sandbox, prefix your command with “firejail”.

I decided to chose the CPIO, a tool to copy files to and from archives which recently was find to be vulnerable to DOS attack. Cvedetails.com explained the CVE-2016-2037 vulnerability where the cpio_safer_name_suffix function in util.c in cpio 2.11 allows remote attackers to cause a denial of service (out-of-bounds write) via a crafted cpio file. In brief, when a user is going to decompress a file, a user will be able to pass out file for the purpose of the attack. This has been achieved by QuickFuzz.

To sandbox the CPIO tools when decompressing files, Firejail application was used to isolate the program by making use of the syscalls. Here is the firejail environment :

You need to watch this video to understand firejail before reading the Profile 🙂

include /usr/local/etc/firejail/server.profile
include /usr/local/etc/firejail/disable-common.inc
include /usr/local/etc/firejail/disable-programs.inc
include /usr/local/etc/firejail/disable-passwdmgr.inc
caps.drop all
seccomp write,read,open,close,execve,access,brk,umask,munmap,fchmod,mprotect,mmap2,lstat64,fstat64,geteuid32,fchown32,set_thread_are,prctl,setresuid32,getgid32,setgroups32,setgid32,getuid32,setuid32,fcntl64,clone,rt_sigaction,nanosleep

Here are what participants in the hackathon are saying:

To prevent further vulnerabilities such as shown below from being used to target users, this firejail profile has been made. https://www.cvedetails.com/vulnerability-list/vendor_id-72/product_id-1670/GNU-Gzip.html – Yash

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

Decompressing .xz file within a sandboxing environment is just fascinating – Akhil

Many shell users, and certainly most of the people working in computer forensics or other fields of information security, have a habit of running /usr/bin/strings on binary files originating from the Internet. Their understanding is that the tool simply scans the file for runs of printable characters and dumps them to stdout – something that is very unlikely to put you at any risk. – Rahul

Previous hackathons carried out by cyberstorm Mauritius are as follows:


Activating MariaDB Audit log

“The purpose of the MariaDB Audit Plugin is to log the server’s activity. Records about who connected to the server, what queries ran and what tables were touched can be stored to the rotating log file or sent to the local syslogd.”MariaDB. The MariaDB  Audit Plugin works for MariaDB, MySQL and Percona server.

Photo Credits: MariaDB
Photo Credits: MariaDB

Links and Basics

Let’s see how to install the MariaDB audit plugin to retrieve all activity on the database server. I am actually testing it on a MariaDB 5.5 series database. You can use a 10.1 series if you want. For MariaDB installation there are articles which i have posted some times back such as MariaDB Galera cluster installation, MariaDB and its improved security features, Master-Master replication on MariaDB and a Master-Slave replication on MariaDB. I think those articles are pretty straight forward to dive into the installation procedures. Imagine having so many users connected on a database performing so many queries. One on the way to trace those requests are through the binlogs or by activating the MariaDB audit log. Let’s see how to activate the MariaDB Audit log.

Verification and Prerequisites

After installing your Database server, get into the console and launch the following command SHOW GLOBAL VARIABLES LIKE ‘plugin_dir’ ; It should prompt you something like this:

Screenshot from 2016-05-30 08-26-50

If you are using a 32-bit system if would be found in /usr/lib/mysql/plugin. Get into that directory and checked if you got a file called server_audit.so By default, on new MariaDB, its already available. However, if you are using MySQL-Server, or an old MariaDB or Percona server you would need to install the server_audit plugin. Download it from this link: http://www.skysql.com/downloads/mariadb-audit-plugin-beta Once downloaded, extract it and copy the file server_audit.so to the plugin dir value path [See screenshot above].

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

This does not means its already activated. You can verify it using the following commands SELECT * FROM information_schema.plugins WHERE plugin_name=’server_audit’; If it returns an emply set, it means its not yet activated otherwise you should have something like this:

Screenshot from 2016-05-30 08-37-56

Activating the Plugin

To activate the plugin, you can restart the service. However, there is another option to prevent any MySQL downtime by launching this command INSTALL PLUGIN server_audit SONAME ‘server_audit.so’;

Screenshot from 2016-05-30 08-42-33

Plugin configuration

After activating the plugin and if you are going to launch the command SHOW GLOBAL VARIABLES LIKE ‘server_audit%’; by default it would output you the following parameters.

Screenshot from 2016-05-30 08-51-03

These default values would normally create a log file called server_audit.log in the data directory. The values are self explicit. For example the parameter server_audit_file_rotate_size with value 1000000 means that when the size of the log is going to reach 1000000 bytes, its going to be rotated and nine files will be used before the log file will be overwritten. You also need to choose which type of events you want to log. Here is an example when activating all CONNECT, QUERY and TABLE event. If you want to audit only the CONNECT even, do set the variable to CONNECT only.

Screenshot from 2016-05-30 09-02-51

You can also turn off the plugin using the command SET GLOBAL server_audit_logging=OFF; As mentioned previously, the logs are saved at /var/lib/mysql/server_audit.log Here is an example of a log.

[google_ad data_ad_slot=” data_ad_format=’rectangle’]

Screenshot from 2016-05-30 09-08-30

Here are some of the most important variables:

  • server_audit_logging – Enables audit logging; if it’s not set to ON, audit events will not be recorded and the audit plugin will not do anything.
  • server_audit_events – Specifies the events you wish to have in the log. By default the value is empty, which means that all events are recorded. The options are: CONNECTION (users connecting and disconnecting), QUERY (queries and their result), and TABLE (which tables are affected by the queries).
  • server_audit_excl_users, server_audit_incl_users – These variables specify which users’ activity should be excluded from or included in the audit. server_audit_incl_users has the higher priority. By default, all users’ activity is recorded.
  • server_audit_output_type – By default auditing output is sent to a file. The other option is syslog, meaning all entries go to the syslog facility.
  • server_audit_syslog_facility, server_audit_syslog_priority – Specifies the syslog facility and the priority of the events that should go to syslog.

Log File Examination

Log file can also be examined. The audit is performed in such a way that if even one user connect and disconnect the the MySQL it would be easily detected. A connect and disconnect would usually appears as this:

Screenshot from 2016-05-30 09-22-42

Queries woud look like this. Even if there are errors on the query, it would keep it in the log

Screenshot from 2016-05-30 09-25-22

The server_audit_events variable specifies which of the five events to log, taking a comma-separated list of the event types as an argument. There are six types of log records:

Photo Credits: MariaDB
Photo Credits: MariaDB

The audit log format looks like this:

[timestamp],[serverhost],[username],[host],[connectionid],
[queryid],[operation],[database],[object],[retcode]

Other Tips and Tricks

  • To avoid a heavy load on the machine, you can exclude a specific user using the parameter server_audit_excl_users=test,toto 
  • If the server_audit_output_type variable is set to SYSLOG instead of the default, FILE, the audit log file format will be as follows:
    [timestamp][syslog_host][syslog_ident]:[syslog_info][serverhost],[username],[host],
    [connectionid],[queryid],[operation],[database],[object],[retcode]
  • Be aware, though, that passwords given with functions PASSWORD() or OLD_PASSWORD() in DML statements will still be logged as plain text in queries. Key strings used with encrypt functions likeENCODE() and AES_ENCRYPT() are also still logged in plain text.
  • DDL and DML statements can also be audited.

DevConMru – Backup in the cloud for the Paranoid by cyberstorm.mu

At Cyberstorm Mauritius we work on several projects and code for fun. One of the interesting projects we have look at is an application called Tarsnap which is used to perform a secure backup on the cloud. At Cyberstorm Mauritius, myself (@TheTunnelix) and Codarren (@Devildron) recently send codes to Tarsnap and same were approved. That’s really cool when someone’s code is approved and used worldwide by thousands of companies. Today, I have the privilege to speak on Tarsnap at the DevConMru 2016 which was held at Voila hotel, Bagatelle. On reaching there, I was impressed by the number of people already waiting inside the conference room who were curious about Tarsnap. Some were entrepreneurs whilst others were students. I should say around 30 people attended the conference. Since it was a Sunday at 11:30 am, the team did not hesitate to bring some beer to the little crowd present there. I was busy setting up my laptop for the presentation.

As usual, I like to get the attention of my audience before the presentation. My first slide showed the logo of Tarsnap upside down.

Screenshot from 2016-05-22 19-05-41

Everyone was turning their head and making the effort to read the content. And here we go. I noticed that they are all ready and curious about it.

Check out the Slide here. Please wait some minutes. It’s loading…

The basics of Tarsnap were explained. Tarsnap take streams of archive data and splits then into variable-length blocks. Those blocks are compared and any duplicate blocks are removed. Data de-duplication happens before its uploaded to the Tarsnap server. Tarsnap does not create Temporary files but instead create a cache file on the client. The cache file is the files that are being back up to the Tarsnap server. After deduplication, the data is then compressed, encrypted, signed and send to the Tarsnap server. I also explained that the archived are saved on an Amazon S3 with EC2 server to handle it. Another interesting point raised was the concept of Tarsnap which uses smart Rsync-like block oriented snapshot operations that upload only data which is charged to minimize transmission costs. One does not need to trust any vendor cryptographic claims and you have full access to the source codes which uses open-source libraries and industry-vetted protocols such as RSA, AES, and SHA.

Getting on to the other part of Tarsnap and Bandwidth, an emphasis was made on Tarsnap which synchronized blocks of data using a very intelligent algorithm. Nowadays, there are companies that still use tapes for backups. Imagine having so many tapes and when restoration time has arrived, this would take tremendous time. Tarsnap compresses, encrypts and cryptographically signs every byte you send to it. No knowledge of cryptographic protocols is required. At this point, I asked a question about volunteers who are thinking to look at the Tarsnap code. There were three persons who raised their hands. The importance of the key file was raised up as some companies secure their private key in a safe. Tarsnap also supports the division of responsibilities where an explanation was laid out where a particular key can only be used to create an archive and not delete them.

An analogy between google drive compared to Tarsnap was given. Many already understood the importance of Tarsnap compared to Google Drive. The concept of deduplication was explained using examples. For the network enthusiasts, I laid emphasis on the port 9279 which should not be blocked on the firewall as Tarsnap runs on the following port number. Coming to confidentiality, the matter was made clear enough to the audience how much the data is secured. If it happens someone lost the key there is no way of getting back the data. 

Tarsnap is not an open source product. However, their client code is open to learn, break and study. I laid emphasis on the reusable open source components that come with Tarsnap, for example, the Script KDF (Key derivation function). KDF derives one or more secret keys from a secret value such as a master key, a password or passphrase or using a pseudo-random function. The Kivaloo data store was briefly explained. Its a collection of utilities which together form a data store associating keys up to 255 bytes with a value up to 255 bytes. Writes are accepted until data has been synced. If A completed before B, B will see the results of A. The SPIPED secure pipe daemon which is a utility for creating symmetrically encrypted and authenticated pipes between socket addresses so that one may connect to one address. 

I also explained to the audience the pricing mechanism which was perceived rather cheap for its security and data deduplication mechanisms. Tarsnap pricing works similarly as a prepaid utility-metered model. A deposit of $5 is needed. Many were amazed when I told them that the balance is a track to 18 decimal places. Prices are paid exactly what is consumed.

Other interesting features such as regular expression support and interesting kinds of stuff with the dry run features of Tarsnap was given. The concept of Tar command compared to Tarsnap was also explained. Commands, hints, and tricks explained.

At the end, i consider it really important to credit Colin, the author of Tarsnap and i have been strongly inspired by the work of Michael Lucas on Tarsnap. Indeed, another great achievement of Cyberstorm Mauritius at the DevConMru 2016.


Operation WTF Hackathon by cyberstorm.mu – Day 2

After having set up our network environment for the operation WTF, cyberstorm.mu team started working on several vulnerabilities around WordPress content management system. It started on Saturday the 14th of May where several proofs of concept (POC) were established. I was shocked to see how come it is easy to exploit a WordPress website. For security purpose, we have to blank parts of  URLs before posting on social networks. No wonder, Loganaden Velvindron of cyberstorm.mu did not hesitate to give his opinion on Medium.com“Many bloggers use it, because it is both easy to set up, and there is a rich ecosystem of WordPress plugins. WordPress has often been criticized due to its security record. What is more worrying is the varying quality of the WordPress plugins.” 

POC - Vulnerability found !
POC – Vulnerability found !

Operation WTF – WordPress Tiny flaws end up on Sunday the 15th of May 2016 after lines of codes were fixed up from many WordPress plugins. We can also notice how dangerous it is if a proper audit is not carried out on WordPress before putting a website to production. Here is another example where the /etc/passwd was retrieved by hackers Mauritius.

CiZlRQFXEAA7Qbq

Patches were also written to fix up bugs. However, it is to be noted that for security reasons patches are not disclosed for the time being as at cyberstorm.mu we follow ethical rules and the aim to stay within the grey line is of paramount importance. 


Analysing an attack from WordPress Hello Dolly plugin

You might notice a heavy CPU usage consumption on your machine. Some may be a natural cause, for example, a known script being executed at a specific time whilst others may be due to a simple attack. Even if the attack is not a successful one, you may encounter a high CPU usage of your server which may eventually cause several kernels hangs or even cause other applications to be deprived of CPU usage. What I mean is that the goal of the attacker though it has not been reached, you may encounter a worse situation on your server.

Let’s see a brief analysis of an attack caused by a WordPress plugin known as “Hello Dolly”. The event started with a high CPU consumption on a server. Of course, by viewing the Htop or Atop processes, you can determine processes consuming more CPU.

Photo credits: Komodosec.com
Photo credits: Komodosec.com

1. Here is an idea of the Processes consuming more CPU by firing a simple ps command. The processes 829 and 4416 were the one consuming more CPU.

[[email protected]:/www/website.com/htdocs/wp-content]# ps aux|grep php

apache     829  6.2  0.5 351212 97492 ?        S    Oct05 127:36 php -q /tmp/tmp

apache    3459  0.3  0.3 416340 60080 ?        S    Oct06   1:43 php-fpm: pool www                                                                                            

apache    4416  7.2  0.5 336860 82656 ?        D    Oct05 146:43 php -q /tmp/tmp

apache    4753  0.2  0.3 420176 64048 ?        S    Oct06   1:20 php-fpm: pool www                                                                                            

root      7539  0.0  0.0 103248   868 pts/3    S+   06:55   0:00 grep php

2. We can notice that the process php -q /tmp/tmp emanate from a plugin on the server. For example, the PID 4416 corroborate with the lsof command.

[[email protected]:/www/website.com/htdocs/wp-content]# lsof plugins/

COMMAND   PID   USER   FD   TYPE DEVICE SIZE/OFF      NODE NAME

php       829 apache  cwd    DIR   0,20     4096 168820763 plugins

php      4416 apache  cwd    DIR   0,20     4096 168820763 plugins

bash    22664   root  cwd    DIR   0,20     4096 168820763 plugins

php     29199 apache  cwd    DIR   0,20     4096 168820763 plugins

php     29304 apache  cwd    DIR   0,20     4096 168820763 plugins

php     30153 apache  cwd    DIR   0,20     4096 168820763 plugins

3. If we make a strace -p of 2919 we can notice that it is trying to open the /etc/hosts file. 

[[email protected]:/www/website.com/htdocs/wp-content/plugins]# strace -p 29199

Process 29199 attached - interrupt to quit

socket(PF_NETLINK, SOCK_RAW, 0)         = -1 EMFILE (Too many open files)

open("/etc/hosts", O_RDONLY|O_CLOEXEC)  = -1 EMFILE (Too many open files)

socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = -1 EMFILE (Too many open files)

socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = -1 EMFILE (Too many open files)

alarm(0)                                = 15

rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, 0x7f66381729a0}, NULL, 8) = 0

poll([{fd=3447, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)

4. Another interesting information to know which website or URL does the intrusion emanating from is by firing a lsof -p on the PID:

[[email protected]:/www/website.com/logs]# lsof -p 29304

COMMAND   PID   USER   FD   TYPE     DEVICE SIZE/OFF       NODE NAME

php     29304 apache  cwd    DIR       0,20     4096  168820763 /www/website.com/htdocs/wp-content/plugins 

php     29304 apache  rtd    DIR      253,0     4096          2 /

php     29304 apache  txt    REG      253,2  4105624      16544 /usr/bin/php

5. If we now try to analyze the log by sorting only the bot, we can find some “POST” being carried out which comes from IP 92.62.129.97 . At first glimpse, it looks like a google bot. Are we sure?

92.62.129.97 - - [05/Oct/2015:20:45:49 -0400] "POST /wp-content/plugins/index.php?cookie=1 HTTP/1.0" 200 13 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com

/bot.html)"

92.62.129.97 - - [05/Oct/2015:21:20:43 -0400] "POST /wp-content/plugins/index.php?cookie=1 HTTP/1.0" 200 13 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com

/bot.html)"

6. Who is 92.62.129.97 ?

92.62.129.97

GeoIP Country Edition: LT, Lithuania

GeoIP City Edition, Rev 1: LT, N/A, N/A, N/A, N/A, 56.000000, 24.000000, 0, 0

GeoIP ASNum Edition: AS42549 UAB Baltnetos komunikacijos

7. Did we notice that this IP is well reputed for attacks?

Check out this website https://cleantalk.org/blacklists/92.62.129.97. You would notice that there were attacks even on some Windows server reported by some people.

8. After more research, we can conclude that several WordPress users have encountered the same situation where the Hello Dolly plugin was causing a heavy load on their servers. After they have removed it, things have changed. Ref:

We can deduce how unknown plugins on WordPress can be dangerous if the codes are not properly audited by security experts. An analysis is very important before using such kind of plugins.

Note: This information might be incomplete in some sort as it may happen that the Hello Dolly was already compromised prior to the attack. The aim of the article is to get show an analysis methodology due to a high CPU consumption.