Category: BSD Exploration

DevConMru – Backup in the cloud for the Paranoid by

At Hackers Mauritius we work on several projects and code for fun. One of the interesting project we have look at is an application called Tarsnap which is use to perform secure backup on the cloud. At Hackers Mauritius, myself (@TheTunnelix) and Codarren (@Devildron) recently send codes to Tarsnap and same were approved. Thats really cool when someone’s code is approved and used world wide by thousand of companies. Thanks to Selven (@eldergod) and Loganaden (@loganaden_42 ) who are the creators of Hackers Mauritius who inpired us. Today, i have the privilege to speak on Tarsnap at the DevConMru 2016 which was held at Voila hotel, Bagatelle. On reaching there, i was impressed on the number of people already waiting inside the conference room who were curious about Tarsnap. Some were entrepreneurs whilst others were students. I should say around 30 people attended the conference. Since it was a sunday at 11:30 am, Selven did not hesitate to bring some beer to the little crowd present there. I was busy setting up my laptop for the presentation.

As usual i like to get the attention of my audience before the presentation. My first slide showed the logo of Tarsnap upside down.

Screenshot from 2016-05-22 19-05-41

Everyone was turning their head and making the effort to read the content. And here we go. I noticed that they are all ready and curious about it.

Check out the Slide here. Please wait some minutes. Its loading..

The basics of Tarsnap were explained. Tarsnap take streams of archive data and splits then into variable-length blocks. Those blocks are compared and any duplicate blocks are removed. Data de-duplication happens before its uploaded to the Tarsnap server. Tarsnap does not create Temporary files but instead create a cache file on the client. The cache file is the files that are being back up to the Tarsnap server. After deduplication, the data is then compressed, encrypted, signed and send to the Tarsnap server. I also explained that the archived are saved on an Amazon S3 with EC2 server to handle it. Another interesting point raised was the concept of Tarsnap which uses smart Rsync-like block oriented snapshot operations that upload only data which is charged to minimise transmission costs. One does not need to trus any vendor cryptographic claims and you have full access to the source codes which uses open-source libraries and industry vetted protocols such as RSA, AES and SHA.

Getting on to the other part of Tarsnap and Bandwidth, emphasis was made on Tarsnap which synchronised blocks of data using very intelligent algorithm. Nowadays, there are companies that still uses tapes for backups. Imagine having so many tapes and when restoration time has arrived, this would take tremendous time. Tarsnap compresses, encrypts and cryptographically signs every byte you send to it. No knowledge of crytographic protocols is required. At this point, i asked a question about volunteers who are thinking to look at the Tarsnap code. There were three persons who raised their hands. The importance of the Keyfile was raised up as some companies secure their private key in a safe. Tarsnap also support division of responsibilities where an explanation was laid out where a particular key can only be used to create archive and not delete them.

An analogy between google drive compared to Tarsnap was given. Many already understood the importance of Tarsnap compared to Google Drive. The concept of deduplication was explained using examples. For the network enthusiasts, i laid emphasis on the port 9279 which should not be blocked on the firewall as Tarsnap runs on the following port number. Coming to confidentiality, the matter was made clear enough to the audience how much the data is secured. If it happens someone lost the key there is no way of getting back the data. 

Tarsnap is not an open source product. However, there client code is open to learn, break and study. I laid emphasis on the reusable open source components that come with Tarsnap for example the Scrypt KDF (Key derivation function). KDF derives one or more secret keys from a secret value such as a master key, a password or passphrase or using a pseudo random function. The Kivaloo data store was briefly explained. Its a collection of utilities which togather form a data store associating keys up to 255 bytes with value up to 255 bytes. Writes are accepted until data has been synced. If A completed before B, B will see the results of A. The SPIPED secure pipe daemon which is a utility for creating symmetrically encrypted and authenticated pipes between socket addresses so that one may connect to one address. 

I also explained to the audience the pricing mechanism which was perceived rather cheap for its security and data deduplication mechanisms. Tarsnap pricing works similar as a prepaid utility-metered model. A deposit of $5 is needed. Many was amazed when i told them that the balance is track to 18 decimal places. Prices are paid exactly what is consumed.

Other interesting features such as regular expression support and interesting stuffs with the dry run features of Tarsnap was given. The concept of Tar command compared to Tarsnap was also explained. Commands, hints and tricks explained.

Some members of hackers Mu
Some members of hackers Mu

At the end, i consider it really important to credit Colin, the author of Tarsnap and i have been strongly inspired by the work of Michael Lucas on Tarsnap. Indeed, another great achievement of Hackers Mauritius at the DevConMru 2016.

GhostBSD for home and office users

GhostBSD is a super user friendly FREE operating system based on FreeBSD. Whilst FreeBSD is more for server and network administration side, GhostBSD is much more pretty for workstations. If you are thinking of having a much more secure desktop environment, i would surely advice GhostBSD.

Photo credits

GhostBSD shares many of the same features as FreeBSD, including:

  • Integrated Firewalls, Jails, Linux emulation, Network Virtualization, and bhyve.
  • KMS and new drm2 video drivers
  • The FreeBSD ports collection
  • The New Binary Packaging System pkgng

GhostBSD added a few extra features of its own, including:

  • A user-friendly installation process
  • Automatic detection of your computer’s hardware
  • Automatic configuration of your network card
  • Pre-installed desktop environments
  • Pre-installed codecs to play multimedia files

Lets see how to install GhostBSD on a Virtual Box Virtual machine. You can also try it on a virtual machine.

1.Download the ISO at the official website.

2.Create your Virtual machine and boot your ISO. It will prompt you for Graphical Install only, Failsafe mode and ACPI off. Well, choose the Graphical Install mode.

Screenshot from 2015-11-25 18:46:40

3. After some minutes, it would login automatically on the GUI interface. You also choose your ISO as per your best desktop environment.

Screenshot from 2015-11-25 18:49:02

4. Double click on the GhostBSD Installer icon. Then choose your language, keyboard, time zone and use the entire disk partition if you are not familiar with disk partitioning on BSD. You will need to complete some formalities about the User setup. And GhostBSD is going to install pretty much easily.

What is most interesting is that by default on GhostBSD, you have the FISH shell. There are several features with FISH such as auto-completion and color readable commands suggesting you for example a directory path. This is much pretty useful for beginners using BSD to be able to cope with all the commands easily. Welcome to GhostBSD. The adventure starts now.

 More article on BSD:

Adding a new disk on FreeBSD from VirtualBox


Sometimes, on Virtualbox, you need to remove the ISO after installation as it will prompt you to install again. I noticed that after installing GhostBSD compared to a CentOS installation which ejects the ISO automatically after installation.

Adding a new disk on FreeBSD from VirtualBox

Adding a new disk on FreeBSD is just a matter of minutes. As usual in the field of system administration i need to do some pre-checks first before carrying out any operation. There are many documentations available on the Official FreeBSD Handbook. However, i create this blog post so that we can discuss more about it.

1.Start by retrieving the trace from dmesg. I fired this command and redirect it into another file.

less /var/run/dmesg.boot > /home/dmesg1.txt

2. You can also redirect the output of the df -h  and or gpart show command.

3. Add the disk from the Virtual Box “Storage” tab, i create a new .vdi (Note that you need to switch off the machine before adding disk)

4. After the disk was added i booted the machine and fired another less /var/run/dmesg.boot > /home/dmesg2.txt

5. Then, i made a diff dmesg1.txt dmesg2.txt to compare both dmesg before and after insertion of disk to be assured that a new disk has been detected.

Screenshot from 2015-10-05 16:09:38

As you can see the result was awesome the new disk “ada1” was detected.

6. Now, we can need to check if we are using GPT or MBR. Through the gpart show command, we already know we are using GPT.

.Screenshot from 2015-10-05 16:14:01

7.  So i add the GPT to the disk and the partition is added with the following commands:

gpart create -s GPT ada1
gpart add -t freebsd-ufs ada1

8. Next step is where the file system on the new disk is created with the following commands:

newfs -U /dev/ada1p1

(tips: do press tab twice to see if you really have those device called ada1p1 to do not get confused if you already have more disks)

9. The final step is to create a new directory and mount the new disk:

mkdir /home/newhdd

10. Add the following entry in the /etc/fstab

/dev/ada1p1      /home/newhdd      ufs     rw     2     2

11. Mount the disk now:

mount /home/newhdd

12. I can now do a gpart show to see my new disk as well as a df -h