How to backup a btrfs filesystem using Amazon S3 and s3backer

A while ago, a friend of mine introduced me to s3backer, a tool that makes you able to use a s3 bucket as a support for any filesystem you want.

I was looking for a way to backup my btrfs filesystem using btrfs send/receive commands, this looked like a good opportunity to do so. In addition I’ll use cryptsetup to encrypt the data on this filesystem.

In order to achieve this you need a few things ready :

• having compile installed s3backer as explained on the related github page
• having a btrfs filesystem and knowing how to use btrfs send/receive commands
• having a knowledge of cryptsetup create/open/close
• an amazon web services account and a s3 bucket ready (it can be done without paying anything if you keep the bucket size under 5Go) with an access key ready to be used

In this example I’ll make an encrypted 1Go btrfs filesystem, and use it to backup one btrfs subvolume.

We have to use s3backer and set it up so we can use our s3 account. Here I’m just following the github wiki of s3backer.

echo "AKIAJD4YVMKLA6LU5UDP:eapw9X9BW1hp11jO09Re17+3k4lz4L2k7vX73Cin" >> ~/.s3backer_passwd
s3backer --blockSize=128k --size=1g --listBlocks testbucket --region="eu-west-3" ~/.s3
s3backer: auto-detecting block size and total file size...
s3backer: auto-detected block size=128k and total size=1g
s3backer: MD5 cache size (10000) is greater that the total number of blocks (8192); automatically reducing
s3backer: listing non-zero blocks....................done
s3backer: found 4386 non-zero blocks
ls -l ~/.s3/
total 4096
-rw------- 1 user users 1073741824  4 févr. 17:03 file
-r--r--r-- 1 user users       1484  4 févr. 17:04 stats

The file in .s3 directory is what has to be seen as a regular partition. This is what we’re going to encrypt now.

sudo cryptsetup -v --cipher aes-xts-plain64 --key-size 256 --hash sha256 --iter-time 2000 --use-urandom --verify-passphrase luksFormat ~/.s3/file

Now we have an encrypted device, we need to decrypt it and expose it to be able to format it as btrfs.

sudo cryptsetup open --type plain  ~/.s3/file backup
Enter passphrase for /home/user/.s3/file: 

After typing the password, we now have a /dev/mapper/backup device that is the unencrypted version of the ~/.s3/file. We can now format it as btrfs.

sudo mkfs.btrfs /dev/mapper/backup

Now we are almost done with it, the last thing is to mount it somewhere as we would with any regular filesystem.

sudo mkdir /mnt/s3backup
sudo mount /dev/mapper/backup /mnt/s3backup
mount | grep btrfs | grep dm
/dev/dm-0 on /mnt/s3backup type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)

From this point /mnt/s3backup can be seen as any other btrfs filesystem could be. So we are going to use it as intended, let’s send something there.

First we need to create a subvolume and backup it

cd ~
sudo btrfs subvolume create tests3
sudo chown user: tests3
sudo btrfs subvolume snapshot -r tests3 tests3-backup
sudo btrfs send tests3-backup | sudo btrfs receive /mnt/s3backup

Now let’s add something into our subvolume

cd ~/tests3
curl -O -L http://archlinux.de-labrusse.fr/iso/2018.02.01/archlinux-2018.02.01-x86_64.iso
cd ..
sudo btrfs subvolume snapshot -r tests3 tests3-backup2
sudo btrfs send -p tests3-backup tests3-backup2 | sudo btrfs receive /mnt/s3backup
sudo du -sh /mnt/s3backup/*
0       /mnt/s3backup/tests3-backup
540M    /mnt/s3backup/tests3-backup2