Experimenting with BtrFS

Introduction

By all accounts, BtrFS is coming soon to a distro near you. I wanted to know more about it and experiment with it a bit, but I didn’t want to jeopardize any of my precious home lab machines with a still-experimental file system. I also didn’t want the hardware abstraction layer that installing it into a virtual machine would give me; however, I’m a hard drive junkie, so I had several USB hard drives sitting around with old backups on them that I wouldn’t need, so I decided I’d use one to see how much of a “Better File System” (it’s pronounced “better fs” or, amusingly, “butter fs”) this thing was. My goal was to install BtrFS on the USB drive, and then use it as a Samba share on my local network for non-critical data and experiment with some of the more interesting features as time permitted.

BtrFS is the next big thing in file systems, because it has an impressive feature list and is compatible with extent-based file systems. One can even convert an ext3 or ext4 partition to BtrFS in place if one were so inclined. It’s space efficient for small files and uses dynamically allocated inodes, but the snapshotting system is where it shines: It uses a copy-on-write scheme that allows for efficient incremental backups, online filesystem defragmentation, multiple devices with a wide variety of options in how individual devices can be utilized, and the ability to create a seed device — a read-only file system that acts as a template for other BtrFS file systems. Since it uses copy on write, all modifications are stored on different devices: the original is unchanged!

They have some pretty ambitious features still in the works, too, like in-band deduplication, hot data tracking, object-level mirroring and striping, and an online file system check, that have the potential to make it the choice of Linux administrators everywhere for a long time to come.

I used to have a real problem with betas. I’d install all of them and run on the super bleeding edge. While I’m older and somewhat wiser than I was in my misspent youth, my experience is covered in the scars of betas. Having said that, I was prepared for daily total cataclysm. I was pleasantly surprised in that I didn’t encounter a single bug or problem running BtrFS at home under almost no stress, but I will caution you before we continue that your mileage may vary and you should not, under any circumstances, use experimental file systems in a mission critical production environment without a solid backup plan that is regularly and rigorously tested. As a new file system, there aren’t a lot of recovery tools for it if things go horribly wrong.

Getting Started

Still with me, brave reader? If I haven’t completely dissuaded you and you’re up for a bit a of an adventure, let’s take that USB drive and do something cool with it. If there’s anything useful on yours, then you should probably back it up. What we’re going to do is going to destroy any data you might have on that drive.

The first step is, naturally, to plug it in to your Linux rig. You should be able to see it using lsblk, and you should make a note of the device name it got assigned.

Your Linux distribution might already have the tools necssary to utilize BtrFS, but you should double check with:

apt-get install btrfs-tools

or

yum install btrfs-progs

… and install the appropriate packages and dependencies.

Now you should enable the kernel module by using

modprobe btrfs 

Next, load up fdisk and remove any partitions from your USB drive (MAKE SURE YOU’VE GOT THE RIGHT ONE!), and, once that’s done,

partition the drive into whatever sized pieces you want; just make sure you use type “8e” for LVM. You could also install btrfs directly on to an unpartitioned drive, but I had mixed success with that, likely due to some eccentricities in my own systems.

If you want to go that route, please let me know if it worked for you and also if you can move it around among different physical machines. I couldn’t, though, again, this was likely user error.

Creating the file system is deceptively easy. If you’ve only got one device, you’ll use something like:

mkfs.btrfs -m single /dev/sdx

Where /dev/sdx is your USB drive. The -m single says you only want one copy of your metadata. This can be dangerous, because if you lose your only copy of the metadata in btrfs, you just lost all your data too. Since you’ve only got one drive, though…

If you have multiple drives or partitions you’re wanting to build a filesystem on, the command is:

mkfs.btrfs /dev/sdx /dev/sdy /dev/sdz {...}

Without any switches like this, it will use the non-redundant RAID0 for data and redundant RAID1 for metadata, which should be all right for our purposes.

Mount the resulting file system like you would any other, just know that if you went with the multiple drive option above you can use any of the devices in the mount statement and btrfs will figure out the rest:

mount /dev/sdx /mntpt

You can always add more devices later and resize the filesystem online (!) by simply running:

btrfs device add /dev/sdnew /mntpt

Then resize it with the size of the added device(s). For example, if /dev/sdnew were a 500 gigabyte drive, then something like:

btrfs filesystem resize -500g /mntpt

… would suffice. But you can also use “max” as well, as:

btrfs filesystem resize max /mntpt

Looking at your free disk space using the df tool, however, doesn’t quite work due to the ability to do raid striping and compression, so

df -h  # probably won't be correct on BtrFS!

results might be very wrong. This was the most frustrating part of btrfs for me — the inability to easily get an accurate reading on free space, which admittedly is a very small complaint. The extremely detailed and technical wiki article on figuring out your free space can be safely summed up with the statement “In general, it is impossible to give an accurate estimate of free space on any btrfs system.” Rather than go into extensive details here, I leave it as an exercise for the reader if you’re interested: https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_free_space_do_I_have.3F

You’ve now got a fully functioning btrfs file system mounted on your Linux rig. You can create new sub-volumes:

btrfs subvolume create /mntpt/newsubvol

Get the ID of that subvolume, and you can then mount it to another mount point:

btrfs subvolume list /mntpt
umount /mntpt
mount -o subvol=231 /dev/sdx /newmntpt

Yes, that means you can mount different subvolumes of variable size on the same physical disk to different mount points.

Snapshots of those subvolumes are super easy, too, and don’t eat up lots of space thanks to the copy-on-write scheme:

btrfs subvolume snapshot /source /destination

If things go horribly wrong, like if one of your 3 hard drives goes bad, you can still probably get some of the data back (you did back it up like we talked about right?) and remove the failed hard drive by mounting the system in degraded mode:

mount -o degraded /dev/sdb /mnt

That will allow you to mount the drive well enough to gain access to whatever remains of the files and allows you to remove the device from the file system.

BtrFS is still experimental, but it can be useful backing for LXC/LXD containers and is going to be an amazing file system once it’s done. It’s already pretty darned impressive, and I’m excited and ready for it to take its place as the defacto Linux file system.


Sources / Resources

Where can we go for more information?

https://btrfs.wiki.kernel.org/index.php/Main_Page

Looking for team training?