30 July 2019

Filesystem benchmarks

Ok, let's get this straight.
When I choose to use JFS, it's because some years ago I see with my own eyes how JFS is reliable in different scenarios and it still is. Yes, EXT4 is reliable too, but the performance isn't on par with JFS, but both offers a reliable and secure solution. XFS in other hand isn't that secure and reliable (it is to point), but offers a quite good performance. When kernel 5.0 came out, they talked a lot about how BTRFS is good now and, different from most people that rely on "everyone uses, so I'll use too" I want to test myself because I'm not the guy that relies on this  "everyone guy" opinion.
The host used for this test is my main desktop using gentoo linux with the last available kernel (5.2.4-gento0) and last available tools to date (i5-3470 on a gigabyte motherboard, 16gb of ram, sata3 1Tb hdd). The disk is entirely formated with the filesystem being tested. All filesystems are mounted using noatime and using bfq scheduler (bfq offers better performance for rotational disks than mq-deadline)). The IOZone tests was executed with a reboot before creating the new filesystem to test to exclude any possible bias.

Test 1: Creating a 1Gb file with dd=/dev/urandom of=test bs=1024 count=1M (in seconds):

Test 2: Cold reboot during the creation of the 1Gb file from one filesystem to another (5 times)

Auto Fixed?5/55/53/54/54/5
Mounted rw without fixing01241
Wasn't able to fixing00121
Continue working with problems without reporting00121

Test 3: Copy a 1Gb file from one filesystem to another

Test 4: Cold reboot during mariadb heavy worload (5 times)

Auto Fixed?5/55/54/54/54/5
Corrupted databases?NYYYN
Mounted rw with problemsNNYYN
Continue working with problems without reportingNNYYN

Test 5: Boot from EFI to lightdm, sata3 hdd, mounting / and /home (5 times)

After a cold reboot0:421:221:402:112:10

Test 6: Shutdown from lightdm to total shutdown (openrc) 

Test 7: IOZone

My overall opinion:
  • JFS and EXT4 are good, performance wise and secure enough for daily usage.
  • XFS was polished through the years and the most awkward problems was taken off (data corruption with power loss and others), it's a good option but have some issues with performance depending of size and mass of data
  • ZFS is good but you need ram to have a good overall performance, I would suggest start with 16Gb for a desktop (this isn't a problem for a server, of course). Also, keep in mind that the mainline kernel doesn't support ZFS, it takes sometime for ZFS reach the last line (right now, ZOL supports up to 5.1.x). Also, despite the comparison of performance between ZFS and BTRFS seems to be similar, ZFS is far more stable and trustful.
  • By cpu/mem footprint in a copy, JFS is the lighter and ZFS the heavier. Of course, will also depend of how much data are you copying from where to where and kind of storage, but in overall, let's put this way: Copy lots of tiny files JFS > EXT4 > BTRFS > ZFS > XFS. Copy lots of big files: JFS/EXT4 > XFS > ZFS > BTRFS. Of course, the mileage will vary depending on the use case (it can be irrelevant with a full non-virtualized server with lots of ram and a good storage system).
  • Only use BTRFS if you like to restore backups often. To date, I can't trust BTRFS, not after seeing a filesystem having corruption with simple tests. -> http://lkml.iu.edu/hypermail/linux/kernel/1907.1/05873.html

Notes by 20190810:
  • XFS resolved the issues caused by power loss and metadata corruption. 
  • For XFS, the performance can benefit more with mq-deadline than bfs, afaik. Otherwise, use this recommendation in udev and don't forget to use noatime if you don't need it.
  • BTRFS still not ready, it can offer some benefits and performance, but the stability is still far from being acceptable, the auto-healing doesn't work as expected (like zfs, for instance) and scrub doesn't fix as expected in some scenarios. They're getting straight, a lot of this stuff seems to being fixed from 5.2 and beyond.
  • The fact of a filesystem receiving more kernel or userland updates doesn't mean it's more stable, or better, or whatever.
  • You SHOULD do your tests instead of talking bullshit like "everyone uses" or "it's an uncommon use". If the kernel supports it, it's supported. Period.

No comments:

Post a Comment