Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Five Years of Btrfs

markmcb.com

Post reply on HN
Print view View original

211–220 of 240 posts

Re: Five Years of Btrfs

#211

Post by scottlamb »

danudey wrote: post #176
SEJeff wrote: post #137

Earlier quoted context omitted.

Historically, the biggest bugs in btrfs were when you came close to filling up the filesystem. For the longest time, you'd get -ENOSPC (no space left) even when you had many Gb of space left due to really bad metadata and block level space usage.

I'm a huge Mac fanboy, but APFS really kicks me in the teeth sometimes. Aside from things like snapshots, clones, etc. not being accessible to users (well, not really ), or being able to create subvolumes at specific mount points which forget those mount points next reboot, it had an extremely strange behavior (possibly relating to snapshots/CoW?) where once it was full, it stayed full forever until you rebooted. Bas…

> it had an extremely strange behavior (possibly relating to snapshots/CoW?) where once it was full, it stayed full forever until you rebooted.

Do you have Time Machine enabled? I think it uses snapshots, which explains why the filesystem stays full. I've hit this myself and was initially surprised to see rm not improving matters (possibly even making it worse) but it makes sense with snapshots. The working on reboot was a surprise. I'd put off fixing the machine for at least a week, and when I went to actually fix it, it was quite anticlimatic to just reboot and have it work. Maybe it checks for this condition on reboot and dumps Time Machine snapshots if so.

That was the less scary part of my macOS filesystem integrity worries. My full disk started when it was staging a full Time Machine backup after I got a dialog saying:

> Time Machine completed a verification of your backups on "my.nas.address". To improve reliability, Time Machine must create a new backup for you.

...for the Nth time. I don't know for certain if the problem is with Apple's software or with my NAS's (Synology) but these backups are clearly not as reliable as one would hope...

Re: Five Years of Btrfs

#212

Post by seized »

clSTophEjUdRanu wrote: post #73
InTheArena wrote: post #31

I went on a quest a few years ago, thinking it would be good for the industry to standardize on a single next generation filesystem for UNIX. I started with ZFS on linux since that seemed to have the most vocal advocates. That lasted about a half year, until a bug in the code resulted in a completely corrupt disk, and I had to restore 4TB of data over a month from offside backups. That plus the licensing confusion ar…

I really don't understand the insane hype around ZFS. You can't read any thread that touches on filesystems without the ZFS zealots coming out.

ZFS is like really good snow tires in the winter. You can tell people with other tires how great it is to have really good tires, but they dont believe you until they experience the benefits for themselves. Or put another way, no one "needs" ZFS until they really need it, then they wont live without it ever again.

I switched to it after the 7200.11 firmware mess, where the drives reported successful writes but didnt write anything. ZFS would have caught that, my Adaptec card certainly couldnt have and didnt.

ZFS to the rescue again a while later when those (now firmware updated) 7200.11 drives started dropping after 15k hours of service. ZFS saved my data when two drives started failing in my RAID5 set at the same time.

All the weird minor problems that would cause random issues or performance issues for other file systems like flaky SATA cables, intermittent HBA/backplane ports, etc. ZFS catches them all and informs you.

Having been hit by bit rot, corrupted files, corrupted file systems, etc etc before switching, ZFS is fantastic. And there is something great about watching it scrub at >1GB/sec, verifying every single bit of your data.

Re: Five Years of Btrfs

#213

Post by nwmcsween »

jmcnulty wrote: post #190
lostlogin wrote: post #87

Earlier quoted context omitted.

> I only have a primary self-hosted one that's on an RPi in my networking cabinet, and the second entry is Google. I was under the impression that there was no such thing as primary and secondary for DNS, just ‘here is one’ and ‘here is another’, with someone going for a terrible naming system of ‘primary’ and secondary’. I’m no expert and my knowledge come from messing about with Pihole and reading their documentati…

The first nameserver listed in resolv.conf is kind of a primary as it will always be consulted first, unless you add "options rotate". The next nameserver only come into play if the first doesn't respond (default 5 seconds, also tunable with options). They're not named primary/secondary in the file but could be considered that way.

Don't rely on this behaviour, many DNS libraries, will query all or n to save on latency.

Re: Five Years of Btrfs

#214

Post by kasabali »

ailideex wrote: post #5
e40 wrote: post #2

I've heard a lot of people say they won't use Btrfs due to reliability. Would have been nice to see that addressed.

What about the reliability? Are many people losing data with Btrfs?

Reliability does not only mean data loss. It may not be losing data but crashing every few hours, or locking up the system, or requiring constant monitoring and maintenance etc.

Re: Five Years of Btrfs

#216

Post by rdslw »

packetlost wrote: post #160
RX14 wrote: post #116

Earlier quoted context omitted.

It seems a lot of people have these stories, and then people like me and OP who have had btrfs survive the most fucked up situations (I've had a btrfs nas built on "random drives I've had lying around" and abused it for 5 years and had 0 bugs at all). I'm not sure what causes it, but there seems to be an effect where btrfs loves you or hates you and few people with mixed experiences regarding data loss. One possible…

In my experience, btrfs is very fragile in power loss or kernel crash/panic scenarios. It very consistently causes soft lockups on file read/writes after power loss until you run a `brtfs check --repair` on it. My experience is mostly on Arch, so it's not a case where it's out of date and missing patches.

Since a year I literally daily turn of my machine by pulling a plug (home automation turns off all plugs at midnight to make me go bed ;).

My quite large 1tb multivolume, multisnapshot BTRFS fs never had any problems.

And it's quite aggressive cfg (big fs commit).

P.S. I do have backups though.

Re: Five Years of Btrfs

#217

Post by pQd »

i've been using BTRFS since 2014 to store backups. there is a noticeable performance penalty when rsync'ing hundreds of thousands of files to a spinning-rust disk connected to USB-SATA dock when BTRFS is used instead of EXT4. i'm accepting it in exchange for ability to run scheduled scrub of the data to detect potential bitrot.

since 2017 i'm also using BTRFS to host mysql replication slaves. every 15 min, 1h, 12h crash-consistent snapshots of the running database files are taken and kept for couple of days. there's consensus that - due to its COW nature - BTRFS is not well suited for hosting vms, databases or any other type of files that change frequently. performance is significantly worse compared to EXT4 - this can lead to slave lag. but slave-lag can be mitigated by: using NVMe drives and relaxing durability of MySQL innodb engine. i've used those snapshots few times each year - it worked fine so far. snapshots should never be the main backup strategy, independently of them there's a full database backup done daily from masters using mysqldump. snapshots are useful whenever you need to very quickly access state of the production data from few minutes or hours ago - for instance after fat fingering some live data.

during those years i've seen kernel crashes most likely due to BTRFS but i did not lose data as long as the underlying drives were healthy.

Re: Five Years of Btrfs

#218

Post by tjoff »

fulafel wrote: post #202
tjoff wrote: post #111

Earlier quoted context omitted.

The hype is quite easy to understand. Snapshots and checksums are two complete game-changers. ZFS has them both. And there are no real alternatives in many cases. I've personally waited for BTRFS longer than a decade but my use-cases are yet to be considered stable (not something you really mess with in regard to filesystems). Honestly, as sure as I have been on the success of BTRFS I now consider BTRFS dead on arriv…

lvm and hence ext4 etc have had snapshots for ages.

As do NTFS. But they are not really comparable to "real" filesystem snapshotting, at least not in my opinion.

Re: Five Years of Btrfs

#219

Post by tjoff »

TurningCanadian wrote: post #204
tjoff wrote: post #111

Earlier quoted context omitted.

The hype is quite easy to understand. Snapshots and checksums are two complete game-changers. ZFS has them both. And there are no real alternatives in many cases. I've personally waited for BTRFS longer than a decade but my use-cases are yet to be considered stable (not something you really mess with in regard to filesystems). Honestly, as sure as I have been on the success of BTRFS I now consider BTRFS dead on arriv…

There were a bunch of btrfs raid56 patches last year. I think the known bugs have been addressed and is just that the wiki page hasn't been updated. Re obsolete, are you referring to RAID1C3?

I'm thinking of this:

https://www.zdnet.com/article/why-raid-5-stops-working-in-20...

I'd much prefer something like raidz3 compared to the authors setup.

RAID1C3 is nice but very expensive for use in bulk storage at home.

Re: Five Years of Btrfs

#220

Post by kasabali »

paulddraper wrote: post #166
takeda wrote: post #136

Earlier quoted context omitted.

Ext4 reserves space that can be used only by root, it is so system services can continue to work when users take all the space. It doesn't have issues like this if you exhaust all of that space. In ZFS and I'm sure in btrfs you can set up quotas and reserved space, globally and or user, but by default it is set to 0. I actually set my quota to 80% because apparently if you fill ZFS more it causes heavy fragmentation.

You're 100% correct; ext4 reserves this for a different purpose. I'm just saying it's not a entirely novel idea (albeit for a different reason).

Ext4 reserved space also helps with fragmentation.

To be more specific, reserved space on ext3 helps the fs so that it can be more flexible during allocation and avoid fragmentation.

Ext4 has delayed allocation mount option for that purpose so reserved space is not as much important for that but it'd still help if you turn off delayed allocations.

Post reply on HN