They do all this work, and then choose possibly the most confusing name they could?
BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
41–48 of 48 posts
Re: BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
#42Sorry, I have to say this is a poor and unpersuasive case for the architecture.
Performance increases are everything, yet unless I’m missing it there is no way to know what the improvements are.
For example, data on the chart here http://www.betrfs.org/faq.html, is enough information provided to reproduce the results including hardware and configuration? If not, the results are utterly meaningless.
Forward looking, it would strain credibility if you hadn’t tried to get a sense of the real world gains on Intel XPoint storage. I would speculate it will not be many years before there are no new green field deployments of storage that spins around in a circle.
Re: BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
#43Is it mandatory to implement this in the kernel? Is this because linux is a monolith?
No, FUSE literally stands for "Filesystems in User Space"
Re: BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
#44Earlier quoted context omitted.
This isn't completely fair, although most of it is. Some of the reasons for modifying Linux are performance enhancements to the filesystem layer that only make sense with a Bε-tree filesystem (and are not possible without patching Linux). They cover this in the hour-long talk at MS research linked elsewhere in this thread.[1] (Yes, it's long, but it's a pretty good presentation.) E.g., they describe modifying the pag…
> E.g., they describe modifying the page cache to write-through small modifications to file data, rather than dirtying the entire page and writing it back later (a form of write amplification). Considering that all the storage on the market now has sectors at least as large as a 4k page, this isn't actually reducing write amplification. At most, in some cases it might save a tiny bit of bus traffic.
They batch small edits into a log. Many edits to one sector written.
Re: BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
#45Sorry, I have to say this is a poor and unpersuasive case for the architecture. Performance increases are everything, yet unless I’m missing it there is no way to know what the improvements are. For example, data on the chart here http://www.betrfs.org/faq.html , is enough information provided to reproduce the results including hardware and configuration? If not, the results are utterly meaningless. Forward looking,…
First off, why would you use a research file system on enterprise hardware? Secondly, if you read just a little bit further you see that they are testing using spinning rust (HDD). Since they mention threads, I would hope that they are testing using something that supports hardware parallelism. So probably some relatively decent modern hardware. Mind you this is all speculation, and conclusions you could draw by reading the FAQ carefully.
Re: BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
#46Re: BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
#47Sorry, I have to say this is a poor and unpersuasive case for the architecture. Performance increases are everything, yet unless I’m missing it there is no way to know what the improvements are. For example, data on the chart here http://www.betrfs.org/faq.html , is enough information provided to reproduce the results including hardware and configuration? If not, the results are utterly meaningless. Forward looking,…
First off, why would you use a research file system on enterprise hardware? Secondly, if you read just a little bit further you see that they are testing using spinning rust (HDD). Since they mention threads, I would hope that they are testing using something that supports hardware parallelism. So probably some relatively decent modern hardware. Mind you this is all speculation, and conclusions you could draw by read…
If you describe results of an experiment, you need to provide the detail or pointer to how to reproduce it.
Benchmarking on enterprise hardware is completely relevant because that is the technology that is going to become dominant over the next few years so if performance gains do not show up on that type of technology they may not significant.
In some cases enterprise hardware is much different from what would be used in other scenarios, or on a massive scale, similar to how Facebook doesn’t go down and buy enterprise servers to run their data center. However in this case, the new generation of memory/storage hybrid will not be that different whether it’s in your laptop or in a server, Size features and scalability withstanding of course.
If I read an FAQ question that doesn’t have an asterisk or a pointer to the full information, it’s not my responsibility as the reader to go hunting around for details. This is the job of the author of the paper or website. You don’t get to brag without putting a
Re: BetrFS: An in-kernel file system that uses Bε trees to organize on-disk storage
#48Earlier quoted context omitted.
No, FUSE literally stands for "Filesystems in User Space"
Sorry the title of the article is in-kernel file system. Did I miss something?
You asked if it was mandatory to implement this in the kernel because Linux is a monolith. I pointed out that FUSE exists, and allows filesystems to be in userspace. I answered your question. As to some of the silly decisions made by this project? I can't speak to that.