Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite is 35% Faster Than The Filesystem (2017)

sqlite.org

Post reply on HN
Print view View original

101–110 of 137 posts

Re: SQLite is 35% Faster Than The Filesystem (2017)

#101

Post by pfraze »

gameswithgo wrote: post #52
chungy wrote: post #37

Earlier quoted context omitted.

> rarely a day goes by that I don't question how Microsoft manages to get anyone to buy that steaming pile they call an operating system. "But muh games!"

yeah maybe instead of making fun of that major industry other operating systems should treat it as an opportunity to gain market share.

SteamDeck is particularly interesting because of how much it is tackling that industry head-on. From the perspective of a mobile gaming console, the fact that it’s a PC is awesome. From the perspective of advancing gaming on Linux, it’s even more exciting

Re: SQLite is 35% Faster Than The Filesystem (2017)

#102

Post by phepranto »

laurent123456 wrote: post #9

Is it a known issue that the filesystem on Windows 10 is so slow? Being 5 times slower than macOS was roughly my experience but I thought there was just something wrong with my Windows laptop. I can't find any benchmark or explanation about this.

At the end of the article it says:

"SQLite is much faster than direct writes to disk on Windows when anti-virus protection is turned on. Since anti-virus software is and should be on by default in Windows, that means that SQLite is generally much faster than direct disk writes on Windows. "

Re: SQLite is 35% Faster Than The Filesystem (2017)

#103

Post by jjoonathan »

tyingq wrote: post #29
jjoonathan wrote: post #20

Earlier quoted context omitted.

It isn't just Windows 10, as far as I know every version of windows has always wanted to spend like a minute (or multiple minutes!) slowly counting the files before it proceeds to slowly delete them one by one... until it finds one that's open in a background process and nixes the whole operation in a half complete state. 5 times slower sounds conservative. It's awful. Hey, maybe they could put rounded corners on the…

I think that's warts in the file explorer more so than warts in the system itself.

Is "you can't delete a file if anything has it open" an explorer thing? I thought that was an OS thing.

Re: SQLite is 35% Faster Than The Filesystem (2017)

#104

Post by 7thaccount »

yashap wrote: post #90
kissgyorgy wrote: post #55

Earlier quoted context omitted.

I think it's just simply a lot of folks on HN (including me) just like SQLite very much and instantly voting up an article about SQLite (even when we already saw it :D).

Yeah, IMO there’s certain tech that HN, in aggregate, likes a lot, and readily upvotes positive articles about - SQLite is in that category, along with Postgres, CockroachDB, Go, Rust, etc. There’s also certain tech HN, in aggregate, strongly dislikes, and readily upvotes negative articles - Mongo, anything “modern JS ecosystem”, systemd, etc.

Often for good reason too. SQLite is amazing. Modern JS and frameworks...not so much.

Re: SQLite is 35% Faster Than The Filesystem (2017)

#105

Post by jjoonathan »

xxs wrote: post #38
jjoonathan wrote: post #20

Earlier quoted context omitted.

It isn't just Windows 10, as far as I know every version of windows has always wanted to spend like a minute (or multiple minutes!) slowly counting the files before it proceeds to slowly delete them one by one... until it finds one that's open in a background process and nixes the whole operation in a half complete state. 5 times slower sounds conservative. It's awful. Hey, maybe they could put rounded corners on the…

This has very little to do with Windows, itself. The issue is explorer that tries to estimate sizes and put the files in the recycled bin. Also likely it checks all the permissions while counting. Try the command line: "rmdir /s" and it's quick.

What about refusing to delete files that some process has open in a dangling descriptor? Is that just a GUI thing? I'm pretty sure it isn't.

Also, is there a better way to hunt down the background process than sysinternals? It seems pretty ridiculous that a low level debugging tool is required to do something as modest as reliably move and delete files, but I never heard of a better workaround.

Re: SQLite is 35% Faster Than The Filesystem (2017)

#106

Post by chungy »

puzzlingcaptcha wrote: post #82
chungy wrote: post #37

Earlier quoted context omitted.

> rarely a day goes by that I don't question how Microsoft manages to get anyone to buy that steaming pile they call an operating system. "But muh games!"

Yeah, 'cause the incredible fragmentation of Linux GUI toolkits and windowing systems is so fun to deal with. Year of Linux on the Desktop when.

Just take a look at the Windows desktop versus Linux desktop. The Windows desktop is far more fragmented and discongruous.

With Linux, the main difference is whether your desktop is primarily Qt or GTK. Sometimes an app using the other toolkit will look kind of funny, but most of those differences are paved over by any competent distro (eg: Debian, Fedora, Ubuntu, OpenSUSE...).

Re: SQLite is 35% Faster Than The Filesystem (2017)

#107

Post by poisonborz »

jjoonathan wrote: post #105
xxs wrote: post #38

Earlier quoted context omitted.

This has very little to do with Windows, itself. The issue is explorer that tries to estimate sizes and put the files in the recycled bin. Also likely it checks all the permissions while counting. Try the command line: "rmdir /s" and it's quick.

What about refusing to delete files that some process has open in a dangling descriptor? Is that just a GUI thing? I'm pretty sure it isn't. Also, is there a better way to hunt down the background process than sysinternals? It seems pretty ridiculous that a low level debugging tool is required to do something as modest as reliably move and delete files, but I never heard of a better workaround.

LockHunter is a nice right-click utility for this

Re: SQLite is 35% Faster Than The Filesystem (2017)

#108

Post by [deleted] »

FractalHQ wrote: post #72
daenz wrote: post #18

Somewhat related, I wrote a fuse-based file system in Rust recently that used SQLite as the backing store for file records, though not the file contents. I imagine I could use it for file content as well, so it's good to know more about its performance. https://amoffat.github.io/supertag/

I’ve wanted to do something like this for ages- definitely going to check this out! The limitations imposed by the hierarchical structure of traditional file systems bothers me rather frequently. Are there any notable trade-offs or limitations you’ve encountered with this approach?

[deleted]

Re: SQLite is 35% Faster Than The Filesystem (2017)

#109

Post by coldtea »

xxs wrote: post #34

No idea what's with the sqlite articles making the front page every other day. This one is pretty old as well. The measurements in this article were made during the week of 2017-06-05 using a version of SQLite in between 3.19.2 and 3.20.0.

It's a social voting site, and it also belongs to a VC group. So when things make it to the front page, they're either voted by many people, or pushed forward by the site owners.

In the case of SQLite, it's probably the first.

Articles also come in batches (of a topic). Somebody sends something about X (e.g. SQLite) and then others see it, some start reading more about X on their own and digging up further articles, and inevitably a few will post those other articles they've found as well. Rinse, repeat.

Age of article also has little to do with being on the front page of HN. There are articles that have been submitted, and made it to the front page, dozens of times from 2010 to today, and their original publication date can be half a century old (eg. some old Lisp articles, Djikstra articles, "Worse is better", etc.).

Re: SQLite is 35% Faster Than The Filesystem (2017)

#110

Post by blooalien »

puzzlingcaptcha wrote: post #82
chungy wrote: post #37

Earlier quoted context omitted.

> rarely a day goes by that I don't question how Microsoft manages to get anyone to buy that steaming pile they call an operating system. "But muh games!"

Yeah, 'cause the incredible fragmentation of Linux GUI toolkits and windowing systems is so fun to deal with. Year of Linux on the Desktop when.

> Year of Linux on the Desktop when.

Couple decades ago for me.

> Yeah, 'cause the incredible fragmentation of Linux GUI toolkits and windowing systems is so fun to deal with.

Sounds like probably "never" for you. You seem hostile toward Linux, so you're probably better off sticking with whatever you use currently.

Post reply on HN