Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we sped up Notion in the browser with WASM SQLite

notion.so

Post reply on HN
Print view View original

81–90 of 107 posts

Re: How we sped up Notion in the browser with WASM SQLite

#81

Post by CyberDildonics »

bearjaws wrote: post #10

Overall I think using SQLite locally to offload database work is incredibly powerful. Given most laptops have an SSD now a days, you can scan an entire 30-100MiB SQLite db in miliseconds. Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. I just got done with a side project using WASM SQLite as well, it's incredibly powerful, even supports full text search. My project "cluttr"…

On a very slow SSD SQLite can query a 20 gigabyte database file in milliseconds.

Re: How we sped up Notion in the browser with WASM SQLite

#82

Post by chii »

funemployd wrote: post #75
remram wrote: post #36

Notion takes 15s to load to an empty page. Then another 5 to dismiss the popup about new AI features and the like. I'm glad they are making their app faster, in the meantime I (browser user) have cancelled my team's subscription and will be using something else.

That's because the big optimization would be "don't cram this entire application into the browser", but we've all collectively decided desktop software is impossible, so.

> we've all collectively decided desktop software is impossible

the customers are the ones who did.

Suppose there was a desktop app, but a competitor has a web version. It's much easier, faster to use the web version, because you dont need to download the desktop app, install it (which might require admin perms). Then your files are local, so if you have a second computer, there needs to be some way to share those files between.

Re: How we sped up Notion in the browser with WASM SQLite

#83

Post by [deleted] »

funemployd wrote: post #77
mrtksn wrote: post #20

Seeing things like that excite me a lot, I can't wait for a future where all the code is written in whatever language you like and runs through WASM in the browser. The HTML and JS can go back doing that thing they are built for: Displaying the UI and handling the interactions.

Computers can already do this! :D No browser required!

[deleted]

Re: How we sped up Notion in the browser with WASM SQLite

#84

Post by resonious »

bearjaws wrote: post #10

Overall I think using SQLite locally to offload database work is incredibly powerful. Given most laptops have an SSD now a days, you can scan an entire 30-100MiB SQLite db in miliseconds. Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. I just got done with a side project using WASM SQLite as well, it's incredibly powerful, even supports full text search. My project "cluttr"…

I wanted to try SQLite in the browser for a side project, but the startup time was unacceptable to me. It takes a solid second or two to crunch through the WASM binary, init SQLite, and open a database. Gets worse on first boot when you need to create all the tables.

There was a web.dev post or something saying "Web SQL is finally here! Just use SQLite with WASM!" Like, sure, that does seem to work, but it requires a huge WASM blob and a huge heap of JS glue. Indexeddb--as horrible as it is--starts up and is usable almost instantly. The sad part is that Indexeddb's querying is so bad you're forced to build your own database on top of it anyway...

I get where the web standards folks were coming from when they didn't like WebSQL just being SQLite in every browser, but frankly that would be so much better than what we have now even 5 years after back-peddling.

Re: How we sped up Notion in the browser with WASM SQLite

#85

Post by infogulch »

RcouF1uZ4gsC wrote: post #16

Honestly, we should just have native SQLite supported by the browser. SQLite has become the defacto local single file database standard. It has a very permissive license. It has support contracts out to the 2050s (service lifetime of Airbus A350 airframe). SQLite will probably have a longer lifetime than whatever browser standard we have now.

I was also disappointed, but this decision seems reasonable to me. SQLite is still evolving, if you pick a specific version then you're stuck on that version or risk breaking people that depend on specifics of that version. Also a vulnerability anywhere in SQLite would expose billions of people to exploits. This is not the same as applications using sqlite internally, the web is a much hotter environment: most apps may expect potentially hostile input in SQL params, but with WebSQL everything is potentially hostile: the queries, the database files, even the schema may conspire under confusing concurrency conditions to corrupt the state of the sqlite engine to break out and exploit the user. Whatever you think you have to agree that this is a LOT of new surface area.

I much prefer this approach where the site brings their own sandboxed SQLite WASM package.

Re: How we sped up Notion in the browser with WASM SQLite

#86

Post by ozgrakkurt »

remram wrote: post #36

Notion takes 15s to load to an empty page. Then another 5 to dismiss the popup about new AI features and the like. I'm glad they are making their app faster, in the meantime I (browser user) have cancelled my team's subscription and will be using something else.

Developing a fast system requires an entire different mindset, unless it is critical it never happens. Most people think it is w/e we don’t need to optimize and when you have people complaining it is already too late

Re: How we sped up Notion in the browser with WASM SQLite

#87

Post by lenkite »

resonious wrote: post #84
bearjaws wrote: post #10

Overall I think using SQLite locally to offload database work is incredibly powerful. Given most laptops have an SSD now a days, you can scan an entire 30-100MiB SQLite db in miliseconds. Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. I just got done with a side project using WASM SQLite as well, it's incredibly powerful, even supports full text search. My project "cluttr"…

I wanted to try SQLite in the browser for a side project, but the startup time was unacceptable to me. It takes a solid second or two to crunch through the WASM binary, init SQLite, and open a database. Gets worse on first boot when you need to create all the tables. There was a web.dev post or something saying "Web SQL is finally here! Just use SQLite with WASM!" Like, sure, that does seem to work, but it requires a…

SQLite is not SQL standards compliant. It doesn't even support data types from core SQL-92. One needs more than INTEGER, REAL, TEXT, BLOB for applications. Adopting it as WebSQL would have been a terrible mistake.

Re: How we sped up Notion in the browser with WASM SQLite

#88

Post by pjc50 »

jauntywundrkind wrote: post #38
AlexErrant wrote: post #6

> OPFS doesn’t come with graceful handling of concurrency out of the box. Developers should be aware of this and design around it. There's a multiple readers and writers proposal [0]. It's been "position: positive" by Firefox [1], implemented in Chrome [2], and ignored by Webkit [3] (of course). 0: https://github.com/whatwg/fs/blob/main/proposals/MultipleReadersWriters.md 1: https://github.com/mozilla/standards-posit…

It sucks that even the newest make-wadm-sqlite-fast FileAccessHandles are still an intermediated virtual file system hosted by the browser. I don't get why we have three different file system APIs on the web and none that just use files. Meanwhile WASI has their own server-side file system APIs but neither WASI nor the browser side seem to have any effort to get on the same page. It'll be 2035 before component-model…

The one thing you do NOT want to let websites have access to is files.

Re: How we sped up Notion in the browser with WASM SQLite

#89

Post by whalesalad »

bastawhiz wrote: post #31
bearjaws wrote: post #10

Overall I think using SQLite locally to offload database work is incredibly powerful. Given most laptops have an SSD now a days, you can scan an entire 30-100MiB SQLite db in miliseconds. Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. I just got done with a side project using WASM SQLite as well, it's incredibly powerful, even supports full text search. My project "cluttr"…

> Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. This is kind of an unfair comparison. Essentially nobody needs a million iops for their database. Even an extremely busy database doesn't need to scan all of the data it holds (or at least, if it does, you're using it very wrong—that's why we have indexes). A fast disk is possible on a laptop because it's a tiny hop to RAM. A…

Cloud providers have been delivering shit IOPS and charging unhinged prices for basic modern day performance that I really can’t agree with you at all. Nobody “needs” high IOPS the same way nobody needs a car that can drive faster than the 70mph. Why would anyone settle for an artificial ceiling?

Re: How we sped up Notion in the browser with WASM SQLite

#90

Post by aragonite »

captn3m0 wrote: post #79
aragonite wrote: post #66

This is both silly and somewhat off-topic, but on the subject of doing database work on the client side ... has anyone experimented with using non-rendered DOM as 'table' and CSS selectors as 'query language'? I'm not talking about 'using the DOM to store data' in the traditional sense. The idea is rather (roughly) to store each database 'table' as a child of a hidden DocumentFragment, with 'rows' as its children, et…

Not this, but the sqlite.js-http-vfs wasm implementation[0] (not the current official one) has the browser DOM available as a virtual table. Sample query from the post: select count(*) as number_of_demos from dom where selector match '.content div.sqlite-httpvfs-demo'; [0]: https://phiresky.github.io/blog/2021/hosting-sqlite-database...

Thank you!
Post reply on HN