Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we sped up Notion in the browser with WASM SQLite

notion.so

Post reply on HN
Print view View original

21–30 of 107 posts

Re: How we sped up Notion in the browser with WASM SQLite

#21

Post by bearjaws »

bigbones wrote: post #14
bearjaws wrote: post #10

Overall I think using SQLite locally to offload database work is incredibly powerful. Given most laptops have an SSD now a days, you can scan an entire 30-100MiB SQLite db in miliseconds. Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. I just got done with a side project using WASM SQLite as well, it's incredibly powerful, even supports full text search. My project "cluttr"…

> Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. Curious about how this number was arrived at and what products were involved

RDS with 256,000 IOPS (the max) on a single primary instance of MariaDB.

Of course its not realistic to run a primary DB on a laptop, but the IOPS are extremely expensive in AWS was the point.

Offloading expensive queries to a browser is a viable solution. If I had decided to make this some sort of SaaS offering I am certain running full text search at scale would cost thousands of dollars per month long term, with the data becoming increasingly irrelevant over time, and I would still be forced to host it.

Re: How we sped up Notion in the browser with WASM SQLite

#23

Post by nailer »

jakubmazanec wrote: post #17

Will the tables in the Notion mobile app still be so painfully slow? Every time I scroll few rows down (in a table with few than 100 rows with dozen columns), the app hangs for several seconds while trying to fetch/render (who knows what it's actually doing) few additional rows. When I edit a row and return back to the table view, scroll position is lost and I have to suffer through the fetching once again.

They’re a very un-user focused company. They used the | cursor for any horizontally scrolling content for years, making a kanban board with non default states is nightmarishly complex, and there’s a billion odd UI slownesses and niggles.

But they’re recently added AI.

Re: How we sped up Notion in the browser with WASM SQLite

#24

Post by ilrwbwrkhv »

nailer wrote: post #23
jakubmazanec wrote: post #17

Will the tables in the Notion mobile app still be so painfully slow? Every time I scroll few rows down (in a table with few than 100 rows with dozen columns), the app hangs for several seconds while trying to fetch/render (who knows what it's actually doing) few additional rows. When I edit a row and return back to the table view, scroll position is lost and I have to suffer through the fetching once again.

They’re a very un-user focused company. They used the | cursor for any horizontally scrolling content for years, making a kanban board with non default states is nightmarishly complex, and there’s a billion odd UI slownesses and niggles. But they’re recently added AI.

I think their real innovation was the community and getting "cosy productivity" during Covid going. They are more of a social phenomenon than a technical one.

Re: How we sped up Notion in the browser with WASM SQLite

#26

Post by bearjaws »

NortySpock wrote: post #13
bearjaws wrote: post #10

Overall I think using SQLite locally to offload database work is incredibly powerful. Given most laptops have an SSD now a days, you can scan an entire 30-100MiB SQLite db in miliseconds. Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. I just got done with a side project using WASM SQLite as well, it's incredibly powerful, even supports full text search. My project "cluttr"…

If you think those read speeds are great, try DuckDB (which has many SIMD improvements) if you want to blow your socks off.

This solutions scales to tens of thousands of images in my testing (probably ~200k or more before searching is too slow). Why use a nuclear bomb when a stick of dynamite is good enough :)

Re: How we sped up Notion in the browser with WASM SQLite

#28

Post by simonw »

nafey wrote: post #5

I believe it's a missed opportunity not to bundle sqlite in major browsers. I understand that making SQLite part of the browser spec introduces all sorts of complications [1] down the line. But nevertheless, it would have proved to be quite a nifty tool. [1] https://hacks.mozilla.org/2010/06/beyond-html5-database-apis...

I'm so glad they didn't. We would all be stuck on some 14 year old baseline version of SQLite at this point.

SQLite as WASM is a much better solution IMO.

Re: How we sped up Notion in the browser with WASM SQLite

#30

Post by kevingadd »

NelsonMinar wrote: post #29

Why is IndexedDB so bad that you have to load a second database in WASM instead? I've used the API successfully but never measured performance.

It has a weird, verbose API that's more like a simple document store than a SQL database, so it's naturally going to be outperformed by SQLite, especially if you're performing complex operations. It's also heavily async which means operations have to spin the event loop.
Post reply on HN