Live data from Hacker News

How we sped up Notion in the browser with WASM SQLite

notion.so

91–100 of 107 posts

Re: How we sped up Notion in the browser with WASM SQLite

#91

Earlier quoted context omitted.

Notion is a great example of my hatred of the current "upgrade or die" delivery method of software. Notion 1.0, when it was mostly text with a unique way to view tabular data, was fast and enjoyable to use. Once it shifted to competing with Atlassian et al, it acquired all the bloat those products already have.

Notion was never fast, and I've been using it on and off for the past 10 years or so.

Compared to a plain text editor, no. Compared to documentation + Access hybrids, I'd say it was good enough. The lack of true offline support has always been a big downside.

And by fast I mostly mean, when you did a / command, the menu came up quickly. Now it's stuffed with so many options as to be useless.

Re: How we sped up Notion in the browser with WASM SQLite

#95

Earlier quoted context omitted.

It has a weird, verbose API that's more like a simple document store than a SQL database, so it's naturally going to be outperformed by SQLite, especially if you're performing complex operations. It's also heavily async which means operations have to spin the event loop.

yeah I gave up on the direct API, I used Dexie for my product. I think it's funny you say a problem is that it's async while another commentor says a problem is it blocks the UI. Maybe it's the worst of both?

Yes, it's the worst of both. Dexie is at the mercy of the aforementioned issues with IDB (differing browser implementations, blocking of the main thread, hard to debug, can't actually guarantee persistence beyond "best effort"). The issues are much more prevalent on mobile browsers in our experience (iOS Safari in particular being the worst), to the point I think we'll need to move off it for our persistence layer in our own app and will unfortunately be forced to require our users to download wrapped apps (Capacitor and Electron) if they want offline persistence guarantees.

But yes, the Dexie API is indeed a much better experience for working with IDB.

Re: How we sped up Notion in the browser with WASM SQLite

#96

Earlier quoted context omitted.

I’d love advice too - I’ve become heavily reliant on Notion’s database-lite features, which is what a lot of the markdown-based apps don’t invest in by default. I’m beginning to get frustrated with loading speeds that feel like spinning disks.

For anything database-lite, https://coda.io is a lot better than notion.

I tried Coda a few months ago. The feature set was very appealing. Coda turned out to be slower and more buggy than Notion, especially when routinely working with databases. The typography and graphic design were also less polished. I kept using Coda until I could no longer tolerate its issues and then have moved everything to Notion.

Re: How we sped up Notion in the browser with WASM SQLite

#97
post #36

Notion takes 15s to load to an empty page. Then another 5 to dismiss the popup about new AI features and the like. I'm glad they are making their app faster, in the meantime I (browser user) have cancelled my team's subscription and will be using something else.

Curious where you're moving? About to begin finding a tool similar to Notion (but not Notion).

Anytype looks promising. It’s at the top of my list of collaborative workspace tools to try.

Re: How we sped up Notion in the browser with WASM SQLite

#98
post #87

Earlier quoted context omitted.

I wanted to try SQLite in the browser for a side project, but the startup time was unacceptable to me. It takes a solid second or two to crunch through the WASM binary, init SQLite, and open a database. Gets worse on first boot when you need to create all the tables. There was a web.dev post or something saying "Web SQL is finally here! Just use SQLite with WASM!" Like, sure, that does seem to work, but it requires a…

SQLite is not SQL standards compliant. It doesn't even support data types from core SQL-92. One needs more than INTEGER, REAL, TEXT, BLOB for applications. Adopting it as WebSQL would have been a terrible mistake.

Is IndexedDB SQL standards compliant? Inventing IndexedDB is a terrible mistake.

Re: How we sped up Notion in the browser with WASM SQLite

#99

Earlier quoted context omitted.

> Meanwhile in AWS you would pay $27k a month to have the same IOPS as a Lenovo Thinkpad X1. This is kind of an unfair comparison. Essentially nobody needs a million iops for their database. Even an extremely busy database doesn't need to scan all of the data it holds (or at least, if it does, you're using it very wrong—that's why we have indexes). A fast disk is possible on a laptop because it's a tiny hop to RAM. A…

Cloud providers have been delivering shit IOPS and charging unhinged prices for basic modern day performance that I really can’t agree with you at all. Nobody “needs” high IOPS the same way nobody needs a car that can drive faster than the 70mph. Why would anyone settle for an artificial ceiling?

It's not an artificial ceiling, because the storage and the compute are decoupled. The disk isn't physically in the server.

Zero times in the half decade that I've been administering terabyte-scale RDS instances has my server had to even restart due to a disk failure. In fact, I never even need to think about disk failures because they're automatically redundant. And if one occurred, nobody needs to crack open the server running my databases to fix anything, because the disk is replicated somewhere else. And if I need more space, I don't need to change the server at all: the volume just scales up (and delivers more iops) because another physical disk somewhere else gets RAIDed in. The cost of addressing the disk over the network is where the iops go.

That's hugely valuable. You simply don't get that if you're building the server yourself (which is really the only time you can achieve millions of iops). But even if you did build and run such a server, I can almost guarantee that you couldn't get a real production database workload to get above a hundred thousand iops unless you delete all of your indexes on terabytes of data. I would bet real money that you can't physically build a server that can maintain 1M iops serving real load on real data without artificial manipulation (both because you don't have a service with enough load to demand iops even close to that and because databases don't pull gigabytes from the disk every second).

Let's say you did build such a server. Now you're CPU bound. You simply can't throw enough CPUs at the database to process all the rows you're scanning at a million iops. Literally: the ability to distribute the rows for processing decreases as the number of cores goes up and you'll hit a ceiling. Bursting up to a million iops for the second needed to fill your RAM is useless if you then spend the next minute processing all that data on 128 cores. And even if you're just serving blobs that don't need processing, you're now network constrained. The only reason a laptop has that many iops is the user physically interacts with the machine, and the goal is to just fill RAM as fast as possible.

It's theoretically possible to build your own extremely optimized purpose-built database that can max out the resources of such a server. But the cost of writing such a server plus the expense of building and racking and maintaining the server will almost certainly scale the cost to an order of magnitude what you'd pay to just horizontally scale that workload to commodity software and cloud infrastructure.

To your analogy, nobody needs a car that can go 400mph. But why would you pay $5M for one? What roads can you drive it on? Why would you want to have to stop for gas or charge every ten miles? Are you okay with seating only the driver with no trunk space? Who is going to insure it?

Re: How we sped up Notion in the browser with WASM SQLite

#100
post #88

Earlier quoted context omitted.

It sucks that even the newest make-wadm-sqlite-fast FileAccessHandles are still an intermediated virtual file system hosted by the browser. I don't get why we have three different file system APIs on the web and none that just use files. Meanwhile WASI has their own server-side file system APIs but neither WASI nor the browser side seem to have any effort to get on the same page. It'll be 2035 before component-model…

The one thing you do NOT want to let websites have access to is files.

I don't see why Origin Private File System needs some fully virtualized nonsense. You can just create some fuck off far out directory for the site for this cause. Should be fine.

There's already pretty safe secure & altogether sensible options for giving websites access to files. Alas it's all via even slower even worse abominations that File Access Handles, because spec authors swear it would be hell on earth if they didn't quintuple buffer every read or write, will swear we could never ever trust devs to ever possibly do a good thing. There's been such a constant hedging that being slow & adding layers of safety to io is necessary and it's taken literally decades to get even these crap ass concessions. I know this is impolite and I'm sorry but what cowardice, what unrelenting sad clutching of pearls. What utter betrayal of Extensibile Web Manifesto. https://github.com/extensibleweb/manifesto

Post reply on HN