Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless SQLite

sql.lspgn.workers.dev

Post reply on HN
Print view View original

61–70 of 167 posts

Re: Serverless SQLite

#61

Post by kuter »

quietbritishjim wrote: post #59
kuter wrote: post #50

Earlier quoted context omitted.

Yes. Maybe you could increase the sizes of B+ pages ? The only reason that databases use B+ trees rather than a red-black tree or avl trees is because of the overhead of reading data from the hard drive. This would be a interesting hack.

You can increase the page size [1]. That will increase the size of the B tree nodes [2] (and other pages too but that's probably what you want): > The upper bound on [the number of keys on an interior b-tree page] is as many keys as will fit on the page. I think a tricky part of this idea would be the locking. Usually SQLite relies on the locking of the underlying filesystem. You could add your own mechanism that cau…

The clients would just send ajax queries with range headers to get the parts of the database that they want.

It is going to be read only.

Locking wouldn't be problem because in the eyes of the sqlite running in the browser it would have it's own read only copy of the database.

Re: Serverless SQLite

#62

Post by sbergot »

peterkelly wrote: post #2

https://www.sqlite.org/serverless.html (first published 2007, before the more recent misuse of the term began)

I think I will always struggle to understand the popularity of the term serverless.

From the two definitions found in https://www.sqlite.org/serverless.html

classic serverless --> "embedded database" exists and is often used

neo-serverless --> I suspect that this is a marketing term used to attract cool people to new cloud offerings (like "jamstack" for services such as netlify). There is no good replacement here because the term was tied to this new kind of infrastructure really early. But anytime I hear someone repeat "of course serverless does not mean there is no server!" I die a little more.

Naming things is hard but at least we ought to try to come up with terms that are not blatantly misleading.

Re: Serverless SQLite

#63

Post by jbverschoor »

So we start adding sqlite. Then comes some sort of small framework for templating. Then we add session storage. Then we add an api endpoint so we can serve a SPA. I think after that it's really time to create a VM in sqlite so we can emulate linux and run somethings else in the 'serverless' endpoint

Re: Serverless SQLite

#64

Post by ramraj07 »

scary-size wrote: post #58
ramraj07 wrote: post #17

Earlier quoted context omitted.

Did I read it right you get 1 second responses? Why even bother with lambdas? A t3 nano reserved instance costs 2 bucks a month or something! Surely most folks don't anticipate 1000x scaling in minutes? Coupled to elastic beanstalk you can get reaonable scaling as well!

This was just a proof-of-concept using the smallest lambdas available. I definitely would opt for a EC2 instance. Though lambda would be enough for something like a configuration service.

Is a lambda easier to configure and set up? With Elastic Beanstalk and flask (and the sample code they provide) I can get an api running in a few minutes..

Re: Serverless SQLite

#65

Post by tmikaeld »

Cthulhu_ wrote: post #36
tmikaeld wrote: post #5

I've also experimented with this, unfortunately the 50ms CPU time isn't enough for datasets larger than 1.5MB. And the wasm init add at least 100ms to each request even when "hot" in cache. Also, out of a cost perspective, running a VM with SSD for cheap with SQlite will give much more requests than a CF worker for much less. Adding writes to this is also very limited due to the max 1sec write per KV key limit.

Mmm, the only use case I can see is equivalent to an in-memory database, which would be faster than embedding sqlite.

I've also tried full-text-search in worker by pre-indexing the content, works very fast even with a JS engine - less than 5ms to make a search in 5MB of text.

It runs out of CPU-time at 6MB of index though.

There's someone that made a WASM for full-text search too, it's definitely faster and can handle quite a lot more text.

https://github.com/wilsonzlin/edgesearch

Re: Serverless SQLite

#66

Post by bArray »

jbverschoor wrote: post #63

So we start adding sqlite. Then comes some sort of small framework for templating. Then we add session storage. Then we add an api endpoint so we can serve a SPA. I think after that it's really time to create a VM in sqlite so we can emulate linux and run somethings else in the 'serverless' endpoint

It's the future of the web they said!

Seriously though, if it's Truing complete, you know it'll be abused to do something completely unintended, like run a ray tracer or something.

Re: Serverless SQLite

#67

Post by onethought »

bArray wrote: post #66
jbverschoor wrote: post #63

So we start adding sqlite. Then comes some sort of small framework for templating. Then we add session storage. Then we add an api endpoint so we can serve a SPA. I think after that it's really time to create a VM in sqlite so we can emulate linux and run somethings else in the 'serverless' endpoint

It's the future of the web they said! Seriously though, if it's Truing complete, you know it'll be abused to do something completely unintended, like run a ray tracer or something.

Ray tracing in lambda could be cool... perfect candidate for horizontal scale... lambda per pixel?

Re: Serverless SQLite

#68

Post by simonw »

This is really clever. I've been wanting to try the WASM version of SQLite for something - this is a really smart usage of it.

My https://datasette.io/ project is built around a similar idea to this: the original inspiration for it was Zeit Now (now Vercel) and my realization that SQLite read-only workloads are an amazing fit for serverless providers, since you don't need to pay for an additional database server - you can literally bundle your data with the rest of your application code (the SQLite database is just a binary file).

If you want to serve a SQLite API in an edge-closest-to-the-user manner for much larger database files (100MB+) I've had some success running Datasette on https://fly.io/ - which runs Docker containers in a geo-load-balanced manner. I have a plugin for Datasette that can publish database files directly to Fly: https://docs.datasette.io/en/stable/publish.html#publishing-...

Re: Serverless SQLite

#69

Post by mtrycz2 »

jbverschoor wrote: post #63

So we start adding sqlite. Then comes some sort of small framework for templating. Then we add session storage. Then we add an api endpoint so we can serve a SPA. I think after that it's really time to create a VM in sqlite so we can emulate linux and run somethings else in the 'serverless' endpoint

You'll be interested in https://www.destroyallsoftware.com/talks/the-birth-and-death...

Re: Serverless SQLite

#70

Post by simonw »

jsf01 wrote: post #4

What would this be best used for?

This is great for any time you have a mostly read-only database (which is true for most publishing use-cases - anything you might consider a static site generator for) which you want to provide API access to with lightning-fast speed because replies will be served from a CDN edge.
Post reply on HN