Live data from Hacker News

Serverless SQLite

sql.lspgn.workers.dev

161–167 of 167 posts

Re: Serverless SQLite

#161

Earlier quoted context omitted.

It doesn't but it will have latency when loading in the KV keys for every non-cached request.

Don't KV loads not count towards the CPU time of the request?

Loading a KV does not take CPU time, however, loading a KV into a variable does parse the content of the KV which does consume CPU time.

Which is why it takes a lot less CPU time to do streaming and even to some degree, a streaming parsing of the content.

Re: Serverless SQLite

#162

Earlier quoted context omitted.

Thanks for making Datasette! I've been using it to build a project to make US ranked choice election results more accessible. It's very intuitive and easy to use. The main difficulty for me is how to get around the limitation that the sqlite files have to be colocated on the same server. These datasets can get pretty large, and I can't host them on Github, and since I can't put the datasette db on an S3 bucket, I've…

Hi Picardo! We are toying around with the idea of launching a Cloud SQLite product targeting this exact use case. How large are your data sets? Would you be interested in a quick call to discuss your needs in more detail?

FYI, I'm using Google Cloud Run for this at the moment. Scales to zero, issues the SSL cert and provides the HTTPS endpoint. That's what I'd be benchmarking your service against. Let me know if you actually do something in this space.

Re: Serverless SQLite

#163

So we start adding sqlite. Then comes some sort of small framework for templating. Then we add session storage. Then we add an api endpoint so we can serve a SPA. I think after that it's really time to create a VM in sqlite so we can emulate linux and run somethings else in the 'serverless' endpoint

Lol I did this (sqlite via rest, custom urls, static file hosting) plus authentication/authorization and smtp with email templates. Everything needed for SPAs, no one was interested.

Well, now I'm interested. Link it, please? I'd love to take a look. Sounds really good!

Re: Serverless SQLite

#164
post #16

Earlier quoted context omitted.

> SQLite was already serverless. Yes but now it runs on someone else server! Wait a minute, this makes no sense... 'serverless' is perhaps the stupidest marketing buzzword developers have come up with.

Indeed it's quite stupid. Any alternative catchy name for the concept?

NoSrv

Re: Serverless SQLite

#165
post #33

Earlier quoted context omitted.

Two-phase commit[0] could do the job [0] https://en.m.wikipedia.org/wiki/Two-phase_commit_protocol

How do you do two phase commit on a sqlite database file?

I don't know SQLite specifically, but I think that it can be implemented in the application logic using the primitives of any RDBMS. This old thread suggests that http://sqlite.1065341.n5.nabble.com/Re-Two-Phase-commit-usin...

Re: Serverless SQLite

#166
post #68

This is really clever. I've been wanting to try the WASM version of SQLite for something - this is a really smart usage of it. My https://datasette.io/ project is built around a similar idea to this: the original inspiration for it was Zeit Now (now Vercel) and my realization that SQLite read-only workloads are an amazing fit for serverless providers, since you don't need to pay for an additional database server - yo…

Thanks for making Datasette! I've been using it to build a project to make US ranked choice election results more accessible. It's very intuitive and easy to use. The main difficulty for me is how to get around the limitation that the sqlite files have to be colocated on the same server. These datasets can get pretty large, and I can't host them on Github, and since I can't put the datasette db on an S3 bucket, I've…

Google Cloud Run works well with database files up to a couple of GBs. Beyond that the best solution at the moment is to run a VPS - I've had good results with DigitalOcean droplets.

Re: Serverless SQLite

#167

Earlier quoted context omitted.

Lol I did this (sqlite via rest, custom urls, static file hosting) plus authentication/authorization and smtp with email templates. Everything needed for SPAs, no one was interested.

Well, now I'm interested. Link it, please? I'd love to take a look. Sounds really good!

I took it down, let the domain name lapse. It was a depressing, eye opening experience to say the least.
Post reply on HN