Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless SQLite

sql.lspgn.workers.dev

Post reply on HN
Print view View original

91–100 of 167 posts

Re: Serverless SQLite

#91

Post by tmikaeld »

k__ wrote: post #85
tmikaeld wrote: post #5

I've also experimented with this, unfortunately the 50ms CPU time isn't enough for datasets larger than 1.5MB. And the wasm init add at least 100ms to each request even when "hot" in cache. Also, out of a cost perspective, running a VM with SSD for cheap with SQlite will give much more requests than a CF worker for much less. Adding writes to this is also very limited due to the max 1sec write per KV key limit.

Cloudflare's "Workers Unbound" don't have that limit.

It doesn't but it will have latency when loading in the KV keys for every non-cached request.

Re: Serverless SQLite

#92

Post by moduspol »

ramraj07 wrote: post #64
scary-size wrote: post #58

Earlier quoted context omitted.

This was just a proof-of-concept using the smallest lambdas available. I definitely would opt for a EC2 instance. Though lambda would be enough for something like a configuration service.

Is a lambda easier to configure and set up? With Elastic Beanstalk and flask (and the sample code they provide) I can get an api running in a few minutes..

Both are trivial and take only minutes if you're already familiar with them, but Lambda doesn't involve managing instances, their OSes, or making sure they scale up / down (at the instance level) the way you want.

Re: Serverless SQLite

#93

Post by [deleted] »

ohnemint wrote: post #86
deepstack wrote: post #79

Earlier quoted context omitted.

You hit the nail on the head. It is all about marketing for the new front-end developers who don't want to learn "server side" manipulation db, files, etc. Personally the whole indexdb, localstorage really break the web page as a stateless model. Why do you need save so much local data to just maintain the session? Stop putting everything in the web browser.

I can see the value of serverless in general (not this hack necessarily) in small startups that don't have sysadmins on payroll but still want to rapidly deploy products with confidence that it will just work - all without worrying about infrastructure, scaling, etc. What I'm curious is if (1) serverless is cheaper than hiring competent sysadmins who can maintain the infrastructure instead and (2) are these savings w…

[deleted]

Re: Serverless SQLite

#94

Post by aeyes »

jsf01 wrote: post #4

What would this be best used for?

The data served from the edge can be modified without touching code, just produce a new version of your SQLite database.

A small DB to search in a list of storefronts which typical retail websites have, a database of flags or promotion codes which you don't want to send to the browser are some examples I came up with.

Personally I would write a code generator to embed the data in the code but to each their own.

Re: Serverless SQLite

#95

Post by k__ »

wongarsu wrote: post #90
k__ wrote: post #84

Earlier quoted context omitted.

Also, serverless solutions like DynamoDB or S3 aren't compute, so worker based and FaaS would simply be wrong.

Would we lose any meaning by just calling DynamoDB and S3 managed solutions, like in the old times?

Elasticsearch and RDS are also managed, but not serverless.

Re: Serverless SQLite

#96

Post by ZephyrBlu »

deepstack wrote: post #79
ohnemint wrote: post #73

Earlier quoted context omitted.

I wouldn't be surprised if the term originated as a marketing tactic (e.g., AWS) to seed in people's heads the idea of not having to care about servers

You hit the nail on the head. It is all about marketing for the new front-end developers who don't want to learn "server side" manipulation db, files, etc. Personally the whole indexdb, localstorage really break the web page as a stateless model. Why do you need save so much local data to just maintain the session? Stop putting everything in the web browser.

You don't need local state to maintain a session other than possibly storing a Token, but that's very small.

Storing data locally is often out of convenience or possibly for cached data for offline support.

There are valid reasons for maintaining local state.

Re: Serverless SQLite

#97

Post by johncena33 »

warpech wrote: post #33
donut wrote: post #15

Earlier quoted context omitted.

Where's the SQLite database written? How do you ensure only one writer is modifying it?

Two-phase commit[0] could do the job [0] https://en.m.wikipedia.org/wiki/Two-phase_commit_protocol

How do you do two phase commit on a sqlite database file?

Re: Serverless SQLite

#98

Post by jbverschoor »

have_faith wrote: post #76
onethought wrote: post #67

Earlier quoted context omitted.

Ray tracing in lambda could be cool... perfect candidate for horizontal scale... lambda per pixel?

That's a very expensive way of rendering, not sure I could afford more tha 30 frames per hour.

But think of all the money you'll save by not having to install and manage the server!

Re: Serverless SQLite

#99

Post by justsomeuser »

fumblebee wrote: post #72

Can someone ELI5 as to the significance of this?

CDNs allow serving files closest to the user cutting out 100s of ms of latency.

But they are limited to read only files.

Cloud flare workers allow running simple programs at the CDN node closest to the user.

SQLite is embeddable C code that is like a normal SQL server just without the network parts - just the sync functions embedded into your process.

This wraps SQLite with the networking and CDN abilities of cloud flare, allowing you to http-get a subset of data dynamically using SQL (that would not be possible with the simple “download 100% of file X”).

Re: Serverless SQLite

#100

Post by picardo »

simonw wrote: post #68

This is really clever. I've been wanting to try the WASM version of SQLite for something - this is a really smart usage of it. My https://datasette.io/ project is built around a similar idea to this: the original inspiration for it was Zeit Now (now Vercel) and my realization that SQLite read-only workloads are an amazing fit for serverless providers, since you don't need to pay for an additional database server - yo…

Thanks for making Datasette! I've been using it to build a project to make US ranked choice election results more accessible. It's very intuitive and easy to use. The main difficulty for me is how to get around the limitation that the sqlite files have to be colocated on the same server. These datasets can get pretty large, and I can't host them on Github, and since I can't put the datasette db on an S3 bucket, I've been exploring mounting AWS Elastic File System to the Docker container. Is there a better way?
Post reply on HN