I don't get it. Does SQLite run now on the Cloudflare servers, or is it running in the browser via WASM?
On the cloudflare servers, which run server-side JavaScript.
Serverless SQLite
101–110 of 167 posts
Re: Serverless SQLite
#102Nice, you can actually do something a bit more flexible using SQLite on AWS Lambda. "Just" mount an elastic file system (EFS) onto your functions. I wrote about it here: https://franz.hamburg/writing/shared-storage-for-lambda-func...
Thanks for sharing your writeup and the potential use cases therein.
We've been building an experimental shared file system that specifically targets the FaaS setting. It supports SQLite and gets big performance gains over NFS/EFS, especially on read-mostly workloads, due to improved local caching and lock elision. See: https://arxiv.org/abs/2009.09845
Re: Serverless SQLite
#103Re: Serverless SQLite
#104Earlier quoted context omitted.
Also, serverless solutions like DynamoDB or S3 aren't compute, so worker based and FaaS would simply be wrong.
Would we lose any meaning by just calling DynamoDB and S3 managed solutions, like in the old times?
EC2 is a managed solution but you still need to manage servers.
Re: Serverless SQLite
#105https://www.sqlite.org/serverless.html (first published 2007, before the more recent misuse of the term began)
I think I will always struggle to understand the popularity of the term serverless. From the two definitions found in https://www.sqlite.org/serverless.html classic serverless --> "embedded database" exists and is often used neo-serverless --> I suspect that this is a marketing term used to attract cool people to new cloud offerings (like "jamstack" for services such as netlify). There is no good replacement here bec…
Look, the industry has settled on the term "serverless" to mean: "You just upload some application code and the system automatically provisions servers to run it."
No one has the power to change this. It's like saying "I think the word 'spoon' is stupid, let's all call it 'scooper' instead." This is our language now, it is what it is. Vendors offering serverless solutions have to call it "serverless" whether they like it or not because that's the word customers understand. You can keep complaining about it, but it won't change. You might as well accept it and move on.
"Cloud" is a stupid term too, your servers aren't really floating in the sky. But here we are.
Re: Serverless SQLite
#106This is really clever. I've been wanting to try the WASM version of SQLite for something - this is a really smart usage of it. My https://datasette.io/ project is built around a similar idea to this: the original inspiration for it was Zeit Now (now Vercel) and my realization that SQLite read-only workloads are an amazing fit for serverless providers, since you don't need to pay for an additional database server - yo…
Thanks for making Datasette! I've been using it to build a project to make US ranked choice election results more accessible. It's very intuitive and easy to use. The main difficulty for me is how to get around the limitation that the sqlite files have to be colocated on the same server. These datasets can get pretty large, and I can't host them on Github, and since I can't put the datasette db on an S3 bucket, I've…
Hi Picardo!
We are toying around with the idea of launching a Cloud SQLite product targeting this exact use case.
How large are your data sets?
Would you be interested in a quick call to discuss your needs in more detail?
Re: Serverless SQLite
#107Can someone ELI5 as to the significance of this?
CDNs allow serving files closest to the user cutting out 100s of ms of latency. But they are limited to read only files. Cloud flare workers allow running simple programs at the CDN node closest to the user. SQLite is embeddable C code that is like a normal SQL server just without the network parts - just the sync functions embedded into your process. This wraps SQLite with the networking and CDN abilities of cloud f…
Now THAT makes sense! (And, i can now see the benefits of this approach.) Thanks for the explanation!
Re: Serverless SQLite
#108Nice, you can actually do something a bit more flexible using SQLite on AWS Lambda. "Just" mount an elastic file system (EFS) onto your functions. I wrote about it here: https://franz.hamburg/writing/shared-storage-for-lambda-func...
Thanks for sharing your writeup and the potential use cases therein. We've been building an experimental shared file system that specifically targets the FaaS setting. It supports SQLite and gets big performance gains over NFS/EFS, especially on read-mostly workloads, due to improved local caching and lock elision. See: https://arxiv.org/abs/2009.09845
Really interesting.
How do you deal with ephemeral file storage? I guess you actually need the provider (say AWS) to build this into the platform?
Re: Serverless SQLite
#109Earlier quoted context omitted.
Serverless is an adjective so you can slap it everywhere. Worker-based or FaaS (function as a service) don't roll off the tongue that well. I'm not advocating for the term, just trying to explain why it is so popular.
Also, serverless solutions like DynamoDB or S3 aren't compute, so worker based and FaaS would simply be wrong.
They probably call them serverless because you are mainly billed by the number of requests instead of the number of hours it runs for (although for Dynamo, there is a cost per table and r/w)
Re: Serverless SQLite
#110https://www.sqlite.org/serverless.html (first published 2007, before the more recent misuse of the term began)
I think I will always struggle to understand the popularity of the term serverless. From the two definitions found in https://www.sqlite.org/serverless.html classic serverless --> "embedded database" exists and is often used neo-serverless --> I suspect that this is a marketing term used to attract cool people to new cloud offerings (like "jamstack" for services such as netlify). There is no good replacement here bec…
the difference between serverless and managed service is actually that when there are no requests the serverless is offline and not costing you much.