Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless SQLite

sql.lspgn.workers.dev

Post reply on HN
Print view View original

111–120 of 167 posts

Re: Serverless SQLite

#111

Post by justsomeuser »

picardo wrote: post #100
simonw wrote: post #68

This is really clever. I've been wanting to try the WASM version of SQLite for something - this is a really smart usage of it. My https://datasette.io/ project is built around a similar idea to this: the original inspiration for it was Zeit Now (now Vercel) and my realization that SQLite read-only workloads are an amazing fit for serverless providers, since you don't need to pay for an additional database server - yo…

Thanks for making Datasette! I've been using it to build a project to make US ranked choice election results more accessible. It's very intuitive and easy to use. The main difficulty for me is how to get around the limitation that the sqlite files have to be colocated on the same server. These datasets can get pretty large, and I can't host them on Github, and since I can't put the datasette db on an S3 bucket, I've…

Why don't you place them onto an AWS disk and attach it to the VM instance running Datasette?

Re: Serverless SQLite

#112

Post by picardo »

sorenbs wrote: post #106
picardo wrote: post #100

Earlier quoted context omitted.

Thanks for making Datasette! I've been using it to build a project to make US ranked choice election results more accessible. It's very intuitive and easy to use. The main difficulty for me is how to get around the limitation that the sqlite files have to be colocated on the same server. These datasets can get pretty large, and I can't host them on Github, and since I can't put the datasette db on an S3 bucket, I've…

Hi Picardo! We are toying around with the idea of launching a Cloud SQLite product targeting this exact use case. How large are your data sets? Would you be interested in a quick call to discuss your needs in more detail?

hey, thanks. That sounds interesting, but this is a volunteer project. I'm donating my time, so I don't think I can justify a service like that right now. The datasets are around 200MB each, just over the 100MB limit of Github.

Re: Serverless SQLite

#113

Post by picardo »

justsomeuser wrote: post #111
picardo wrote: post #100

Earlier quoted context omitted.

Thanks for making Datasette! I've been using it to build a project to make US ranked choice election results more accessible. It's very intuitive and easy to use. The main difficulty for me is how to get around the limitation that the sqlite files have to be colocated on the same server. These datasets can get pretty large, and I can't host them on Github, and since I can't put the datasette db on an S3 bucket, I've…

Why don't you place them onto an AWS disk and attach it to the VM instance running Datasette?

Yep, that's probably what I'll end up doing.

Re: Serverless SQLite

#114

Post by adolph »

kentonv wrote: post #105
sbergot wrote: post #62

Earlier quoted context omitted.

I think I will always struggle to understand the popularity of the term serverless. From the two definitions found in https://www.sqlite.org/serverless.html classic serverless --> "embedded database" exists and is often used neo-serverless --> I suspect that this is a marketing term used to attract cool people to new cloud offerings (like "jamstack" for services such as netlify). There is no good replacement here bec…

Look, the industry has settled on the term "serverless" to mean: "You just upload some application code and the system automatically provisions servers to run it." No one has the power to change this. It's like saying "I think the word 'spoon' is stupid, let's all call it 'scooper' instead." This is our language now, it is what it is. Vendors offering serverless solutions have to call it "serverless" whether they lik…

“less” has been in use for a long time and typically means a paradigm shift in how the is performed. Horseless carriages still had means of locomotion; driverless cars will still have a means of effecting driver functions; eggless recipes typically have some other substance to perform the job of eggs. In my mind ”less” is not as objectionable as ”-free” like “sugar-free” which isn’t to say that something isn’t sweet, but that the sweetening comes from something else.

Re: Serverless SQLite

#115

Post by avereveard »

geospiza-fortis wrote: post #39

Using SQLite compiled to Wasm in order to push computation closer to the user is a powerful idea. I'm partial to the method of serving up the SQLite files directly and building applications around the SQL.js library [1], which includes math extensions and the ability to embed Javascript udfs. I wrote a data visualization using SQLite as the data store [2] and can attest that it's refreshing to use SQL inside of a sta…

I did exactly that for a minor app I built for a relative (a vocabulary) - a single html page, loading a remote sqlite file, allowing for indexed user searches with no perceivable network latency. the downside is having to transfer 4mb to the client upfront, of course, but gzip shrinks that on the wire down to 1.5mb, which while heavy is acceptable nowadays.

Re: Serverless SQLite

#116

Post by vinay_ys »

adolph wrote: post #114
kentonv wrote: post #105

Earlier quoted context omitted.

Look, the industry has settled on the term "serverless" to mean: "You just upload some application code and the system automatically provisions servers to run it." No one has the power to change this. It's like saying "I think the word 'spoon' is stupid, let's all call it 'scooper' instead." This is our language now, it is what it is. Vendors offering serverless solutions have to call it "serverless" whether they lik…

“less” has been in use for a long time and typically means a paradigm shift in how the is performed. Horseless carriages still had means of locomotion; driverless cars will still have a means of effecting driver functions; eggless recipes typically have some other substance to perform the job of eggs. In my mind ”less” is not as objectionable as ”-free” like “sugar-free” which isn’t to say that something isn’t sweet,…

whereas "sugarless" means its not sweet :)

Re: Serverless SQLite

#117

Post by rakoo »

kuter wrote: post #35

I was thinking of making it possible for SQLite to be used with static pages. My idea is to modify SQLite to use ajax with the HTTP Range header to fetch B+ pages from the server as they are needed. SQLite already has a VFS (virtual file system) so this shouldn't be too hard. I am not sure how fast it would be and it would waste a lot of bandwidth. That's why I haven't made it yet. This would only be useful for using…

Someone has built it already for bittorrent, maybe you can reuse some parts: https://github.com/lmatteis/torrent-net

Even with an artificially slow connection it seems to be reactive enough. Actually fetching pages on-demand can only be better for the bandwidth, the real issue is going to be latency

Re: Serverless SQLite

#118

Post by pjc50 »

kentonv wrote: post #105
sbergot wrote: post #62

Earlier quoted context omitted.

I think I will always struggle to understand the popularity of the term serverless. From the two definitions found in https://www.sqlite.org/serverless.html classic serverless --> "embedded database" exists and is often used neo-serverless --> I suspect that this is a marketing term used to attract cool people to new cloud offerings (like "jamstack" for services such as netlify). There is no good replacement here bec…

Look, the industry has settled on the term "serverless" to mean: "You just upload some application code and the system automatically provisions servers to run it." No one has the power to change this. It's like saying "I think the word 'spoon' is stupid, let's all call it 'scooper' instead." This is our language now, it is what it is. Vendors offering serverless solutions have to call it "serverless" whether they lik…

If only somebody would come up with a Common Interface for application Gateways that would let the webserver dynamically select based on URL routing which executable to provision temporarily to generate that page.

Re: Serverless SQLite

#119

Post by tyingq »

kuter wrote: post #35

I was thinking of making it possible for SQLite to be used with static pages. My idea is to modify SQLite to use ajax with the HTTP Range header to fetch B+ pages from the server as they are needed. SQLite already has a VFS (virtual file system) so this shouldn't be too hard. I am not sure how fast it would be and it would waste a lot of bandwidth. That's why I haven't made it yet. This would only be useful for using…

Dqlite might be interesting to look at. They used the Sqlite VFS to front their raft cluster backend. https://github.com/canonical/dqlite

Re: Serverless SQLite

#120

Post by scrollaway »

pjc50 wrote: post #118
kentonv wrote: post #105

Earlier quoted context omitted.

Look, the industry has settled on the term "serverless" to mean: "You just upload some application code and the system automatically provisions servers to run it." No one has the power to change this. It's like saying "I think the word 'spoon' is stupid, let's all call it 'scooper' instead." This is our language now, it is what it is. Vendors offering serverless solutions have to call it "serverless" whether they lik…

If only somebody would come up with a Common Interface for application Gateways that would let the webserver dynamically select based on URL routing which executable to provision temporarily to generate that page.

CGI is in no way more understandable than serverless as a concept, especially when it shares its acronym with the very popular computer generated imagery.
Post reply on HN