261–270 of 274 posts
Earlier quoted context omitted.
> Make it web scale. Oh god whenever I read/hear that it reminds of the ridiculously funny video about nodejs and Apache servers. [1] [1]. https://youtu.be/bzkRVzciAZg
Glad you liked it! The reference to this series of videos was intended of course. The specific video I had in mind was actually https://youtu.be/b2F-DItXtZs Enjoy!
Earlier quoted context omitted.
How is this handling privledge separation? Typically you'd have all accesses as root in SQLite.
>Typically you'd have all accesses as root in SQLite. So? It's the backend talking to the db.
Your "backend" should not be assumed to be the infallible security. Outside of limiting DB access procledges, we have proper selinux and other configs to lock down the software abilities so even in the case of Arbitrary code execution caused by a common exploit it may be very difficult to use. This is important when you're running a box with multiple other applications with their own DB's as well.
Infact these reasons are why many times the "backend" software and DB are hardware isolated or at least VM isolated so that those db access limitations aren't bypassed.
Earlier quoted context omitted.
> Make it web scale. Oh god whenever I read/hear that it reminds of the ridiculously funny video about nodejs and Apache servers. [1] [1]. https://youtu.be/bzkRVzciAZg
This video is just gold. asyncio is probably one of the worst things that happened to programming in the last 2 decades. It's essentially a step backwards, to the technology of the 90s (cooperative multitasking), when a badly written text editor could cause an entire OS to freeze.
Some advantages: cancellation and error handling are trivial, there's no need for error-prone thread coordination, threads don't have to waste time waiting on locks (because we use asyncio sync primitives instead, so we don't submit work to a thread pool unless it's doable right now with no wait).
Of course, it depends on the tasks being written correctly. But "don't block the event loop" is a much easier rule to follow than "don't deadlock these 10 different threads that share queues and locks".
We didn't write an entire OS with it, but I don't think that was ever the point of asyncio, was it?
It's a "read only" and small website (at least for now), with just a bunch of daily visitors, a perfect use case for SQLite.
Funny thing is that in my case the database it's so small that it's pushed directly on the repo.
Especially for startups and little projects, SQLite is your best friend.
I've been doing some ETL exploration with opening a sqlite :memory: connection, ingesting small-to-medium data, and then doing "VACUUM INTO somefile.sqlite;" to dump the RAM copy to disk. What a great tool.
What does that do? How is VACUUM different from BACKUP in this context?
Earlier quoted context omitted.
Most people don't need everything SQLite offers, but almost all programs needs some of its features. And even if you literally do nothing more than storing rows of text, the API SQLite offers is still more convenient than most filesystem API's once you go beyond the bare minimum. That is to say that there are remarkably few usecases where SQLite isn't better than plain file access.
As I said before, my use cases, while producing useful software, have yet to require more than is offered by a filesystem.
Earlier quoted context omitted.
>Typically you'd have all accesses as root in SQLite. So? It's the backend talking to the db.
Systems of least privilege prevent the levels of access exploitation (either vuln or misconfigure) from going further than is risk assessed. Or from "accidents" like a little utility script nuking the database with root privilege. Or having multiple entry points with their own tables of data. That's the difference between "hackers stole X but not Y and Z" and "hackers dumped everything". Your "backend" should not be…
We've been using SQLite in production as our exclusive means for getting bytes to/from disk for going on 6 years now. To this day, not one production incident can be attributed to our choice of database or how we use it. We aren't using SQLite exactly as intended either. We have databases in the 100-1000 gigabyte range that are concurrently utilized by potentially hundreds or thousands of simultaneous users. Performa…
SQLite is great, but it's not a more simple drop in replacement for DB servers like HN often suggests it is. My team at work has adopted it and generally likes it, but the biggest hurdle we've found is that it's not easy to inspect or fix data in production the way we would with postgres.
The value in SQLite is its light weight, and not it's SQL side. If you're building a mobile app and you're loading a lot of local data, it might be the right choice.
SQLite is great, but it's not a more simple drop in replacement for DB servers like HN often suggests it is. My team at work has adopted it and generally likes it, but the biggest hurdle we've found is that it's not easy to inspect or fix data in production the way we would with postgres.
I'd say the team is using it wrong. SQLite is really intended for embedded use, not a Postgres replacement. The two shouldn't even be mentioned in the same sentence. SQLite is weakly typed, performing autoconversion from ints to strings. The value in SQLite is its light weight, and not it's SQL side. If you're building a mobile app and you're loading a lot of local data, it might be the right choice.
We are using it as a replacement for RocksDB - we need a richer way to store data than a simple key value store. It still runs on a server though, and therefore it would be useful to be able to read data remotely, even if that isn't the primary purpose.