291–300 of 378 posts
Earlier quoted context omitted.
My standard for any serious service is at least minimal redundancy for improved availability during failures. At least two webservers.
I’m with you, but you could also make the case that most small web service businesses still run a single Postgres instance with no redundancy— just backups. So, you have a single point of failure. You can get quite decent uptime out of a single VPS.
Earlier quoted context omitted.
My standard for any serious service is at least minimal redundancy for improved availability during failures. At least two webservers.
Does this practically improve the situation? The odds of two servers breaking at the same time for the same reasons seems very high. I actually can't think of a single example where the secondary sever would keep running. Regression via a code or dependency update? Full disk? DNS is down? Too much load? All of these would bring down both servers in quick succession. I guess something like a "once every 2 days" race c…
This sentiment pops up regularly on HN, and I've seen at least one article per month for the past few months, but the trouble is, none of them seem to help you actually deploy it. They assume you're comfortable spinning up public web servers. If you want to use a PaaS to deploy an app, because you don't want to spend your time learning to be a sysadmin, then all the tutorials are going to put you on the Postgres path…
> I'm an application developer and I do not want to become a release engineer. I resent even having to learn Docker. :-) Sorry, but that's a terrible attitude to have. You don't need to be a full-blown sysadmin to know how to do basic deployments, and learning these things will make you a better developer.
This sentiment pops up regularly on HN, and I've seen at least one article per month for the past few months, but the trouble is, none of them seem to help you actually deploy it. They assume you're comfortable spinning up public web servers. If you want to use a PaaS to deploy an app, because you don't want to spend your time learning to be a sysadmin, then all the tutorials are going to put you on the Postgres path…
> I'm an application developer and I do not want to become a release engineer. I resent even having to learn Docker. :-) Sorry, but that's a terrible attitude to have. You don't need to be a full-blown sysadmin to know how to do basic deployments, and learning these things will make you a better developer.
Earlier quoted context omitted.
The problem is when you start to approach oltp. This is not a space for sqlite. I have thought about jacking inotify into sqlite_busy_handler() to get the exclusive writers doing better than random waits, but this full api isn't exposed to PHP (where I need it), and doing it at C would suggest alternate approaches, maybe even with xargs at the shell. Oracle has a DBWR process that manages itself, but a write-heavy ap…
I don’t get this. SQLite can perform 500k writes/s (or 5k write txns/s). What app are you building which requires more than this?
The sqlite commit is a specific pattern, it is either write or close/write.
Using inotify events can see these faster than random waits.
AWS has a habit of taking a open source project and creating a "managed service" offering of it. Is it possible to offer SQLite as a managed / serverless offering? A light weight and cheap relational data store that we just consumer using an API
Nope. SQLite is already available in the same process and using the same file system as your server. In some cases (ex Python) without adding any new dependencies. It's downright silly to try to think of a way to make it easier. Here's your easy cheap and lightweight relational datastore API: import sqlite3 conn = sqlite3.connect("db.sqlite3") cur = conn.cursor() cur.execute("SELECT * FROM products LIMIT 25") print(c…
We have a bunch of serverless functions that are generating records that fit a relational data structure. (AWS Lambda)
We would like to write these records to a persistent relational data store so another downstream process can read it.
Amazon RDS seems like an overkill.
Amazon DynamoDB (NoSQL) seems like a misfit because we want to execute relational queries and some joins against this data.
We could write to CSV on S3 and query using Athena/Presto. Seems clunky and slow.
Am I missing any obvious solution here OR there is a space here for a service that offers a lightweight relational datastore that multiple loosely coupled readers and writers can use.
Earlier quoted context omitted.
Well, he did cherry-pick one joking comment out of a well-intentioned post in order to say I have a bad attitude, so that's not exactly welcoming. And very patronizing, considering I have actually done that kind of work with Salt Stack a few years ago, and just don't want to bother with it at the moment because I have other priorities. I learn new things every day, and one day, Docker was that thing. I am now better…
Sure, that comment came a bit rough. But I'm not even sure it was triggered by the Docker comment, rather than the developer/release engineer opposition. At least that's how I see it. And maybe I just don't see the added value of PaaS because I've been spinning up public web servers for too long now ;-)
Any cool, beginner friendly resources you can share? Many thanks
Earlier quoted context omitted.
"Just point it a file" skips all the bits you need for a production system. How do you back it up, replicate it, handle two different processes/containers/servers wanting to access the same data. Using a PAAS solution for a database, you get all that functionality.
It's a file is relevant. It allows you to think of it like other files: - how do you back up a JPG? You copy it, cause it's a file - how do you replicate it? You copy it, cause it's a file. Unless you're talking about fancy DB replication, in which case well, that's not really a thing we do with files much. You'll have to do more research. but that's cause you're trying to do non-file things to a file. - how do you h…
Earlier quoted context omitted.
Nope. SQLite is already available in the same process and using the same file system as your server. In some cases (ex Python) without adding any new dependencies. It's downright silly to try to think of a way to make it easier. Here's your easy cheap and lightweight relational datastore API: import sqlite3 conn = sqlite3.connect("db.sqlite3") cur = conn.cursor() cur.execute("SELECT * FROM products LIMIT 25") print(c…
Hmm. We have a bunch of serverless functions that are generating records that fit a relational data structure. (AWS Lambda) We would like to write these records to a persistent relational data store so another downstream process can read it. Amazon RDS seems like an overkill. Amazon DynamoDB (NoSQL) seems like a misfit because we want to execute relational queries and some joins against this data. We could write to C…
The problem with AWS lambda and SQLite are 1) network file systems often don’t support the APIs that SQLite needs to implement concurrent access; SQLite is not a database server 2) local storage for AWS lambda is ephemeral, your DB will be deleted, and if there’s two lambdas, they won’t be using the same DB.
Just use one EC2 instance and SQLite, or lambda with RDS.
Earlier quoted context omitted.
Does this practically improve the situation? The odds of two servers breaking at the same time for the same reasons seems very high. I actually can't think of a single example where the secondary sever would keep running. Regression via a code or dependency update? Full disk? DNS is down? Too much load? All of these would bring down both servers in quick succession. I guess something like a "once every 2 days" race c…
Zero downtime upgrades, hardware fault, aws decides that specific instance needs to die. It also doesn't let you cheat statelessness very easily, so it's easier to scale horizontally.
I like your statelessness point. I suppose in your view it’s better to have the concentrated stateful core with stateless servers as opposed to just one stateful instance. Two instances mean you can’t easily store foo in memory and hope the server doesn’t die until it’s not needed there anymore. Counterpoint is that the extra layer of indirection is 10x slower and horizontal scaling won’t be needed as much if you don’t pay that price in the first place, but you are right, the temptation to store foo in memory would still be in its prime. The thing is, if one machine can scale, putting foo in memory isn’t actually bad. It’s only when things don’t scale that it’s bad.