[deleted]
Spending $5k to learn how database indexes work
191–200 of 209 posts
Re: Spending $5k to learn how database indexes work
#192Earlier quoted context omitted.
If you take out online schema changes and sharding, what's the use case for vitess?
If you take sharding out what's the point of using anything other than a normal sql database?
Vitess offers a lot of quality of life improvements over stock MySQL, including built-in backups to S3/GCS, managed replication, plus soon to be auto-failover detection. Additionally, with vreplication, you can do some pretty powerful materialized views that aren't possible with MySQL. Finally, Vitess Messaging is an awesome way to do async work, allowing for transaction guarantees where you can ack a message, do data work, then add to another queue, all without having to deal with weird side effects.
Re: Spending $5k to learn how database indexes work
#193That pricing model seems rather inherently tricky to me, and also quite expensive. At $1.50 per 10 million rows read this can get very expensive the moment you do a full table scan on any non-trivial table. And while this example is a trivial case where you only need minimal database knowledge to ensure that no full table scan is necessary, many real world cases are much more complex. It also seems very expensive com…
I agree with this. You are also only one bug in the query planner away from going bankrupt. Imagine Planetscale upgrading to a version which contains a small edge-case bug and now you owe them tens of thousands because of it.
If we caused a bug that did that we would refund the customer of course.
Re: Spending $5k to learn how database indexes work
#194As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
> I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and slower-scaling (unless explicitly overidden/allowed, of course). +1 on the budget control, but I don't think there are good arguments in favor of slower scaling. The ability to scale on demand is sold (and bought) based on the expectation that services just meet the workload that's thrown at them without a…
I may have not been as clear as I should have; I'm not necessarily arguging that typical, or expected scaling action should be slowed down. Ie, throttling scaling from X -> 1.5X doesn't really make sense.
A scaling change that would be considered anomalous, and introduces an order-of-magnitude change over historical usage could be scaled more slowly.
> Arguing in favor of slower scaling implies arguing in favor of downtime.
Sure, I guess that in a limited scope, that is what I am saying. I would much rather have a short-term "downtime that requires human intervention" problem, then a long term "Johnny deployed bad code and now the company is bankrupt" problem.
> The ability to scale on demand is sold (and bought) based on the expectation that services just meet the workload that's thrown at them without any impact on availability or performance. That's one of the main selling points of managed services, if not the primary selling point.
I tend to disagree with this. Managed services are often bought on the expectation that they do not require management, deep operational knowledge, and are reliable. There's also often the trade off of upfront costs (either human or capex costs).
Scalability of obviously part of the analysis, but "scability" and "the ability to scale from 1X -> 100X in a couple seconds" are not necessarily the same thing.
Re: Spending $5k to learn how database indexes work
#195As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
> In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. Computers do what you tell them to do. If you are totally clueless and don't bother to take even a few minutes to try to understand a system you are using, the results are going to be poor. Thinking any system can overcome total user ignorance is the thing here that isn't sane. What the…
This is a lot of victim-blaming in a such a small response.
> If you are totally clueless and don't bother to take even a few minutes to try to understand a system you are using, the results are going to be poor.
Having a hosted system which behaves different than the underlying technology it's modelled on is not immediately clear. The realm of "things you don't know that you don't know" expands drastically with managed services.
> Thinking any system can overcome total user ignorance is the thing here that isn't sane.
It's never been suggested that this is possible. There is a large range of options in between "solve all user error" and "don't hand everyone a loaded foot-gun".
Re: Spending $5k to learn how database indexes work
#196Earlier quoted context omitted.
> I know we disagree, I know you're going to continue disagreeing, I know I don't want to have the conversation. Please consider not chiming in on the next article like this then. I think your attitude of (paraphrasing) "no good programmer would have made the costly mistake you shared, and articles about it aren't worthwhile" is super harmful to our industry. It's the polar opposite of the blameless postmortem approa…
This one in particular is not worthwhile on the front page of HN, that's my take. They're most definitely useful for beginners, or maybe people just learning about databases. I'm not going to not post simply because you find it disagreeable, there are plenty of people here who seem to agree with me. Blameless post mortems are great, for your team. I am not his team mate, and I don't really feel a kinship with every d…
Your submissions are nothing but "ask HN". Leech. To chide about "not knowing" but then yourself ask the community seems a bit hypocritical.
Re: Spending $5k to learn how database indexes work
#197Earlier quoted context omitted.
This one in particular is not worthwhile on the front page of HN, that's my take. They're most definitely useful for beginners, or maybe people just learning about databases. I'm not going to not post simply because you find it disagreeable, there are plenty of people here who seem to agree with me. Blameless post mortems are great, for your team. I am not his team mate, and I don't really feel a kinship with every d…
Your submissions are nothing but "ask HN". Leech. To chide about "not knowing" but then yourself ask the community seems a bit hypocritical.
I asked before making a mistake, I also asked to do some light market validation. Try to keep the ad hominems down. If you can't make an argument against the point and have to attack me as a person, it's just validation. Most of my contributions here are in my upvoted comments like my op, which clearly a good amount of people share.
Generally speaking I don't want to submit most things, largely and because like this post I find most thoughts people have to be garbage, including my own, most things and people simply aren't interesting or useful.
Re: Spending $5k to learn how database indexes work
#198As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
It can't be like that. I have discussions with vendors sometimes and the first question I ask - if something lapses and we weren't paying attention - you won't cut our service right? I think too, in most cases, people would rather run over than cut service. Also how would such a system work? Let's say you sign up for some API and what, set your billing limit to 500 requests per day. Let's say you're now hitting fabul…
It's important to think about this in an a la carte design, not one fixed solution for all use cases.
Step 1: You give people the ability to put in soft limits - "Warn me when I hit 500",
Step 2: You also give the ability to put in hard limits "Pull the plug at 10k" (caveat to both these things - you guarantee this at an eventual consistency level, like "Well you hit 500 but by the time our stats updated you were at 600",
Step 3: You introduce rate limits - "We're expecting 500 in a month, warn us if we hit 50 in a day or 10 in an hour".
Step 4: You introduce predictive warnings "Our statistics show you'll hit your monthly limit on the 23rd of the month"
Step 5: You put in predictive limits to allow scaling - "The last 3 months we've seen the following use trend, warn us if we exceed double that trend, cut off if we see 50x that trend"
You might set some of these limits or none of these limits depending how predictable your use case is.
Re: Spending $5k to learn how database indexes work
#199As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
If you create an inefficient process, you should be responsible for the consequences. Why would you expect some third party to take the responsibility?
If you create a horrible internal combustion engine, your gas station should not bear the costs.
Re: Spending $5k to learn how database indexes work
#200As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
If you create an inefficient process, you should be responsible for the consequences. Why would you expect some third party to take the responsibility? If you create a horrible internal combustion engine, your gas station should not bear the costs.
In theory yes; in practice it’s very easy to push inefficient code to production by accident, as shown in the article.