Earlier quoted context omitted.
> For reference, reading a million items of up to 1 kB each costs $0.125 with on-demand dynamodb. Is that the same counting method as PlanetScale's "row read"? That is, `select title from posts order by title limit 10` on a table with 10 million rows and no index on `title` would cost $1.25 per query?
DynamoDB doesn't have SQL queries; but yes, if you're performing an operation which reads 10 million items from ddb it will be absurdly expensive. It will also take an absurdly long time; by default ddb is limited to 40k read request units (= 80k eventually consistent reads of up to 1 kB) per second. Being so slow would probably make users realize they're doing something wrong.
Spending $5k to learn how database indexes work
181–190 of 209 posts
Re: Spending $5k to learn how database indexes work
#182As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
Re: Spending $5k to learn how database indexes work
#183As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
> In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. Computers do what you tell them to do. If you are totally clueless and don't bother to take even a few minutes to try to understand a system you are using, the results are going to be poor. Thinking any system can overcome total user ignorance is the thing here that isn't sane. What the…
In theory I agree, but this website features something like "how I nearly bankrupted myself with an AWS bill" on the homepage every month or so. People are blissfully unaware about the extreme costs they're paying to the scaling cloud providers that they often don't even need in the first place.
While I don't think services should block extreme spend all together, a monthly/weekly/daily limit would go a long way to prevent these stories. Very few services that abstract away performance costs have a good way to limit expenses. I don't know if that's intentional or if these companies just don't care, but it's infuriating to me.
It's fine to expose the same tool to both someone who doesn't know the difference between indexes and foreign keys and someone who's been building cloud infra for many years, but as a company you should be prepared to respond to your customers' most likely mistakes. This specific case would probably be hard to detect automatically, but so many wasted CPU cycles, kilowatts and forgiven bills could be prevented if someone would just send an email saying "hey, you've been using more than 10x the normal capacity today, everything alright?"
Re: Spending $5k to learn how database indexes work
#184That pricing model seems rather inherently tricky to me, and also quite expensive. At $1.50 per 10 million rows read this can get very expensive the moment you do a full table scan on any non-trivial table. And while this example is a trivial case where you only need minimal database knowledge to ensure that no full table scan is necessary, many real world cases are much more complex. It also seems very expensive com…
Re: Spending $5k to learn how database indexes work
#185Earlier quoted context omitted.
...and Hetzner just started offering their services in the US a few days ago. (EDIT: not affiliated) If you do something stupid with your code at least you won't go bankrupt, only your service will be slower.
Just to clarify, Hetzner now has Cloud servers in the US. Dedicated servers are still only available in Europe.
Re: Spending $5k to learn how database indexes work
#186Earlier quoted context omitted.
I rented a VPS from the French website (their native language), and it was confusing. I remember looping through the same 2 pages 10 times before actually finding the instance's access instructions. Of course, you don't have to care about the console once you have SSH access to the server. It seems to be pretty good service for the money.
Hetzner is German, not french. Ovh is french.
Re: Spending $5k to learn how database indexes work
#187Re: Spending $5k to learn how database indexes work
#188As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…
+1 on the budget control, but I don't think there are good arguments in favor of slower scaling.
The ability to scale on demand is sold (and bought) based on the expectation that services just meet the workload that's thrown at them without any impact on availability or performance. That's one of the main selling points of managed services, if not the primary selling point.
Arguing in favor of slower scaling implies arguing in favor of downtime. A service that's too slow to scale is a service that requires a human managing it. A managed service that is unable to meet demand fluctuations is a managed service that can't justify the premium that is charged for it.
Re: Spending $5k to learn how database indexes work
#189That is such a bad habit like everyone getting on git and getting burned and now it's irreversible with all the existing ecosystem.
How hard is it to just spin up a beefy cloud instance and run a MySQL of your own with whatever backup strategy you got and do things the way it is than getting bitten by using stuff you're not even familiar with.
Re: Spending $5k to learn how database indexes work
#190Earlier quoted context omitted.
I do use it in production and have for years, just not the online schema changes. It's fantastic and FKs are supported in a single shard, which we use heavily.
If you take out online schema changes and sharding, what's the use case for vitess?