Live data from Hacker News

Spending $5k to learn how database indexes work

briananglin.me

181–190 of 209 posts

Re: Spending $5k to learn how database indexes work

#181
post #143

Earlier quoted context omitted.

> For reference, reading a million items of up to 1 kB each costs $0.125 with on-demand dynamodb. Is that the same counting method as PlanetScale's "row read"? That is, `select title from posts order by title limit 10` on a table with 10 million rows and no index on `title` would cost $1.25 per query?

DynamoDB doesn't have SQL queries; but yes, if you're performing an operation which reads 10 million items from ddb it will be absurdly expensive. It will also take an absurdly long time; by default ddb is limited to 40k read request units (= 80k eventually consistent reads of up to 1 kB) per second. Being so slow would probably make users realize they're doing something wrong.

[deleted]

Re: Spending $5k to learn how database indexes work

#182

As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…

A billing limit feature is something that's been wanted for years, yet the most that's offered is budget alerts.

Re: Spending $5k to learn how database indexes work

#183

As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…

> In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. Computers do what you tell them to do. If you are totally clueless and don't bother to take even a few minutes to try to understand a system you are using, the results are going to be poor. Thinking any system can overcome total user ignorance is the thing here that isn't sane. What the…

> Computers do what you tell them to do. If you are totally clueless and don't bother to take even a few minutes to try to understand a system you are using, the results are going to be poor. Thinking any system can overcome total user ignorance is the thing here that isn't sane.

In theory I agree, but this website features something like "how I nearly bankrupted myself with an AWS bill" on the homepage every month or so. People are blissfully unaware about the extreme costs they're paying to the scaling cloud providers that they often don't even need in the first place.

While I don't think services should block extreme spend all together, a monthly/weekly/daily limit would go a long way to prevent these stories. Very few services that abstract away performance costs have a good way to limit expenses. I don't know if that's intentional or if these companies just don't care, but it's infuriating to me.

It's fine to expose the same tool to both someone who doesn't know the difference between indexes and foreign keys and someone who's been building cloud infra for many years, but as a company you should be prepared to respond to your customers' most likely mistakes. This specific case would probably be hard to detect automatically, but so many wasted CPU cycles, kilowatts and forgiven bills could be prevented if someone would just send an email saying "hey, you've been using more than 10x the normal capacity today, everything alright?"

Re: Spending $5k to learn how database indexes work

#184

That pricing model seems rather inherently tricky to me, and also quite expensive. At $1.50 per 10 million rows read this can get very expensive the moment you do a full table scan on any non-trivial table. And while this example is a trivial case where you only need minimal database knowledge to ensure that no full table scan is necessary, many real world cases are much more complex. It also seems very expensive com…

I agree with this. You are also only one bug in the query planner away from going bankrupt. Imagine Planetscale upgrading to a version which contains a small edge-case bug and now you owe them tens of thousands because of it.

Re: Spending $5k to learn how database indexes work

#185
post #104
post #90

Earlier quoted context omitted.

...and Hetzner just started offering their services in the US a few days ago. (EDIT: not affiliated) If you do something stupid with your code at least you won't go bankrupt, only your service will be slower.

Just to clarify, Hetzner now has Cloud servers in the US. Dedicated servers are still only available in Europe.

[deleted]

Re: Spending $5k to learn how database indexes work

#186

Earlier quoted context omitted.

I rented a VPS from the French website (their native language), and it was confusing. I remember looping through the same 2 pages 10 times before actually finding the instance's access instructions. Of course, you don't have to care about the console once you have SSH access to the server. It seems to be pretty good service for the money.

Hetzner is German, not french. Ovh is french.

Sure, but who is Hetzner? My comment was about OVH.

Re: Spending $5k to learn how database indexes work

#188

As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…

> I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and slower-scaling (unless explicitly overidden/allowed, of course).

+1 on the budget control, but I don't think there are good arguments in favor of slower scaling.

The ability to scale on demand is sold (and bought) based on the expectation that services just meet the workload that's thrown at them without any impact on availability or performance. That's one of the main selling points of managed services, if not the primary selling point.

Arguing in favor of slower scaling implies arguing in favor of downtime. A service that's too slow to scale is a service that requires a human managing it. A managed service that is unable to meet demand fluctuations is a managed service that can't justify the premium that is charged for it.

Re: Spending $5k to learn how database indexes work

#189
Why do people get on stuff some known people use blindly?

That is such a bad habit like everyone getting on git and getting burned and now it's irreversible with all the existing ecosystem.

How hard is it to just spin up a beefy cloud instance and run a MySQL of your own with whatever backup strategy you got and do things the way it is than getting bitten by using stuff you're not even familiar with.

Re: Spending $5k to learn how database indexes work

#190

Earlier quoted context omitted.

I do use it in production and have for years, just not the online schema changes. It's fantastic and FKs are supported in a single shard, which we use heavily.

If you take out online schema changes and sharding, what's the use case for vitess?

Architected correctly, there's minimal need for cross-shard foreign keys. A common use case is sharding by tenant/customer id, which means that all records for a single customer live on a single shard. That lets you have all the FKs that you want, and any operations for that customer happen on a single shard, which gives you maximum speed and transactional guarantees.
Post reply on HN