Live data from Hacker News

Spending $5k to learn how database indexes work

briananglin.me

201–209 of 209 posts

Re: Spending $5k to learn how database indexes work

#201

Ssh into an ec2 instance, install mysql, and you'll never pay more than $7.50 a month!

You need to spend a lot more on AWS if you want good performances.

But I'm talking about the contrapositive, where if you don't need good performance, you don't need to spend 5k.

Re: Spending $5k to learn how database indexes work

#202

Earlier quoted context omitted.

Rent metal and run your own MySQL/Postgres/... One insert every 3 seconds. Could run that off a 10 year old laptop.

They clearly don't have the skills for that. And at one insert every 3s, a managed service like RDS won't really cost much.

... until you forget to create an index, apparently.

Re: Spending $5k to learn how database indexes work

#203

As the author touches on, the main problem here isn't learning about indexes. It's about "infinity scaling" working too well for people who do not understand the consequences. In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. I am a strong believer that services that are based on "scale infinitly" really need hard budget controls, and sl…

If you create an inefficient process, you should be responsible for the consequences. Why would you expect some third party to take the responsibility? If you create a horrible internal combustion engine, your gas station should not bear the costs.

If you create an inefficient internal combustion engine, you'd know because you have to go to the gas station every 5 miles. In this case it would be like someone was filling up the the gas without you knowing, and then a few weeks later you get the bill, and then you realize that your engine is inefficient.

Re: Spending $5k to learn how database indexes work

#204
post #99

Earlier quoted context omitted.

I am somewhat shocked to find that an RDBMS is considered a “loaded footgun” in 2021. Perhaps grandparent isn’t the most charitable in their interpretation, but I am in full agreement. It continues to astound me how little about the basics of databases most developers know, and how strongly resistant they are to trying to learn.

An RDBMS that scales infinitely while charging you per-row goes against the usual assumptions learned in the past decades, so I'd say yes that's a loaded footgun.

Have a friend who had a BigQuery query that ended up costing $3k each time. It ran for only a minute, because BigQuery chews through data really fast. But you don't realize that when you push the run query button. And there's no spend guard rails. They switched to pay for given amount of parallelism after that.

Re: Spending $5k to learn how database indexes work

#205

Earlier quoted context omitted.

Hetzner is German, not french. Ovh is french.

Sure, but who is Hetzner? My comment was about OVH.

Omg. Hetzner is similar to OVH and DigitalOcean.

It seems that i misread somewhere? Weird

I think the sentence: how can DO and Vultr even compete was my interpretation of the better offering of Hetzner ( including uptime). Sorry

Re: Spending $5k to learn how database indexes work

#206

Earlier quoted context omitted.

The question that follows would be: how do you know what was intended to be less performant versus optimized on-demand? The intentions can be easily inferred when the query at hand was a simple join, and to no surprise, many cloud database offerings _do_ provide optimization automation (Azure SQL will for example even automatically add obvious indexes if you let it). But what if the query did need to scan all the row…

> The question that follows would be: how do you know what was intended to be less performant versus optimized on-demand? I'm saying that the cloud provider shouldn't try to make assumption either way, and I'm definitely not saying that it should try to manage indexes for you. If you are typically using X ops/s, and begin using 50X ops/s, the default should not be "this customer probably wants to spend 50x their prev…

[deleted]

Re: Spending $5k to learn how database indexes work

#207

Earlier quoted context omitted.

> In no sane version of the world should "not adding a db index" lead to getting a 50x bill at the end of the month without knowing. Computers do what you tell them to do. If you are totally clueless and don't bother to take even a few minutes to try to understand a system you are using, the results are going to be poor. Thinking any system can overcome total user ignorance is the thing here that isn't sane. What the…

This is a lot of victim-blaming in a such a small response. > If you are totally clueless and don't bother to take even a few minutes to try to understand a system you are using, the results are going to be poor. Having a hosted system which behaves different than the underlying technology it's modelled on is not immediately clear. The realm of "things you don't know that you don't know" expands drastically with mana…

> Having a hosted system which behaves different than the underlying technology it's modelled on is not immediately clear. The realm of "things you don't know that you don't know" expands drastically with managed services.

So don't use managed services? They are expensive and the only thing that works consistently and well is the lock in, everything else is pretty iffy. Somehow people look at me like an idiot when I say this, but it's LESS effort to NOT use AWS and build everything yourself. I guess this seems impossible somehow, but at the scale you are ever going to operate it's not hard to just build a service to store and serve files (s3), and if you scale to the point where you can't build it easily, you will build it anyway because you can afford to hire enough engineers to build it and still save huge amounts of money. The same goes for every managed service offered on the cloud, they are not a good deal at any point, ever, for anybody.

> It's never been suggested that this is possible.

The gist of the article is they got a refund because they didn't bother to pay attention close enough to realize their queries were doing full table scans, and they didn't bother to pay close enough attention to realize this was causing the service to scale in capacity to an absurd degree.

Re: Spending $5k to learn how database indexes work

#208
post #90

I'm just leaving this here: https://www.hetzner.com/dedicated-rootserver/ax161/configura... Draw that nice red slide all the way to the right. No, it's not storage. Yeah, it's actually affordable. Yeah, that was a sexual sound you just made. You do have to be prepared to know some basic sysadmin, or pay somebody to do it for you. My newest server has about 60 cores and half a tera of ram. Surprisingly, it's not uber…

...and Hetzner just started offering their services in the US a few days ago. (EDIT: not affiliated) If you do something stupid with your code at least you won't go bankrupt, only your service will be slower.

I used their VPS service before and I didn't have to pay for overage, but now that it's part of their cloud offering, overage is charged.

Re: Spending $5k to learn how database indexes work

#209
post #174

Earlier quoted context omitted.

That's why we use DigitalOcean a lot in Africa. You know upfront how much you will spend.

You can calculate how much RDS is gonna cost you per month beforehand. In fact, it is slightly cheaper at AWS. Ondemand PostgreSQL, Single Node, 1vCPU, 1GB MEM, 10GB Storage is $15 at DO Ondemand PostgreSQL, Single Node, 2vCPU, 1GB MEM, 10GB Storage is $14.29 at AWS (db.t3.micro at us-east-2) if reserved for 1yr no upfront Reserved PostgreSQL, Single Node, 2vCPU, 1GB MEM, 10GB Storage is $10.57 at AWS (db.t3.micro at…

My experience is that on AWS there are hidden costs. Paying for traffic, and other stuff.
Post reply on HN