Live data from Hacker News

Spending $5k to learn how database indexes work

briananglin.me

31–40 of 209 posts

Re: Spending $5k to learn how database indexes work

#31

I've seen things you people wouldn't believe. Millions burnt on consultants and licensing Oracle. I watched C series startups throwing it all away in a move to NoSQL. All those Amazon RDS fees will be lost in time.

Like tears in the rain

These two might just be the best comments i have ever seen on HN

Re: Spending $5k to learn how database indexes work

#32
post #6

Earlier quoted context omitted.

It's not quite so crazy when you phrase it as $0.15 for reading a million items from the database .

No that's still crazy. Scanning a million items isn't a big workload.

I'm not saying that it's not crazy. Just that it's less crazy.

Re: Spending $5k to learn how database indexes work

#33
post #27

Earlier quoted context omitted.

It's better than ads though.

It's better than targetted ads built on intrusive tracking that also enables several other abusive business practices. It's not better than "good old fashioned ads". Heck I'd even be ok with targeted ads if it could be done without the rest of the "destroy civil society" that seems to come along for the ride.

Even if you solve the privacy problem, there's still a problem with advertising which is that it's inherently at odds with the user's interests.

An advertising-funded product will always prioritize engagement - they want you to "engage" with the product even if it means degrading the experience intentionally such as making a process manual or take more steps than necessary (so that you are exposed to more ads). The "destroy civil society" problem you mention is a direct consequence of the pursuit of engagement.

In contrast, with a paid product, the company's interests are directly aligned with yours and they have no incentive to intentionally degrade the experience or get in your way any more than necessary. They don't care about how much you "engage" with the product as long as the bill gets paid (if anything, the less you engage the better as it uses less server resources).

Re: Spending $5k to learn how database indexes work

#34
The real answer here is cost limiting. I don't want my cloud provider to keep working at the cost of an order of magnitude higher bill than I was expecting because of a bug in my code. I want to be able to set a billing limit and have them degrade or stop there service if I exceed the limit.

AFAIK AWS doesn't have that. They do have the ability to send me alerts if my bill is unexpectedly high, but they still keep working until I go bankrupt. It's possible to use those alerts to implement your own "broke man's switch", but they don't have it built in.

Re: Spending $5k to learn how database indexes work

#35

I'm not a DB expert, but "750k users in a month." doesn't sound like a quantity that you'd need to use some kind of fancy special tooling for.

It's far from big data even if they grow 10x in the next year, but if you are unfamiliar with database migrations and branches I can see the appeal of the product.

Re: Spending $5k to learn how database indexes work

#36
post #14

I feel like indexes are a pretty fundamental type of DB knowledge. In fact I'd say it's table stakes knowledge you should have if you're working with them. Further more, knowing that ForeignKeys typically apply an index to that column is also in my head basic knowledge. I'm sorry you got burnt, and congrats on learning a lesson, but you could have gotten the same knowledge by ever googling MySql ForeignKeys and saved…

> I'm not entirely sure why buzz around "developer learns basic knowledge" has this on the front page. The problem is that in the old days, not knowing about indexes left you with an underperforming system or downtime. But in The Cloud™ it leaves you with an unreasonably huge bill and that somehow as an industry we're accepting this as normal.

Which really is a head scratcher. You'd figure especially as a startup seeing a 5k oopsie isn't really as acceptable. Mistakes do happen and I don't mean any shade to this particular person (they'll never make this mistake again) but as an industry the aggregate consequence of this is you have a lot of waste and stupid choices that then have to be cleaned up when more knowledgeable (read highly paid) people are introduced later on.

They'll have to clean up the mess which causes real business consequences that, and I've personally seen this, will directly impact bottom line and have no quick or easy solution to wiggle out of.

Maybe it's acceptable for products like this because the balance between good engineering and company health probably aren't as cut and clear but stuff like this always makes me sad because it's such low hanging fruit, it doesn't require any real effort, just basic curiosity around your job.

Re: Spending $5k to learn how database indexes work

#37
post #17

Don’t use DB providers that charge for rows/data scanned. Use Amazon RDS or Google Cloud SQL or just install it yourself on a VM. Pay for CPU, memory, and storage instead.

Rows returned model works really well for certain data loads (where all data customers use is customer-keyed)....

This model also scales DOWN really well .. while still providing good scalable availability...

That said, I DO agree with the sentiment of paying for a set performance level (clu, memory, storage), to provide predictable pricing.. obviously these guys were bit by the scaling capability.

I do a lot of pet projects, and I find DynamoDB works really well because my pet projects cost $0 most months... And I don't have to worry about servers, maintenance, or what not... I'm happy to do that at work, but I don't want that for my friends & fun projects... And I've not seen a decent DB managed RDS for <$5/month

Re: Spending $5k to learn how database indexes work

#38
post #6

0,15 $ per query... The world has gone insane.

It's not quite so crazy when you phrase it as $0.15 for reading a million items from the database .

One of my employers was using BigQuery. I was so scared that I might accidentally run queries and get a big bill, even though our tables weren’t that big.

It is funny to look back, but getting huge bills without even realizing that we’re doing something wrong is very real possibility. Cloud vendors happily make their pricing opaque as it benefits them.

I’d avoid even the best product in the industry, if their pricing is opaque. Or if there is a “Contact Us” button when there needn’t be.

Re: Spending $5k to learn how database indexes work

#39
post #10

No foreign keys to make migrations easier. That doesn't sound like the best trade off to me. Having the database constrained as much as possible makes maintenance so much easier. Many bugs don't escape into production as they're caught by the database constraints. Those that do get out do less damage to the data. I know scale comes with trade offs but that seems extreme to me.

It's possible during a migration to drop a constraint, make the update and restore the constraint. If a schema migration tool doesn't automate this or at least permit it, it's not a good schema migration tool.

Re: Spending $5k to learn how database indexes work

#40
That pricing model seems rather inherently tricky to me, and also quite expensive. At $1.50 per 10 million rows read this can get very expensive the moment you do a full table scan on any non-trivial table. And while this example is a trivial case where you only need minimal database knowledge to ensure that no full table scan is necessary, many real world cases are much more complex.

It also seems very expensive compared to just renting DBs by instance, if you put any real load onto this. I can see this being attractive if your use case only queries single rows by key, but it's essentially a big minefield for any query more complex than that. A database with a rather opaque query planner doesn't seem like a good fit for this kind of pricing.

Post reply on HN