Live data from Hacker News

A terrible schema from a clueless programmer

rachelbythebay.com

181–190 of 493 posts

Re: A terrible schema from a clueless programmer

#181

Earlier quoted context omitted.

She has never been "clueless" and has written insightfully for years.

Well she's clueless about what actually sped up this database query.

Without knowing which database engine (MySQL comes with more than one) she was using, nor the testing that she actually performed, what makes you say that?

Re: A terrible schema from a clueless programmer

#183

This post is bizarre, precisely because there is nothing particularly wrong about the original schema, and the author seems to believe that the problem is that the column values were stored as strings, or that the schema wasn't in "third normal form". Which is nonsense. The problem with the original DB design is that the appropriate columns weren't indexed. I don't know enough about the problem space to really know i…

I was kinda surprised by the last chapter, suddenly this is about 'advancing in programming as a woman', and it's over! Like the last few chapters make sense in that view.

What I want to know is how she went from a strong database format (ID + data, keyed) into a weak database model (not a database... is a key-value store) which is likely to be much slower, but also, this is what happens to varchars under the hood already (the rows only hold pointers, to where the strings/objects are placed, rather than inlining varchars) (at least in litesql!), so optimally it's only two dereferences instead of one..

Like, eh, this programmer went through an anti-learning process somehow, and came up with a senseless optimisation. The post ends midway through the learning!

Re: A terrible schema from a clueless programmer

#184

This post is bizarre, precisely because there is nothing particularly wrong about the original schema, and the author seems to believe that the problem is that the column values were stored as strings, or that the schema wasn't in "third normal form". Which is nonsense. The problem with the original DB design is that the appropriate columns weren't indexed. I don't know enough about the problem space to really know i…

I think if anything, all of it could've been put into a single indexed column since the query was AND ... AND ... not OR. So you could've had a indexed column of "fingerprint", like ip1_blahblah_evil@spammer.somewhere_victim1@our.domain And indexed this, with only single WHERE in the query. I don't understand at all how multiple tables thing would help compared to indices, and the whole post seemed kind of crazy to m…

Right, and when you do that, you don't even need a RDBMS. An key-value store would suffice. This essentially just becomes a set! Redis or memcached are battle-tested workhorses that would work even better than a relational DB here.

But this was also back in the early '00s, when "data store" meant "relational DB", and anything that wasn't a RDBMS was probably either a research project or a toy that most people wouldn't be comfortable using in production.

Re: A terrible schema from a clueless programmer

#185

Earlier quoted context omitted.

Origin: https://xkcd.com/356/

Old enough some people under 35 may not have encountered it… ;)

I feel like that phrase has been there forever, and I'm exactly 35. Will this happen more often as I grow older? Ugh. Feels weird. Maybe also a bit depressing.

Re: A terrible schema from a clueless programmer

#186

This post is bizarre, precisely because there is nothing particularly wrong about the original schema, and the author seems to believe that the problem is that the column values were stored as strings, or that the schema wasn't in "third normal form". Which is nonsense. The problem with the original DB design is that the appropriate columns weren't indexed. I don't know enough about the problem space to really know i…

Based on this piece of old documentation I found [0] for MySQL 3.23 (the most recent version in 2002 as far as I can tell), certain types of indices were only available on certain types of engines. Furthermore, columns were restricted to 255 characters, which may be too short for some of the fields saved in the database. Modern databases abstract away a lot of database complexity for things like indices. It's true th…

Based on: https://web.mit.edu/databases/mysql/mysql-3.23.6-alpha/Docs/...

I'd say there's very little performance gain in normalizing (it usually goes the other way anyway: normalize for good design, avoiding storing multiple copies of individual columns; de-normalize for performance).

I'm a little surprised by the tone of the article - sure, there were universities that taught computer science without a database course - but it's not like there weren't practical books on dB design in the 90s and onward?

I guess it's meant as a critique of the mentioned, but not linked other article "being discussed in the usual places".

Re: A terrible schema from a clueless programmer

#187
I'd say that pretty much everyone is a better DB programmer than I am, so I don't really go out of my way to design anything especially robust. I try to design stuff I'm not good at, in a way that makes it easy for someone that knows more than I do, to come in and toss out the work I did, to be replaced with good work.

There's lots of stuff that I'm bad at. I'm also good at a fair bit of stuff. I got that way, by being bad at it, making mistakes, asking "dumb" questions, and seeing how others did it.

Re: A terrible schema from a clueless programmer

#188

Actually, the Correct Answer is a bloom filter. (And, yes, we had math in the early 2000s.) Snark aside, I'm frustrated for the author. Her completely-reasonable schema wasn't "terrible" (even in archaic MySQL)—it just needed an index. There's always more than one way to do something. It's a folly of the less experienced to think that there's only One Correct Way, and it discourages teammates when there's a threat of…

To be fair, the “correct way” to do databases at that time was to use third normal form. Putting indices on string columns would have been considered a hack, much like using MySQL was.

Re: A terrible schema from a clueless programmer

#189
I think there's an easy "best of both worlds" take here:

1. The majority of the performance problem could've and probably should've been summarized as "you need to use an index". (Maybe there were MySQL limitations that got in the way of indexing back then? But these days there aren't.)

2. Everyone makes mistakes! New programmers make mistakes like not knowing about indexes. Experienced programmers make mistakes like knowing a ton about everything and then teaching things in a weird order. All of these things are ok and normal and part of growing, and it's important that we treat each other kindly in the meantime.

Re: A terrible schema from a clueless programmer

#190
Oh well played!! I was setting up to give the author a hard time about being judgemental, particularly because storing IPs or UUIDs as strings is a mistake I've seen some pretty darn good devs make. Some folks just aren't super strong on schema design and performance but are great at other things.

Plus MySQL kind of rocks. Fight me. There are some interesting optimizations that Percona has written about that may improve the performance of OPs schema. The learning never ends.

TBH I haven't seen a shortage of mentors. In fact most of our dev team is 35+ and are very approachable and collegial and our internal mentoring programs have been extremely successful - literally creating world experts in their specific field. I don't think we're unique in that respect.

I think it's rather unfortunate that the top comment here is a spoiler. Actually reading the post and going on the emotional rollercoaster that the author intended to take you on is the point. Not the punch line.

Post reply on HN