Live data from Hacker News

A terrible schema from a clueless programmer

rachelbythebay.com

111–120 of 493 posts

Re: A terrible schema from a clueless programmer

#111
post #15

Feels like you could just concatenate and hash the 4 values with MD5 and store the hash and time. Edit: I guess concatenate with a delimiter if you're worried about false positives with the concat. But it does read like a cache of "I've seen this before" . Doing it this way would be compact and indexed well. MD5 was fast in 2002, and you could just use a CRC instead if it weren't. I suppose you lose some operational…

Isn't that exactly what adding an index would do internally?

Re: A terrible schema from a clueless programmer

#112
post #91

I'd really love to be snarky here but I'll try to be polite: all those comments about the example situation are missing the whole point of the post. And it really worries me that there is a good chunk of the tech workers that just ignores the real meaning of something and just nitpick about stupid implementation details. The post is about managing rookie errors, being empathetic and also warn the ageism that pervades…

I don't buy it. If the only point was managing rookie errors, etc., the blog post shouldn't have been 15~ paragraphs of technical discussion and 2 paragraphs at the bottom of "my point is ..." You can't advocate that people just ignore 80% of the article because they're "not the point of the post." I'm sure a good chunk of tech workers are worried that when an influential blogger writes something like this a couple h…

[deleted]

Re: A terrible schema from a clueless programmer

#113

This post is bizarre, precisely because there is nothing particularly wrong about the original schema, and the author seems to believe that the problem is that the column values were stored as strings, or that the schema wasn't in "third normal form". Which is nonsense. The problem with the original DB design is that the appropriate columns weren't indexed. I don't know enough about the problem space to really know i…

[deleted]

Re: A terrible schema from a clueless programmer

#115
post #8

Sorry, no. The original schema was correct, and the new one is a mistake. The reason is that the new schema adds a great deal of needless complexity, requires the overhead of foreign keys, and makes it a hassle to change things later. It's better to stick the the original design and add a unique index with key prefix compression , which all major databases do these days. This means that the leading values gets compre…

[deleted]

Re: A terrible schema from a clueless programmer

#116
post #15

Feels like you could just concatenate and hash the 4 values with MD5 and store the hash and time. Edit: I guess concatenate with a delimiter if you're worried about false positives with the concat. But it does read like a cache of "I've seen this before" . Doing it this way would be compact and indexed well. MD5 was fast in 2002, and you could just use a CRC instead if it weren't. I suppose you lose some operational…

Isn't that exactly what adding an index would do internally?

2002 MySQL had some limitations on indexing variable length string (varchar) columns. That's the gist of the linked story.

Re: A terrible schema from a clueless programmer

#117

This post is bizarre, precisely because there is nothing particularly wrong about the original schema, and the author seems to believe that the problem is that the column values were stored as strings, or that the schema wasn't in "third normal form". Which is nonsense. The problem with the original DB design is that the appropriate columns weren't indexed. I don't know enough about the problem space to really know i…

I usually store IPv4 as unsigned int, and IPv6 as two 64 bit unsigned ints. But supporting both variants require that I have an a table for each type and a glue table to reference the record in the correct table. Storing simple stuff quickly turns complicated…

Re: A terrible schema from a clueless programmer

#118
One of the most useful lessons I ever learned about designing database schemas was the utter uselessness of indexing and searching on datetime fields. Since virtually every value is going to be different, indexing and searching on that field is (almost) no better than having no index on the datetime field at all.

It was a revelation to me when I decided to experiment with having a indexed date-ONLY field and using that to search instead. It improved performance by almost two orders of magnitude. In hindsight, this should have been totally obvious if I had stopped to think about how indexing works. But, as is stated in this article, we have to keep relearning the same lessons. Maybe I should write an article about how useless it is to index a datetime field...

EDIT: This was something I ran into about 10 years ago. It is possible there was something else going on at the time that I didn't know about that caused the issue I was seeing. This is an anecdote from my past self. I have not had to use this technique since then, and we were using an on-prem server. It's possible that the rest of the table was not designed well and index I was trying to use was already inefficient for the resources that we had at the time.

Re: A terrible schema from a clueless programmer

#119
post #46

Earlier quoted context omitted.

She doesn’t mention the write characteristics of the system but she implies that it was pretty write heavy. In that case it’s not obvious to me that putting a key prefix index on every column is the correct thing to do, because that will get toilsome very quick in high write loads. Given that she herself wrote the before and after systems 20 years ago and that the story was more about everyone having dumb mistakes wh…

It's not an index on every column. It's a single index across all columns.

Which will be updated every time there is a new combination.

Re: A terrible schema from a clueless programmer

#120
post #100

Earlier quoted context omitted.

In what way is the industry designed that way now? Maybe some get sucked into management roles, but I don’t see any evidence of a broad design to push them out.

Look at the way we screen resumes, or valorize work conditions incompatible with having family obligations. Ageism in the tech industry is rife.

Or how leetcode heavy hiring favors folks that tend to be more fresh out of college & folks who have time to prepare for them.
Post reply on HN