Nobody ever got fired for using a struct • Hacker News Board

Re: Nobody ever got fired for using a struct

#41

Post by orthoxerox » Fri, Mar 06, 2026, 5:00 AM UTC

SoftTalker wrote: post #2

> But SQL schemas often look like this. Columns are nullable by default, and wide tables are common. Hard disagree. That database table was a waving red flag. I don't know enough/any rust so don't really understand the rest of the article but I have never in my life worked with a database table that had 700 columns. Or even 100.

It's OLAP, it very common for analytical tables to be denormalized. As an example, each UserAction row can include every field from Device and User to maximize the speed at which fraud detection works. You might even want to store multiple Devices in a single row: current, common 1, 2 and 3.

Re: Nobody ever got fired for using a struct

#42

Post by linolevan » Fri, Mar 06, 2026, 5:01 AM UTC

holden_nelson wrote: post #8

SoftTalker wrote: post #2

> But SQL schemas often look like this. Columns are nullable by default, and wide tables are common. Hard disagree. That database table was a waving red flag. I don't know enough/any rust so don't really understand the rest of the article but I have never in my life worked with a database table that had 700 columns. Or even 100.

https://jimmyhmiller.com/ugliest-beautiful-codebase

This is awesome. Got completely lost reading this and was struggling to figure out where I got this link from. Amazing story.

Re: Nobody ever got fired for using a struct

#43

Post by [deleted] » Fri, Mar 06, 2026, 5:04 AM UTC

vharuck wrote: post #24

SoftTalker wrote: post #2

> But SQL schemas often look like this. Columns are nullable by default, and wide tables are common. Hard disagree. That database table was a waving red flag. I don't know enough/any rust so don't really understand the rest of the article but I have never in my life worked with a database table that had 700 columns. Or even 100.

https://apps.naaccr.org/data-dictionary/data-dictionary/vers... 771 columns (and I've read the definitions for them all, plus about 50 more that have been retired). In the database, these are split across at least 3 tables (registry, patient, tumor). But when working with the records, it's common to use one joined table. Luckily, even that usually fits in RAM.

[deleted]

Re: Nobody ever got fired for using a struct

#44

Post by saghm » Fri, Mar 06, 2026, 5:08 AM UTC

I feel like I'm missing something, but the article started by talking about SQL tables, and then in-memory representations, and then on-disk representation, but...isn't storing it on a disk already what a SQL database is doing? It sounds like data is being read from a disk into memory in one format and then written back to a disk (maybe a different one?) in another format, and the second format was not as efficient as the first. I'm not sure I understand why a third format was even introduced in the first place.

Re: Nobody ever got fired for using a struct

#45

Post by gz09 » Fri, Mar 06, 2026, 5:11 AM UTC

amluto wrote: post #36

There are many systems that take a native data structure in your favorite language and, using some sort of reflection, makes an on-disk structure that resembles it. Python pickles and Java’s serialization system are infamous examples, and rkyv is a less alarming one. I am quite strongly of the opinion that one should essentially never use these for anything that needs to work well at any scale. If you need an industr…

> Depending on your needs, the right tool might be Parquet or Arrow or protobuf or Cap’n Proto

I think parquet and arrow are great formats, but ultimately they have to solve a similar problem that rkyv solves: for any given type that they support, what does the bit pattern look like in serialized form and in deserialized form (and how do I convert between the two).

However, it is useful to point out that parquet/arrow on top of that solve many more problems needed to store data 'at scale' than rkyv (which is just a serialization framework after all): well defined data and file format, backward compatibility, bloom filters, run length encoding, compression, indexes, interoperability between languages, etc. etc.

Re: Nobody ever got fired for using a struct

#46

Post by [deleted] » Fri, Mar 06, 2026, 5:14 AM UTC

dyauspitr wrote: post #25

[flagged]

[deleted]

Re: Nobody ever got fired for using a struct

#47

Post by Spivak » Fri, Mar 06, 2026, 5:19 AM UTC

jayanmn wrote: post #11

ambicapter wrote: post #7

Earlier quoted context omitted.

What's in them?

Property1 to 20 or more is an example. There are better ways to do it but I have seen columns for storing ‘anything’

Sounds like a generic form of single table inheritance. I don't honestly see any other way to do it (punting to a JSON field is effectively the same thing) when you have potentially thousands of parts all with their own super specific relevant attributes.

I've worked on multiple products that have had a concept of "custom fields" who did it this way too.

Re: Nobody ever got fired for using a struct

#48

Post by reverius42 » Fri, Mar 06, 2026, 5:20 AM UTC

jeswin wrote: post #40

sublinear wrote: post #37

Earlier quoted context omitted.

If you don't know enough to design a data structure, requirements are missing and someone talking to the client is dropping the ball big time.

Where did I say any of that? I'm saying that if you care about performance, data structures should be designed with approach specific tradeoffs in mind. And like I've said above, in typical business apps, it's ok to start with data structures because (a) performance is usually not a problem, (b) staying close to the domain is cleaner.

You said: "You can't design a data structure without knowing how you will use it."

But the whole discussion involves knowing how you will use it; the advocacy is for careful consideration of data structures (based on how you will use them) resulting in less pain when designing/choosing algorithms.

Re: Nobody ever got fired for using a struct

#49

Post by fiddlerwoaroof » Fri, Mar 06, 2026, 5:21 AM UTC

sublinear wrote: post #32

gz09 wrote: post #4

Earlier quoted context omitted.

Hi, I'm the author of the article. As to your hard disagree, I guess it depends... While this particular user is on the higher end (in terms of columns), it's not our only user where column counts are huge. We see tables with 100+ columns on a fairly regular basis especially when dealing with larger enterprises.

Can you clarify which knowledge domains those enterprises fall under with examples of what problems they were trying to solve? If it's not obvious, I agree with the hard disagree. Every time I see a table with that many columns, I have a hard time believing there isn't some normalization possible. Schemas that stubbornly stick to high-level concepts and refuse to dig into the subfeatures of the data are often seen fr…

Normalization is possible but not practical in a lot of cases: nearly every “legacy” database I’ve seen has at least one table that just accumulates columns because that was the quickest way to ship something.

Also, normalization solves a problem that’s present in OLTP applications: OLAP/Big Data applications generally have problems that are solved by denormalization.

Re: Nobody ever got fired for using a struct

#50

Post by reverius42 » Fri, Mar 06, 2026, 5:22 AM UTC

jeswin wrote: post #40

sublinear wrote: post #37

Earlier quoted context omitted.

If you don't know enough to design a data structure, requirements are missing and someone talking to the client is dropping the ball big time.

Where did I say any of that? I'm saying that if you care about performance, data structures should be designed with approach specific tradeoffs in mind. And like I've said above, in typical business apps, it's ok to start with data structures because (a) performance is usually not a problem, (b) staying close to the domain is cleaner.