Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"We ran out of columns"

jimmyhmiller.github.io

Post reply on HN
Print view View original

11–20 of 588 posts

Re: "We ran out of columns"

#11

Post by lexicality »

jrochkind1 wrote: post #5
PreInternet01 wrote: post #3

> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…

Oh good question on date essay was written -- put dates on your things on the internet people! Internet Archive has a crawl from today but no earlier; which doesn't mean it can't be earlier of course. My guess is it was written recently though.

created 14 hours ago https://github.com/jimmyhmiller/jimmyhmiller.github.io/commi...

Re: "We ran out of columns"

#12

Post by _elf »

cowsandmilk wrote: post #4

He acts like sequence key is odd, but that’s quite normal in database world. https://www.postgresql.org/docs/current/sql-createsequence.h...

Databases have built-in features for this now. What the author is talking about is a regular table.

In reality, that wasn't too unusual to see because frameworks would use that technique because it's a lowest common denominator across RDMS.

Re: "We ran out of columns"

#13

Post by mnahkies »

cowsandmilk wrote: post #4

He acts like sequence key is odd, but that’s quite normal in database world. https://www.postgresql.org/docs/current/sql-createsequence.h...

I think the intriguing part was purposefully using the same sequence value for rows in multiple tables.

I've worked with globally unique (to our application) integer keys, and per table integer sequences (which obviously aren't globally unique), but I don't recall seeing anyone use a global sequence but purposefully reuse elements of the sequence before.

Re: "We ran out of columns"

#14

Post by hobs »

PreInternet01 wrote: post #3

> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…

This is perfectly fine when you are driving some app that has a per-user experience that allows you to wrap up most of their experience in some blobs.

However I would still advise people to use a third normal form - they help you, constraints help you, and often other sets of tooling have poor support for constraints on JSON. Scanning and updating every value because you need to update some subset sucks.

You first point is super valid though - understanding the domain is very useful and you can get easily 10x the performance by designing with proper types involved, but importantly don't just build out the model before devs and customers have a use for anything, this is a classic mistake in my eyes (and then skipping cleanup when that is basically unused.)

If you want to figure out your data model in depth beforehand there's nothing wrong with that... but you will still make tons of mistakes mistakes, lack of planning will require last minute fixes, and the evolution of the product will have your original planning gather dust.

Re: "We ran out of columns"

#15

Post by __MatrixMan__ »

lexicality wrote: post #11
jrochkind1 wrote: post #5

Earlier quoted context omitted.

Oh good question on date essay was written -- put dates on your things on the internet people! Internet Archive has a crawl from today but no earlier; which doesn't mean it can't be earlier of course. My guess is it was written recently though.

created 14 hours ago https://github.com/jimmyhmiller/jimmyhmiller.github.io/commi...

But clearly in retrospect. It sounds like some of the things I was doing in 2009.

Re: "We ran out of columns"

#16

Post by jsonis »

PreInternet01 wrote: post #3

> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…

> These days, my go-to solution is SQLite with two fields (well, three, if you count the implicit ROWID, which is invaluable for paging!): ID and Data, the latter being a JSONB blob.

Really!? Are you building applications by chance or something else? Are you doing raw sql mostly or an ORM/ORM-like library? This surprises me because my experience dabbling in json fields for CRUD apps has been mostly trouble stemming from the lack of typechecks. SQLite's fluid type system haa been a nice middle ground for me personally. For reference my application layer is kysely/typescript.

Re: "We ran out of columns"

#17

Post by datavirtue »

Waterluvian wrote: post #2

The first line really hits me hard. There’s something so incredibly freeing about being a kid and doing stuff like coding. There’s simply no expectations. Even the smallest project felt like such an achievement. But now I code professionally and I don’t know how to turn off engineering brain. I don’t know how to be okay doing something poorly, but on my terms.

It's because you don't get to write apps from scratch enough. Everything is a POC. You get it working. Optimization is just dumb until the project is so big it doesn't fit in the developers brain anymore. Then you write tests for the core part so you can keep interating fast. You will see parts that clearly need refactorwd before you can move on.

Also, be aware of, but don't get hung up on, engineering best practices. Chances are that someone is going to see your POC and put it in front of a client or promise it to someone. It will enter service being less than perfect, if you are lucky.

When you pull this off people will tell stories about you and brag about you when you are not around. None of them will know anything about the code engineering.

Re: "We ran out of columns"

#18

Post by collinvandyck76 »

mnahkies wrote: post #13
cowsandmilk wrote: post #4

He acts like sequence key is odd, but that’s quite normal in database world. https://www.postgresql.org/docs/current/sql-createsequence.h...

I think the intriguing part was purposefully using the same sequence value for rows in multiple tables. I've worked with globally unique (to our application) integer keys, and per table integer sequences (which obviously aren't globally unique), but I don't recall seeing anyone use a global sequence but purposefully reuse elements of the sequence before.

I've seen this fairly recently and I was surprised to also see it in this article because I think that makes it twice that I've seen it in ~25 years.

Re: "We ran out of columns"

#20

Post by teaearlgraycold »

PreInternet01 wrote: post #3

> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…

Byte vs. Int is premature optimization. But indexing, primary keys, join tables, normalization vs. denormalization, etc. are all important.
Post reply on HN