Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"We ran out of columns"

jimmyhmiller.github.io

Post reply on HN
Print view View original

41–50 of 588 posts

Re: "We ran out of columns"

#41

Post by gonzo41 »

PreInternet01 wrote: post #3

> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…

This is essentially just a data warehousing style schema. I love me a narrow db table.

But I do try and get a schema that fits the business if I can.

Re: "We ran out of columns"

#42

Post by zo1 »

pelagicAustral wrote: post #39
trte9343r4 wrote: post #21

Slicing columns into multiple tables is fairly common type of sharding. Sort of SQL way to do columnary store.

This is such an horrendous practice, I am yet to find a database were this makes sense. Maybe my brain is not wired for it

The problem is maybe not so much the splitting and putting extra columns in a separate table. It's that you even have a table that large that it necessitates such a thing. Worst case you have a main table and a detail table that has a one to one correlation to the main entity table.

Re: "We ran out of columns"

#43

Post by jimmyhmiller »

Waterluvian wrote: post #2

The first line really hits me hard. There’s something so incredibly freeing about being a kid and doing stuff like coding. There’s simply no expectations. Even the smallest project felt like such an achievement. But now I code professionally and I don’t know how to turn off engineering brain. I don’t know how to be okay doing something poorly, but on my terms.

This is something I've had to train myself to overcome. The first step for me was having a place where I clearly signaled that this wasn't my best work. Where the rules were allowed to be broken. That place is my junk drawer of code[1].

I have a zsh alias `alias changes='git add . && git commit -am "Changes" && git push'` that I use for my commits in the repo.

This all may feel silly, but it's what I needed to get back to that time of playing. Where I never worried about the code. But where I also didn't feel I was wasting my time working on code no one could see. I'd definitely recommend trying something like that if you are struggling with it.

[1]: https://github.com/jimmyhmiller/PlayGround

Re: "We ran out of columns"

#44

Post by scythmic_waves »

ibejoeb wrote: post #28
klysm wrote: post #7

Earlier quoted context omitted.

I’m still in the think hard about the schema camp. I like to rely on the database to enforce constraints.

Yeah, a good database is pretty damn handy. Have you had the pleasure of blowing young minds by revealing that production-grade databases come with fully fledged authnz systems that you can just...use right out of the box?

Can you say more? I’m interested.

Re: "We ran out of columns"

#45

Post by stavros »

Nexialist wrote: post #31

My worst codebase story: In my first real job, I worked for a company that maintained a large legacy product programmed in a combination of COBOL and Java. In order to work on the Java side of the product, you checked out individual files from source control to work on, which 'locked' the files and prevented other developers from checking out the same files. This functionality was not part of our actual source contro…

We all take too much for granted how much git "just works", even when it doesn't.

Re: "We ran out of columns"

#46

Post by whatever1 »

Nexialist wrote: post #31

My worst codebase story: In my first real job, I worked for a company that maintained a large legacy product programmed in a combination of COBOL and Java. In order to work on the Java side of the product, you checked out individual files from source control to work on, which 'locked' the files and prevented other developers from checking out the same files. This functionality was not part of our actual source contro…

I bet now they deal with broken pipelines and dependency hell.

Tools do not fix bad design.

Re: "We ran out of columns"

#47

Post by tryauuum »

Would be great to work in such a company as a Linux guru.

There are so many entangled services and machines that you feel like an Indiana Jones. You ssh into a machine and feel century-old dust beneath your footsteps. And you never know what will you find. Maybe a service which holds the company together. Maybe a CPU eating hog which didn't do anything useful last 3 years.

I don't enjoy writing new code much. But in such an environment even with my limited skills I can do decent improvements, especially from security point of view. Feels great

Re: "We ran out of columns"

#48

Post by stavros »

scythmic_waves wrote: post #44
ibejoeb wrote: post #28

Earlier quoted context omitted.

Yeah, a good database is pretty damn handy. Have you had the pleasure of blowing young minds by revealing that production-grade databases come with fully fledged authnz systems that you can just...use right out of the box?

Can you say more? I’m interested.

I guess they mean something like Postgres' row-level security:

https://www.postgresql.org/docs/current/ddl-rowsecurity.html

Re: "We ran out of columns"

#49

Post by throwup238 »

PreInternet01 wrote: post #40
jsonis wrote: post #16

Earlier quoted context omitted.

> These days, my go-to solution is SQLite with two fields (well, three, if you count the implicit ROWID, which is invaluable for paging!): ID and Data, the latter being a JSONB blob. Really!? Are you building applications by chance or something else? Are you doing raw sql mostly or an ORM/ORM-like library? This surprises me because my experience dabbling in json fields for CRUD apps has been mostly trouble stemming f…

> my experience dabbling in json fields for CRUD apps has been mostly trouble stemming from the lack of typechecks Well, you move the type checks from the database to the app, effectively, which is not a new idea by any means (and a bad idea in many cases), but with JSON, it can actually work out nicely-ish, as long as there are no significant relationships between tables. Practical example: I recently wrote my own S…

> And, no, no need for an ORM, except `JsonSerializer.Deserialize`... (And yeah, all of this is just a horrible hack, but one that seems surprisingly resilient so far, but YMMV)

I do the same thing with serde_json in Rust for a desktop app sqlitedb and it works great so +1 on that technique.

In Rust you can also tell serde to ignore unknown fields and use individual view structs to deserialize part of the JSON instead of the whole thing and use string references to make it zero copy.

Re: "We ran out of columns"

#50

Post by stavros »

jrochkind1 wrote: post #5
PreInternet01 wrote: post #3

> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…

Oh good question on date essay was written -- put dates on your things on the internet people! Internet Archive has a crawl from today but no earlier; which doesn't mean it can't be earlier of course. My guess is it was written recently though.

At least we know it wasn't written after today!
Post reply on HN