Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"We ran out of columns"

jimmyhmiller.github.io

Post reply on HN
Print view View original

411–420 of 588 posts

Re: "We ran out of columns"

#411

Post by switch007 »

failbuffer wrote: post #220
agentultra wrote: post #132

The ending is pure gold. Some of the best times in my career were working on a codebase for an application serving folks I knew on a first name basis and had had lunch with. I could talk through pain points they were having, we’d come up with a solution together, I’d hack up a quick prototype and launch it just to them to try out. We’d tweak it over a couple of weeks and when it was good I’d launch it to all customer…

... and get them talking directly to the user. I feel that's where the real magic happens.

Surely it is much more efficient for a PM to ask all the wrong questions and relay the answers using totally different words to the developers. As many many companies love hiring tons of PMs, this is surely the optimal system

Re: "We ran out of columns"

#412

Post by throwaway7e8te »

mg wrote: post #107

That is why people these days tend to use a single JSON blob instead of multiple columns. And because it is so popular, SQLITE and other DBs are building better and better JSON support into the DB. I wonder if better support of EAV tables would solve this issue better. If one could do "SELECT price,color,year FROM cars! WHERE status='sold'" and the "!" would indicate that cars is an EAV table ... entity attribute val…

Have you looked at the "hstore" type in Postgres? It seems to cover that use case.

Re: "We ran out of columns"

#413

Post by ptrik »

codeptualize wrote: post #218

What an amazing read, funny, nostalgic, the duality of the perfect mess but still so much opportunity and ability to make progress, and somehow it all just chugging along. I feel a lot of this is the difference between theory and practice. Sure each of these things are bad, but probably a lot might have been the right choice at the time, and in a way, most companies, even most projects running for many years, end wit…

This is the main takeaway for me. The decentralized way of software development in a large scale. It does echoes with microservices a lot, but this can be done with a more traditional stack as well. It's ultimately about how you empower teams to develop features in parallel, and only coordinate when patterns emerge.

Re: "We ran out of columns"

#414

Post by darkwater »

cruffle_duffle wrote: post #357
oooyay wrote: post #316

Earlier quoted context omitted.

Before there was GitHub there was Sourceforge. Sourceforge supported several different protocols and git still won. Gits internals can be confusing at first, but once you understand them there's really nothing you can't do. Being distributed by design also helps.

Sourceforge was complete garbage though. I hated, hated when projects were hosted on it. It was slow, full of ads, impossible to find what you need to download.. GitHub is to sourceforge what Facebook was to MySpace. MySpace was first but it was buggy as hell.

SF started to be filled with ads only in a second phase. By memory I would say around 2010, and checking Wikipedia it says it changes ownership in 2012. But when it was the de facto "central repository" for Linux softwares codebases, I don't remember it being full of ads.

Re: "We ran out of columns"

#415

Post by croes »

hleszek wrote: post #126

When I started at my first company, they had a very complex VB application running on dozens of customers around the country, each having some particular needs of course. There was a LOT of global variables (seemingly random 4 uppercase letters) controlling everything. At some point, the application had some bugs which were not appearing when the application was run in debug mode in Visual Studio. The solution was ob…

That's why cloud solutions exist.

Now you only need to run the app in debug mode on your own server.

Re: "We ran out of columns"

#416

Post by immibis »

PreInternet01 wrote: post #40
jsonis wrote: post #16

Earlier quoted context omitted.

> These days, my go-to solution is SQLite with two fields (well, three, if you count the implicit ROWID, which is invaluable for paging!): ID and Data, the latter being a JSONB blob. Really!? Are you building applications by chance or something else? Are you doing raw sql mostly or an ORM/ORM-like library? This surprises me because my experience dabbling in json fields for CRUD apps has been mostly trouble stemming f…

> my experience dabbling in json fields for CRUD apps has been mostly trouble stemming from the lack of typechecks Well, you move the type checks from the database to the app, effectively, which is not a new idea by any means (and a bad idea in many cases), but with JSON, it can actually work out nicely-ish, as long as there are no significant relationships between tables. Practical example: I recently wrote my own S…

> Fortunately, my session database is just 'JSON(B) in a single table', so I was able to add those additional fields without the need for any migrations.

> And SQLite's `json_extract` makes adding indexes after-the-fact super-easy.

That's a migration.

> Of course, these additional fields need to be explicitly nullable, and I need to skip processing based on them if they're absent

That's an effect of not migrating - having to process null and absent fields instead of just null fields. After doing more of these, you'll run into the same thing that made people stop using NoSQL databases: with no schema, your code has to parse all previous versions of the data format and they probably aren't even well-documented. While an RDBMS can just set the new column to null in existing rows.

Re: "We ran out of columns"

#417

Post by immibis »

jsonis wrote: post #309
Izkata wrote: post #150

Earlier quoted context omitted.

> Fortunately, my session database is just 'JSON(B) in a single table', so I was able to add those additional fields without the need for any migrations. And SQLite's `json_extract` makes adding indexes after-the-fact super-easy. Our solution for a similar situation involving semi-structured data (in postgres) was to double it up: put all the json we send/receive with a vendor into a json field, then anything we actu…

This is exactly what I've tried (and failed at) doing! Can I ask how you handle normalization from vendor data when it contains relationships and multilevel nesting? How do you know when to create a new child table, and which ones to create, and their relationships etc. I haven't found a good balance yet.

Odd-shaped miscellaneous data that you only need to retrieve is a good candidate for a JSON field. Once you're heavily using some piece of data, or if you need to index it (which means you are heavily using it), you should insert the data in the database "properly".

If some vendor is giving you a list of categories you don't care about, there's no need to make a vendor categories table and a many-to-many link table until you actually need them.

The point is that putting data properly in the database lets you use database features on it and get database performance.

Re: "We ran out of columns"

#418

Post by immibis »

jvans wrote: post #55
PreInternet01 wrote: post #40

Earlier quoted context omitted.

> my experience dabbling in json fields for CRUD apps has been mostly trouble stemming from the lack of typechecks Well, you move the type checks from the database to the app, effectively, which is not a new idea by any means (and a bad idea in many cases), but with JSON, it can actually work out nicely-ish, as long as there are no significant relationships between tables. Practical example: I recently wrote my own S…

why is a migration such a burden in that scenario

because they put everything in JSON. Migration means running a script to parse and edit each JSON item instead of letting the database do database things automatically.

Re: "We ran out of columns"

#419

Post by greenthrow »

This is obviously a work of fiction. Not because there isn't bad code similae to this out there, but because the way it is told and many of the details don't add up and have all the hallmarks of fabrication.

Re: "We ran out of columns"

#420

Post by immibis »

ssdspoimdsjvv wrote: post #82
_elf wrote: post #12

Earlier quoted context omitted.

Databases have built-in features for this now. What the author is talking about is a regular table. In reality, that wasn't too unusual to see because frameworks would use that technique because it's a lowest common denominator across RDMS.

Does SQLite have sequences yet?

Sqlite only has automatically assigned row ID. You can type a column as "integer primary key" to make it an alias for the row ID, or use ROWID directly (not recommended for compatibility).
Post reply on HN