Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"We ran out of columns"

jimmyhmiller.github.io

Post reply on HN
Print view View original

191–200 of 588 posts

Re: "We ran out of columns"

#191

Post by addaon »

llmblockchain wrote: post #175
malux85 wrote: post #166

Earlier quoted context omitted.

Here’s a few of my horror stories where I was a consultant at various companies: 1. Previous consultants left no documentation or anything, and a running Hadoop cluster handling (live!) 300 credit card transactions a second. Management hired 8 junior sysadmins - who were all windows sysadmins, had never used Linux before, and were expected to take over running this Linux cluster immediately. They all looked at me whi…

"Senior" holds no weight with me. I've had plenty dumb founding conversations with "seniors". My favorite was at the company that was self hosting their code. The senior team lead wanted me to help him find a memory leak that plagued the product for months. Customers were told to restart the application every few weeks (this was a C++ application). I sat down with the senior and looked at the code. I spotted the erro…

> I was like, "You know when you do new[] you need to use delete[]?" as all of his deletions were without [].

This seems like a pretty major lack of a specific piece of knowledge on the senior developers part, yes, but it seems like a much more unforgivable miss on the part of the code reviewers. Was the team stuck in a rut where only a single person (with coincidentally the same blind spot) was reviewing his code, or did multiple reviewers somehow miss this?

Re: "We ran out of columns"

#192

Post by donall »

fouronnes3 wrote: post #130
hleszek wrote: post #126

When I started at my first company, they had a very complex VB application running on dozens of customers around the country, each having some particular needs of course. There was a LOT of global variables (seemingly random 4 uppercase letters) controlling everything. At some point, the application had some bugs which were not appearing when the application was run in debug mode in Visual Studio. The solution was ob…

This is amazing. I can so well imagine a bright young hire joining that team, helpfully offering to "setup this thing called git" only to be laughed out of the meeting by all the "senior" staff.

I had a visceral reaction to this comment! I once joined a company doing ETL with Apache camel and a half dozen underpowered pet machines. Ingesting their entire dataset and running a suite of NLP models took 3-6 months (estimated; it was so slow nobody ever reprocessed the data to fix bugs or release improvements). I drew up a simple architecture using Kafka, hbase, and MapReduce to implement a lambda architecture. The CTO very patronizingly told me that just because something is shiny and new it doesn't mean we need to implement it. This was 2017 :laugh-cry:.

Re: "We ran out of columns"

#193

Post by wiether »

Nexialist wrote: post #31

My worst codebase story: In my first real job, I worked for a company that maintained a large legacy product programmed in a combination of COBOL and Java. In order to work on the Java side of the product, you checked out individual files from source control to work on, which 'locked' the files and prevented other developers from checking out the same files. This functionality was not part of our actual source contro…

> In my first real job, I worked for a company that maintained a large legacy product programmed in a combination of COBOL and Java.

> In order to work on the Java side of the product, you checked out individual files from source control to work on, which 'locked' the files and prevented other developers from checking out the same files.

COBOL + Java and file lock... I thought we worked on the same project and it brought back many nightmares!

But we were using Subversion's lock system and had a proper CI/CD with Jenkins.

So you won, your project was worse!

Re: "We ran out of columns"

#194

Post by zbentley »

datavirtue wrote: post #17
Waterluvian wrote: post #2

The first line really hits me hard. There’s something so incredibly freeing about being a kid and doing stuff like coding. There’s simply no expectations. Even the smallest project felt like such an achievement. But now I code professionally and I don’t know how to turn off engineering brain. I don’t know how to be okay doing something poorly, but on my terms.

It's because you don't get to write apps from scratch enough. Everything is a POC. You get it working. Optimization is just dumb until the project is so big it doesn't fit in the developers brain anymore. Then you write tests for the core part so you can keep interating fast. You will see parts that clearly need refactorwd before you can move on. Also, be aware of, but don't get hung up on, engineering best practices…

> When you pull this off people will tell stories about you and brag about you when you are not around.

True!

> None of them will know anything about the code engineering.

Not true. In environments where it's very easy to fuck up (hyperconnected codebases, lots of legacy tech debt, fragile systems generally), programmers who create tools/libraries that let other engineers get work done without blowing their feet off or spending huge amounts of time on silly things do get praised. Maybe the praise comes from different people than in your example, but it definitely is rewarded.

Two caveats: this only happens in certain environments; some gigs are just culturally unable to care about quality tools (death march feature factories and the like). Also, you actually have to save time and reduce defect rates with your engineering work; starting an architecture-astronaut rewrite that isn't actually useful for other programmers' day-to-day work doesn't count. Making "one tool to rule them all" that requires a Ph. D or pairing session with the author doesn't count either.

Re: "We ran out of columns"

#195

Post by Atreiden »

qxmat wrote: post #85

Jira's now discontinued server version had a sequence table to stop you sharding it. It also made disaster recovery from a hard shutdown awful. I have nothing good to say about Atlassian.

Atlassian, and JIRA specifically, are responsible for so much wasted time and capital expenditure. If I could get ahold of metrics like "hours spent building, using, and maintaining JIRA" versus "value obtained from JIRA" for each of the companies I've worked at, I'm pretty sure I could generate a report so scathing that nobody would ever use it again.

Gatekeeping your work organization system from the people working on, and often MANAGING it is such a huge friction point I'm amazed they ever got clientele. I'm an Admin on a project right now and I can't change our status types without going through our assigned Atlassian rep.

And don't even get me started on the dumpster fire that is BitBucket. Ever tried to use the API? it's somehow even more worthless than the UI.

Re: "We ran out of columns"

#196

Post by MBCook »

This reminds me of my first job at a very small shop. Here’s two stories:

The calendar table struck a chord. We had one for holidays. One system needed to know when they were to calculate pay dates. Except once a year it would “run out” and someone would have to go add in the next year’s worth after a bad calculation was spotted.

The second time I was told to do it, I put in 10 years worth. The company didn’t survive long enough to need more.

My first “big” project was actually that pay date code. Every once in a while the server would hang, and people had deduced the date calculation was the problem. But no one knew why. And it wasn’t frequent enough to be worth taking time from the other two programmers. But I was new and thus “spare”.

After not being able to find any errors, I brute forced it. I ran that calculation for every day of the year for every pay type (weekly, monthly, bi-monthly, etc) and it quickly got locked in an infinite loop. Armed with the “bad” data that triggered it, it was easy to solve and provide tests to prove it would never happen again. I don’t remember what it was, exactly.

Re: "We ran out of columns"

#197

Post by culi »

hleszek wrote: post #126

When I started at my first company, they had a very complex VB application running on dozens of customers around the country, each having some particular needs of course. There was a LOT of global variables (seemingly random 4 uppercase letters) controlling everything. At some point, the application had some bugs which were not appearing when the application was run in debug mode in Visual Studio. The solution was ob…

Can I ask roughly what year this was? I can't even imagine something like that today

Re: "We ran out of columns"

#198

Post by redserk »

brabel wrote: post #171
fiddlerwoaroof wrote: post #151

Earlier quoted context omitted.

I use CVS, RCS and Subversion on my personal projects for several years before I learned git. I don’t remember any of them being a pain to setup for small projects and Sourceforge provided free CVS hosting for small projects.

I started with Rational Clearcase, which I interacted with via Eclipse, and I remember that you did have to lock a file to make changes to it... pretty sure there was no merge capability (but maybe I was just too junior to know stuff like that, it was definitely not something you would do as a matter of routine, like we do now with git). Someone leaving for vacation and forgetting to unlock a file could be a huge pai…

Rational Clearcase is a disaster but you could do merges.

The fun/frustrating part was getting branching and "views" correct.

I worked at an organization on this side of the century where it was one person's full-time job to handle merging code from different teams. The same organization didn't actually use raw Clearcase either -- it was wrapped in hundreds-of-thousands-of-lines-of Perl scripts.

Re: "We ran out of columns"

#199

Post by thayne »

I've seen something very similar to the Sequence key table. But there wasn't just one of them, it was a common pattern. And it had 2 rows and 2 columns.

The reason we had it is we had master-master replication, but without requiring the peer to acknowledge befor committing a transaction. To avoid inconsistencies, we preferred one server for even ids and the other for odd ids. But when writing a new record, an autogenerating sequence would just give the next id without regard to what "side" the request was on. So we had a table to keep track of the next id to use for each side, where we incremented the next id by 2 each time a new id was allocated.

It was a little weird, but it worked fairly well. Although we eventually changed the architecture and removed the need for those tables.

Re: "We ran out of columns"

#200

Post by cyberax »

jsonis wrote: post #16
PreInternet01 wrote: post #3

> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…

> These days, my go-to solution is SQLite with two fields (well, three, if you count the implicit ROWID, which is invaluable for paging!): ID and Data, the latter being a JSONB blob. Really!? Are you building applications by chance or something else? Are you doing raw sql mostly or an ORM/ORM-like library? This surprises me because my experience dabbling in json fields for CRUD apps has been mostly trouble stemming f…

In my current company, we're using a similar approach: just shove everything into a JSON blob. If you need a constraint or an index, you can create a computed column (in PostgreSQL) that pulls out a field from JSON.

For the data schema, we're using Protobufs with buf validate. This works surprisingly well, you can use the same types in the backend API and on the frontend. We even have a cron job that reads all the data periodically and verifies that the JSON blobs conform to the schema. Our next intern is going to write a PostgreSQL extension to do that on commit/update :)

One real advantage of this approach is that it's easy to do stuff like "search everywhere".

Post reply on HN