Imaginary problems are the root of bad software • Hacker News Board

Re: Imaginary problems are the root of bad software

#171

Post by dagmx » Sun, Jun 18, 2023, 7:08 PM UTC

notatoad wrote: post #167

dagmx wrote: post #64

Earlier quoted context omitted.

The problem is that their brief is too vague and the real issue is communication. At no point is it clear that they’ve discussed off the shelf solutions, or infrastructure (storage, server hiding, CDN etc). If they had, then they either messed up because 15k over two months is too high or too low. It’s squarely in the: “nobody has actually defined the project territory” for me. Yes, a Wordpress solution would work, b…

>At no point is it clear that they’ve discussed off the shelf solutions, or infrastructure (storage, server hiding, CDN etc). why should they? the requirements are listed in the article. the job is to meet the requirements. "where should we store the files" is a job the client hires you to answer, not something requiring communication.

The requirements are insufficient if you’re going to be so particular as to stick exactly to them. You can’t guarantee the uptime’s requested without talking about infrastructure.

Re: Imaginary problems are the root of bad software

#172

Post by neon_electro » Sun, Jun 18, 2023, 7:09 PM UTC

noirbot wrote: post #130

stevehind wrote: post #20

This resonates and one way to describe it is an incentive problem. Someone whose incentives are tightly aligned with the business is going to solve the actual problem and simply and effectively as possible. Someone who is incentivized to build career capital and experience other than via impact (e.g. so they can get uplevelled, pass an external interview loop, etc) is much more likely to focus on unimportant hard pro…

Doing the sort of simple solutions to your specific job's actual problems can also be something that constrains your ability to work anywhere else. Often the best simple solution that's tightly integrated into your job's environment is something that is inconceivable as a good idea anywhere else. You're optimizing around other old decisions, good or bad. You're often correctly overfitting a solution to your specific…

I share this problem in the last year+ of job searching I've been up to.

Re: Imaginary problems are the root of bad software

#173

Post by Nevermark » Sun, Jun 18, 2023, 7:11 PM UTC

OMG, never has truth been so funny and yet so tragic.

This article, its art and its font choice, shall be preserved forever in the archive of Historical Documents.

Our eternal response to unnecessary complexity? "Never give up! Never surrender!"

--

ADDENDUM:

The article makes a good case for formally adding "manufactured complexity" and "miscommunication complexity" to "accidental complexity" and "necessary complexity".

They are quite common distinct causes.

Re: Imaginary problems are the root of bad software

#174

Post by WinLychee » Sun, Jun 18, 2023, 7:11 PM UTC

pavlov wrote: post #90

lewisjoe wrote: post #73

If anything it's the incentive system in software industry, which is at fault. 1. No designer is given promotion for sticking to conventional designs. It's their creative & clever designs that get them attention and career incentives. 2. No engineer is paid extra for keeping the codebase without growing too much. It's re-writes and the effort he puts in to churn out more solutions (than there are problems) that offer…

> “No designer is given promotion for sticking to conventional designs. It's their creative & clever designs that get them attention and career incentives.” This is a massive change from my first software industry job in 1997. I was essentially a “design intern who knows HTML” on a team that built a shrinkwrap Windows application for enterprises. The core of the design team was a graphic designer, a cognitive scienti…

There's been a large influx of people chasing dollars and status, they could not care less about the product or who uses it. You can still find orgs that do care, but they're swiftly pushed out by the VC-funded "growth at any cost" model or are acquired into oblivion.

The indie scene is looking excellent however, a lot of programmers who have already made their money seem to be pushing excellent hobby projects out.

Re: Imaginary problems are the root of bad software

#175

Post by davidmnoll » Sun, Jun 18, 2023, 7:13 PM UTC

pydry wrote: post #144

davidmnoll wrote: post #112

Earlier quoted context omitted.

I have seen it validated by reality several times… more times than the opposite. I had a boss refuse to let me do a refactor that changed these sketchy dynamic field tables into json columns because “it’s not customer facing.” They were unable to show off features in an important demo because the endpoints were timing out despite putting 2 other people on it for 2 weeks to find code-based optimizations. 3 days later…

>I had a boss refuse to let me do a refactor that changed these sketchy dynamic field tables into json columns because “it’s not customer facing YAGNI isnt about not refactoring existing technical debt. It's about not trying to pre-empt future requirements. If youre refactoring in anticipation of as yet unmaterialized requirements then YAGNI applies - e.g. generalizing code when when today there is 1 specific use cas…

All of these definitions are fuzzy... refactor versus upgrade versus feature. When the people wrote it the way they did, they were almost certainly thinking that they don't need to overthink or over-engineer, and that they should discount hypothetical future concerns.

I can give you an abundance of examples. We were creating a page that was going to use state in a certain way. I was trying to insist that we address the way state will be handled across pages ahead of time. These concerns were dismissed as premature optimization. A few months later we had 5 pages with the state being handled in 5 different ways, and being synced in different ways between each page, complete with if statements, sometimes passing state through URLs, sometimes through local storage, sometimes through session, sometimes through JWT data, generally through a combo of several of them. Then we'd end up with confusing redirect loops for certain edge cases, state getting overwritten, etc.. We spend weeks fixing these bugs, and, eventually, weeks refactoring to manage state in a simpler way. These bugs often got caught by customers, drawing us away from feature delivery that was critical for demos to large customers.

All of that could have been avoided by spending 1 day thinking a little harder and planning for the future.

It ultimately boils down to a couple assumption that people like to make. (1) engineers know nothing about the domain, they can never predict what will be needed. That might be true in a large company with obscure domain-specific things for engineers who work far away from the day-to-day, but sometimes the engineers know exactly what's going to come up. (2) You can hill-climb your way into optimal program implementation. You can get to local maxima this way, but there are regular ways that programs grow based on how the business is growing and you can predict certain places where you will soon hit diminishing returns for current implementations. As long as you're up front about it and double-check your assumptions about the way the business is growing (and hence the application), I think there are ample places where you actually are going to need it.

Re: Imaginary problems are the root of bad software

#176

Post by BeFlatXIII » Sun, Jun 18, 2023, 7:17 PM UTC

noirbot wrote: post #151

djbusby wrote: post #56

Earlier quoted context omitted.

RDD: Resume Driven Development

This is absolutely a thing, but I'd say there's a related option which is "Job Listing Driven Development". The more niche, dated, or specific your platform is, the harder it is to hire people onto the team who don't need months of on-the-job practice and training to be useful. You see the most extreme versions of dangers of this in stories about governments and older companies having to pay insane salaries to bring…

What's kind of funny is that MUMPS is equally as archaic and idiosyncratic as Fortran or Cobol, yet there are companies willing to put new hires through a bootcamp to make them productive. Are all the Fortran and Cobol companies too small to afford a month or three of training time on new devs?

Re: Imaginary problems are the root of bad software

#177

Post by thwarted » Sun, Jun 18, 2023, 7:17 PM UTC

ilyt wrote: post #127

pavlov wrote: post #90

Earlier quoted context omitted.

> “No designer is given promotion for sticking to conventional designs. It's their creative & clever designs that get them attention and career incentives.” This is a massive change from my first software industry job in 1997. I was essentially a “design intern who knows HTML” on a team that built a shrinkwrap Windows application for enterprises. The core of the design team was a graphic designer, a cognitive scienti…

Remember times where you could change the theme and all of the apps followed suit ? Even in Linux there were tools to sync gnome with QT look so you could have one theme applied to every app for nice and consistent look, all the way to how the common icons look. Nowadays ? Every fucking app gotta have their own different styling. Will the setting icon be three dots, gear, or honey badger ? WHO FUCKING KNOWS. You'd be…

And it got worse with client side decorations. Now even window interactions are app-specific and don't adhere to global settings.

Re: Imaginary problems are the root of bad software

#178

Post by jtriangle » Sun, Jun 18, 2023, 7:18 PM UTC

chefandy wrote: post #140

dtagames wrote: post #3

The author hits the nail on the head with his claim that imaginary problems are more fun than real ones. As developers and smart folks in general, we like complicated problems that are big and far away. How many times have I heard in a meeting, "Yeah, but when we have 1M users..." It's great fun to think your product will get to 1M users. It's also very unlikely. It's not nearly as fun to finish and ship and market a…

I worked with a project manager that went too far in the opposite direction, though. Their pushback against premature optimization manifested in wanting to start with a functionalish "proof of concept" without any real design phase so they can say we cranked out an MVP in the first sprint... and before you know it, like most non-blocking technical debt, the "migrate functionality to final codebase" kanban card moves…

The trouble is, an 'MVP' is often far from 'minimum' in those sorts of situations.

The reality is that MVP should be missing most planned functionality and should really just be a few core features and functions that the rest of the application builds off of, the trunk of the dependency tree so to speak. That idea is, unfortunately lost on the majority of PM's, and ultimately it costs more time/money to get to a finished v1 because of it.

Re: Imaginary problems are the root of bad software

#179

Post by gopalv » Sun, Jun 18, 2023, 7:19 PM UTC

joshlemer wrote: post #150

lewisjoe wrote: post #73

If anything it's the incentive system in software industry, which is at fault. 1. No designer is given promotion for sticking to conventional designs. It's their creative & clever designs that get them attention and career incentives. 2. No engineer is paid extra for keeping the codebase without growing too much. It's re-writes and the effort he puts in to churn out more solutions (than there are problems) that offer…

I don't know, the more I advance in my career, the more I see it as the opposite. Wide eyed developers with big designs, who are obsessed with the technical aspects of a solution, who disregard the practicalities, long term implications at the social level (who is going to maintain this, do we have people that have that skillset, is this worth the effort, does it really matter to be this elegant, or is it more import…

> the more effective engineers who understand these priorities are given more respect and authority.

The problem with "given more authority" I see is that management plucks these engineers out to make their day job basically "sit in meetings" if you're even slightly effective at simplifying life for everyone else.

Because that is the place of most leverage to place those people, but then those people are in a constant tug-of-war with the first group of "fresh ideas".

Eventually, the people who are in charge of the "prevention of bad architecture" become the bad guys because they (or me, I'm projecting) get jaded into just finding out what's wrong with something as fast as possible to be able to keep up with that workload.

You go from a creative role to a fundamentally sieve role with destructive tendencies, where you are filtering out the good from the bad as fast as possible, be.

First of all, not all new ideas are bad and "there's something bad about X" is not a "let's not do X".

Secondly, going from making things to shooting down things is intellectual suffering if you have a bit of empathy.

Some people on the "committee" with you don't have empathy & literally enjoy it - you're either trying to damage control on a case-by-case or building a "these assholes need to be fired" doc out of the meetings.

I realized what I would become if I conflated authority and respect ("respect my authoritah!").

Quitting was really the only way out of it. But it wasn't hard to explain to my spouse that i needed to leave for a job a level down and which paid half as much, because she could see me bringing my "why the hell do we have to do this" attitude home & to family decisions.

Re: Imaginary problems are the root of bad software

#180

Post by ericbarrett » Sun, Jun 18, 2023, 7:21 PM UTC

fauigerzigerk wrote: post #116

ericbarrett wrote: post #66

Earlier quoted context omitted.

The trick was there was enough growth that the savings from the compiler were massive . (I worked there at the time.) The inefficiency of the PHP interpreter was a great problem to have, because it came from the success it enabled.

So I think the interesting question is whether the rest of us can learn anything from what happened there. I believe Mark Zuckerberg simply used the technology he knew and took it from there. That's fine. I probably would have done the same thing. But many people are making an ideology out of this, arguing that not giving a shit about performance is always the right choice initially because that's how you grow fast e…

I take away two lessons from it, which are in a kind of essential tension that can only be mediated by wisdom and experience:

1) Pick technologies that you can optimize.

2) Don't over-optimize.

Also, the concept of "optimization" here has very little to do with the language itself. It's far more about the overall stack, and it definitely includes people and processes (like hiring). It's not like FB invested $0 toward performance before swapping out PHP interpreters! Its massive caching layer, for example, was already taking shape well before HPHP (the C++ transpiler which preceded HipHop), not to mention the effort and tooling behind the MySQL sharding and multi-region that still exists in some form today. Many backend FB services were already written in C++ by 2010. But they had already gone very, very far—farther than most businesses ever will—on "just" PHP. Heroics like HPHP only happened after enormous pipelines of money were already flowing into the company.