Live data from Hacker News

Imaginary problems are the root of bad software

cerebralab.com

141–150 of 511 posts

Re: Imaginary problems are the root of bad software

#141

If anything it's the incentive system in software industry, which is at fault. 1. No designer is given promotion for sticking to conventional designs. It's their creative & clever designs that get them attention and career incentives. 2. No engineer is paid extra for keeping the codebase without growing too much. It's re-writes and the effort he puts in to churn out more solutions (than there are problems) that offer…

[deleted]

Re: Imaginary problems are the root of bad software

#142
post #18
post #3

The author hits the nail on the head with his claim that imaginary problems are more fun than real ones. As developers and smart folks in general, we like complicated problems that are big and far away. How many times have I heard in a meeting, "Yeah, but when we have 1M users..." It's great fun to think your product will get to 1M users. It's also very unlikely. It's not nearly as fun to finish and ship and market a…

And people underestimate how well some solid, dumb solutions can scale. Boring spring boot with a decent data model and a bit effort to stay stateless scales to the moon. Or, we're having a grand "data export" system customers use to collect data from us for their own DWHs. It has survived 2 attempts at replacement so far. At it's core, it's psql + rsync, or was recently migrated to psql + s3 when we decomissioned ou…

> And people underestimate how well some solid, dumb solutions can scale.

I'd say they underestimate how long "just throwing money" (servers) at the problem can work.

If you earn decent money now, scaling number of app servers 10x times to serve 10x times the traffic will still earn decent money. Doesn't matter that "PHP is slow", deal with it when your infrastructure cost warrant hiring more/better developers to fix it.

Especially now. Pair of fat servers on some NVMes and 1TB of RAM gonna cost you less than few dev-months and that can serve plenty of users in most use cases even before any extra caching will be needed.

Re: Imaginary problems are the root of bad software

#143

Earlier quoted context omitted.

The trick was there was enough growth that the savings from the compiler were massive . (I worked there at the time.) The inefficiency of the PHP interpreter was a great problem to have, because it came from the success it enabled.

So I think the interesting question is whether the rest of us can learn anything from what happened there. I believe Mark Zuckerberg simply used the technology he knew and took it from there. That's fine. I probably would have done the same thing. But many people are making an ideology out of this, arguing that not giving a shit about performance is always the right choice initially because that's how you grow fast e…

Learn what? That you should use the language that you’re more comfortable with and then scale? Or that languages have become more efficient? Php 8, for example, is many times faster than the php 4 and 5 that Facebook was using.

Re: Imaginary problems are the root of bad software

#144
post #68

Earlier quoted context omitted.

Ive seen a lot of engineers complain about YAGNI being taken too far but none who have seen their concerns validated by reality.

I have seen it validated by reality several times… more times than the opposite. I had a boss refuse to let me do a refactor that changed these sketchy dynamic field tables into json columns because “it’s not customer facing.” They were unable to show off features in an important demo because the endpoints were timing out despite putting 2 other people on it for 2 weeks to find code-based optimizations. 3 days later…

>I had a boss refuse to let me do a refactor that changed these sketchy dynamic field tables into json columns because “it’s not customer facing

YAGNI isnt about not refactoring existing technical debt. It's about not trying to pre-empt future requirements.

If youre refactoring in anticipation of as yet unmaterialized requirements then YAGNI applies - e.g. generalizing code when when today there is 1 specific use case because tomorrow you think there will be 3+.

If youre cleaning up existing code while working on it and the boss stops you because "it's not customer facing" then he's just asking you to violate the boy scout rule.

Re: Imaginary problems are the root of bad software

#145
post #135

Earlier quoted context omitted.

The trick was there was enough growth that the savings from the compiler were massive . (I worked there at the time.) The inefficiency of the PHP interpreter was a great problem to have, because it came from the success it enabled.

Did they succeed because of PHP or was it just a tech used at the time and anything else similar at the time would be fine either way?

They succeeded because of php. It was easy to use for them. So it enabled them to materialize their ideas. It was the right tool for them. Anything else would have been fine either way, if it was the language they were the most comfortable with. In their case, it happened to be php.

Re: Imaginary problems are the root of bad software

#146
post #51

Earlier quoted context omitted.

HipHop (later HHVM) was around 2010, so they scaled from 2004-2010 before that became needed. MyRocks was 2015. Wikipedia says FB was around 300 million users in 2009, then 400 million users in 2010.

Yes, good point. But you have to wonder what kind of engineering effort went into scaling PHP and MySQL up to the point where they decided to build a compiler and a storage engine.

When you have half a billion users that both read and write all day, you have to optimize, no matter the tech.

Re: Imaginary problems are the root of bad software

#147
post #3

The author hits the nail on the head with his claim that imaginary problems are more fun than real ones. As developers and smart folks in general, we like complicated problems that are big and far away. How many times have I heard in a meeting, "Yeah, but when we have 1M users..." It's great fun to think your product will get to 1M users. It's also very unlikely. It's not nearly as fun to finish and ship and market a…

Reminds me of a PM I used to work with. "Will this work for 1000 simultaneous users?" After almost 2 months, we have less than 100 users total, maybe 5 of them log in a day, and maybe 1 will actually do anything of interest. There is no technical problem. The problem is nobody worked on actually marketing the product. Build it and nobody shows up is the norm.

I was interviewing with a company that had barely any customers and they were asking scaling questions with Spark, etc. The salaries they paid could barely hire a team capable of dealing with the complexities of Spark, so they asked, "what would you do."

I told them I'd buy another stick of RAM and scale vertically until I had more customers, and save money on staff in the meantime. The interviewer went cold, I didnt get the job.

Re: Imaginary problems are the root of bad software

#148
post #83

This resonates and one way to describe it is an incentive problem. Someone whose incentives are tightly aligned with the business is going to solve the actual problem and simply and effectively as possible. Someone who is incentivized to build career capital and experience other than via impact (e.g. so they can get uplevelled, pass an external interview loop, etc) is much more likely to focus on unimportant hard pro…

> Someone whose incentives are tightly aligned with the business is going to solve the actual problem and simply and effectively as possible. Equity is the entirely the answer for cutting through all the bullshit. At least in my head. I don't know how it plays in other people's minds but mine sounds like: "If we ship and go live, I get x% of all profit moving forward in my personal scrooge mcduck money bin". Pretty b…

>> Equity is the entirely the answer for cutting through all the bullshit.

I agree for small companies which are largely founder owned. I think outside of that, Equity doesnt do much because so much effort is put into obfuscating the value/share of the equity. If you cant see the cap table, and you cant see the preference overhang, the equity is as good as worth zero. There is no discernable value for a fraction with no denominator.

Re: Imaginary problems are the root of bad software

#149

This resonates and one way to describe it is an incentive problem. Someone whose incentives are tightly aligned with the business is going to solve the actual problem and simply and effectively as possible. Someone who is incentivized to build career capital and experience other than via impact (e.g. so they can get uplevelled, pass an external interview loop, etc) is much more likely to focus on unimportant hard pro…

I don't think its just an incentive problem only. I know plenty of engineers doing premature optimization or scope creep in good faith.

Re: Imaginary problems are the root of bad software

#150

If anything it's the incentive system in software industry, which is at fault. 1. No designer is given promotion for sticking to conventional designs. It's their creative & clever designs that get them attention and career incentives. 2. No engineer is paid extra for keeping the codebase without growing too much. It's re-writes and the effort he puts in to churn out more solutions (than there are problems) that offer…

I don't know, the more I advance in my career, the more I see it as the opposite. Wide eyed developers with big designs, who are obsessed with the technical aspects of a solution, who disregard the practicalities, long term implications at the social level (who is going to maintain this, do we have people that have that skillset, is this worth the effort, does it really matter to be this elegant, or is it more important to ship quickly and economically) come off as a bit immature and the more effective engineers who understand these priorities are given more respect and authority.
Post reply on HN