Live data from Hacker News

How terrible code gets written by sane people

chrismm.com

41–50 of 151 posts

Re: How terrible code gets written by sane people

#41
post #28
post #26

Earlier quoted context omitted.

I sympathize with this, but perhaps it is really more economical to write software like this in your company's case (?)

It appears that way short term, but long term, having a bad codebase can retard a company's growth through lack of scalability and maintainability (read easily add/remove/change features). Companies spend a hell of a lot of money for growth. If they don't know how to manage software development, then the software will nullify all that cost spend on growing, because the system can't handle it. I am truly amazed at how…

What I've learned is that most companies that are really not in it for the long haul and just want to exit will pass the technical debt off to a much larger company that can supposedly shoulder the burdens of technical debt. The M&A process at large companies really don't look at how sustainable the codebase or infrastructure is - they only look at regulatory liabilities like super bad security practices and that's only if you're in a strongly regulated industry.

So, the current incentives mean "bang out code super fast, get rich, and someone else will figure it out." This attitude is a huge part of how so many bad acquisitions seem to be happening for the past maybe 10 years by various large technology companies as the VC owned market has grown so much compared to IPOed companies. Companies like HP, Yahoo, Dell, IBM, etc. are all in varying states of decomposition. Yes, the major tech giants are doing just fine but their M&A approach seems to be substantially different and they try to take on smaller companies and grow them before they have too much inertia keeping them from being adaptable.

Enterprise integration is a really hard problem that at this point is almost entirely impossible to approach algorithmically or from a technical solution. Sadly, you're not about to impress anyone in a technical interview how you managed to get a company's horrendous codebase to cleanly integrate with a big behemoth ESB - that doesn't signal anything about your ability to code or work with others evidently. Worse for the acquirer, the refactoring and painstaking tasks of integration are usually done with an army of hired guns that are very expensive and the work is non-scalable due to being entirely business-specific. In the process leading to the acquisition, many execs burn through their technical staff or their cap tables are messy and they wind up giving little to the engineers. The resentment alone causes a mass exodus and those most knowledgeable about the codebase depart while the execs have solidly backed their exits legally to get plenty of compensation while marking off a successful exit for future investors to look at as a positive signal for investment. Golden parachutes and different kinds of leashes hardly help until the next company trying to do the very same thing comes calling renewing the cycle of rewarding throwaway technology and IP. Then again in enterprise, 90% of what's being bought are patents and customer bases that have high switching costs with some vague notion of "alignment" with some marketecture diagram made by someone that hasn't touched or seen anything besides a sales demo in decades, so perhaps perception and suspension of disbelief is all that matters.

In many respects, I view a lot of codebases out there as evidence of the tragedy of the commons - it is a side effect of ignored externalities by every actor. There are so few incentives put into the market to make the cost of software maintenance lower it's mind-boggling how technology companies can stay in business.

Re: How terrible code gets written by sane people

#42
post #36
post #28

Earlier quoted context omitted.

It appears that way short term, but long term, having a bad codebase can retard a company's growth through lack of scalability and maintainability (read easily add/remove/change features). Companies spend a hell of a lot of money for growth. If they don't know how to manage software development, then the software will nullify all that cost spend on growing, because the system can't handle it. I am truly amazed at how…

> It appears that way short term, but long term, having a bad codebase can retard a company's growth through lack of scalability and maintainability Still, it depends on the specifics. Many, many projects get redone/canceled/completely respecified after first contact with market / customer. Many are also canceled for reasons that are independent of the progress speed. For all these projects, speed-to-market has value…

>Still, it depends on the specifics.

I agree, the devil is in the details, but intentionally writing hastily thought out software is rarely a good option. Even for a POC or MVP, code is expected to be churned out fast, only to be heavily refactored or rewritten once proven successful, but that never seems to get budgeted and people end up building on top of that.

I always go back to the KISS, DRY and YAGNI principles. They not only save overall development time, even in the short term, but they typically lead to maintainable, scalable, and expandable software.

Re: How terrible code gets written by sane people

#43
post #26
post #22

I work on a lot of projects like this. The non-techies running the project are usually clueless and programmers are out of touch with modern best practices. The funny thing is that it makes me look slow for not finishing tasks. People in charge want to see features banged out. The guys who write hundred-line if-defs or copy/paste code from one file to another, get to go home at 5pm and live their life. Someone else p…

I sympathize with this, but perhaps it is really more economical to write software like this in your company's case (?)

Maybe. Usually I'm hired to help out later in the project, when the original developers can't handle the ever increasing number of bugs and edge-cases.

Re: How terrible code gets written by sane people

#44
A number of points in the post/article are questionable.

First, it assumes the developers had substantial control over the schedule for the project ("Giving excessive importance to estimates"). Certainly in my experience this is unusual. More frequently, the schedule is dictated by management, frequently by sales/marketing executives in commercial software development. It is very difficult to push back and a good way to lose your job.

Sales: We have closed this great deal with BigCorp. Can you do X (complicated, challenging software project) by the end of the quarter?

Developers: Err, um, X sounds like a project that will take six months.

Sales: We really need to make our quarterly numbers. Our CEO Bob used to be a developer and he says any competent programmer can do it and we only hire the best. Competent doesn’t cut it here! You are a rockstar ninja, aren’t you? Can you prove you can’t do it by the end of the quarter?

Developers: Well, no. The schedules are driven by some unexpected problem or problems that usually happen. But, well, if nothing unexpected happens, we can do it by the end of the quarter.

Sales: Great! Bob is expecting results by the end of the quarter.

So much for the beautiful, elegant software design methodologies taught in college and university CS programs and peddled by high priced consultants.

Second (“Giving no importance to project knowledge”), high technology employers seem to have extremely high turnover rates of software developers and other employers. Payscale produced a study claiming that the average employee tenure and Amazon and Google is only one year. Many companies seem to target employees with more than seven years of paid work experience — Logan’s Run style — for layoffs and “constructive discharge,” (https://en.wikipedia.org/wiki/Constructive_dismissal) where employees are made uncomfortable and quit “voluntarily.” Undoubtedly, this is costly as the author implies, but it seems to be common practice.

Yes, metrics like “issues closed,” “commits per day,” or “lines of code” don’t work very well. Once employees realize they are being tracked and evaluated on some metric, they have a strong motivation to figure out how to manipulate the metric. Even if the employees don’t try to manipulate the metrics, the metrics all have serious weaknesses and map imperfectly to value added (biz speak).

Third, are code reviews and unit testing proven processes especially for normal non-Microsoft companies? In the early days of Test Driven Development (TDD), Kent Beck and his colleagues made numerous claims about the success of Test Driven Development in the Chrysler Comprehensive Compensation System (C3) payroll project, an attempt to create a unified company wide payroll system for Chrysler. This project in fact had a range of problems and was eventually cancelled by Chrysler in 2000, without replacing the Chrysler payroll systems successfully.

As the problems with C3 have become well documented and well known, TDD enthusiasts have shifted to citing studies at Microsoft and some other gigantic companies that claim practices like TDD and code reviews work well. Are these really true or do these case studies have hidden issues as C3 did?

Further, Microsoft, Google, and other companies that have played a big role in promoting these practices are very unusual companies, phenomenally successful super-unicorns with sales in the range of 40-100 billion (with a B) dollars with near monopoly positions and anomalously high revenues and frequently profits per employee. Microsoft claims to have revenues of $732,224 per employee. Google claims an astonishing $1,154,896 per employee. (http://www.businessinsider.com/top-tech-companies-revenue-pe...) This compares to $100-200,000 per employee for most successful companies.

Fergus Henderson at Google recently published an article on Google’s software engineering practices (https://arxiv.org/abs/1702.01715) with the following statements:

2.11. Frequent rewrites

Most software at Google gets rewritten every few years.

This may seem incredibly costly. Indeed, it does consume a large fraction of Google’s resources.

Note: “incredibly costly”

Companies like Microsoft and Google have enormous resources including monopoly power and can follow practices that are extremely costly and inefficient, which may work for them. Even if these practices are quite harmful, they have the resources to succeed nonetheless — at least for the immediate future, the next five years.

From a business point of view, it may even be in the interests of Microsoft, Google, and other giant near monopolies to promote software development practices that smaller competitors and potential competitors simply can’t afford and that will bankrupt them if adopted.

Both code reviews and unit tests are clearly time consuming up front. Code reviews using tools like Google’s Gerrit or Phabricator (a spin-off from Facebook, another super-unicorn) are committee meetings on every line of code.

Regarding:

Imagine my dismay when I had to collaborate with a colleague on that legacy project and his screen displayed Notepad in its full glory. Using “search” to find methods might have been rad back in the nineties, but these days, refraining from using tools such as modern IDEs, version control and code inspection will set you back tremendously. They are now absolutely required for projects of any size.

Using “search” to find methods was not rad back in the 1990’s. IDE’s and code browsers specifically have been in widespread use since the 1980’s. Turbo Pascal (https://en.wikipedia.org/wiki/Turbo_Pascal) was introduced in 1983 and featured a fully functional IDE, soon to be followed by IDE’s in many other products. Version control dates back at least to SCCS (https://en.wikipedia.org/wiki/Source_Code_Control_System) which was released in 1972. RCS was released in 1981 and version control was common in the 1980s and since.

Code reviews have been around for a long time. However, in the 1990’s and earlier they were restricted to relatively special projects such as the Space Shuttle avionics where very high levels or safety and reliability, far beyond most commercial software, were required. This speaks to the “incredibly costly” quote about Google above.

Without more context, it is difficult to evaluate the use of Notepad. Simple code/text editors like Notepad and vim (formerly vi ) are very fast to start up and can be a better option for some quick projects than starting an IDE.

Some IDE’s are particularly hard to use. Early versions of Apple’s Xcode circa 2010 were particularly difficult to use in practice; it has improved somewhat in the current releases.

People vary significantly. Some developers seem to find stripped down tools like vim or Notepad or Notepad++ (on Windows) a better option than complicated IDE’s. I am more of an emacs or IDE person.

The fact that someone else works differently than you do does not mean they are worse (or better) than you. The fact that something works well for someone else also does not mean it will work well for you — or vice versa.

There are sound reasons for duplicating code, cutting and pasting, rather than creating a function or object called in several location in the code. If the developer anticipates that the code may subsequently diverge, then duplication is often best.

Like grand master chess players, highly experienced developers, especially under tight time constraints (like a chess tournament), code by intuition, not by laboriously reasoning out every step. If it feels like the code is likely to diverge in the future, duplicate. If it does not diverge, no problem, it can be merged back later if needed.

In the bad old days of structured design (1980’s) and object-oriented design (OOD — 1990s), software development projects suffered from Big Design Up Front (BDUF), grandiose attempts to design a perfect software system before writing a line of code. This often resulted in massive cost and schedule overruns and total failures. It often proves better to just throw (“hack”) something together quickly — a prototype, proof of concept, Minimum Viable Product (MVP). Just “get something working.”

Inevitably these prototypes and early stage software projects are going to compare poorly to some theoretical perfectly designed system with 20-20 hindsight. That is what seduced people into BDUF twenty, thirty years ago.

Modern Agile software development methodologies are foolishly trying to have it both ways, have an initial quick iteration BUT that first iteration should be perfectly designed up front — beautiful, elegant, with hundreds of tests, endless committee meetings on coding style and design (code reviews), all sorts of supposed best practices, no code duplication, etc. This is a seductive fantasy doomed to fail in most cases.

Re: How terrible code gets written by sane people

#46
post #38

Honestly, I think the best developers are not the ones that write beautiful code and put quality above everything else. The best ones are the ones that can push out a solution given too little time and given a (maybe self inflicted) bad code base. Because that's real life, and not the pony farm. Money trumps everything else in capitalism. Getting money means paying your bills today. And even more so than skill, quali…

> Money trumps everything else in capitalism. Getting money means paying your bills today. And even more so than skill, quality and code getting money requires kowtowing and overpromising to someone who currently has money.

That has little to do with what makes development and a lot more to do with capitalism, business, and markets. What this really means is that current business environments don't necessarily value development, but that doesn't change the definition of development. At this point, you're not discussing a good developer, but a person who can hybridize development and business sense. It's a different skillset, perhaps one that's somewhat contradictory to high quality development, even.

There are situations in which there is no money at all, yet you absolutely need good development skills and they are distinguishable (open source). Any given person can be devoted to selling, whether they're coding or weaving baskets, but that doesn't suddenly mean that a core factor of weaving baskets is being able to sell them. It's not. That's a different skill set entirely. It may often not be found side by side.

It's a shame that we are so obsessed with money and selling right now that pure skills seem to have little value to some people, despite the fact that many crucial things nonetheless run off of these core skills...

This argument is equivalent to saying that only applied science has value.

Re: How terrible code gets written by sane people

#47
post #37
post #28

Earlier quoted context omitted.

It appears that way short term, but long term, having a bad codebase can retard a company's growth through lack of scalability and maintainability (read easily add/remove/change features). Companies spend a hell of a lot of money for growth. If they don't know how to manage software development, then the software will nullify all that cost spend on growing, because the system can't handle it. I am truly amazed at how…

> I am truly amazed at how bad many US businesses are at managing their software systems. I'm guessing you'd be equally amazed at how many people are embracing rigorous software engineering principles, and not ever reaching the point of getting a viable business off the ground. So, perhaps the best way to go about it is to sell "prototypes" at first, and then when the business catches on, hire some CS people to manag…

>I'm guessing you'd be equally amazed at how many people are embracing rigorous software engineering principles, and not ever reaching the point of getting a viable business off the ground.

Yes. It's very strange the process business impose on software teams when the business is really just stabbing in the dark. I think it's because they don't realize software development is really an art rather than a science.

>So, perhaps the best way to go about it is to sell "prototypes" at first

Yes, I think that is a good method. Really everything is a prototype from a mile high view. Word 2.0 from today's view is certainly a prototype. A sellable running prototype.

In my experience, business process means almost nothing in software development, it's the people. Good people are expensive but you will most likely fail without at least a few of them.

Re: How terrible code gets written by sane people

#48
post #18
post #14

Earlier quoted context omitted.

So true. A 'good' programmer who commits terrible code is a bad programmer. In other words, one trait of a senior developer is knowing to pick your battles. If you can't do good work, you have to get out.

I think is is true at the earlier stages of one's career. One you get older and have to start feeding a family, the idealism wears off and you start finding yourself compromising your morals and coding shit to make a deadline. But the company you work for pays you so well, coupled with the expert domain knowledge that makes you nearly irreplaceable, you become complacent. You are a fixed cog in the system. At this po…

No one said it was easy...

Re: How terrible code gets written by sane people

#50
post #35
post #21

Earlier quoted context omitted.

>is the stupidest thing rationalists ever cooked up. Oh, you don't like bonuses then? Unsolicited, forced code review doesn't have much value that I've found, unless your team is fairly junior. Senior guys know what good code looks like, even in a crunch. If you hold forced, unsolicited code reviews with senior developers, you are really just throwing away money and aggravating people. It's just another half thought…

A reviewer soon understands that senior devs are producing good code and stops investing time on reviewing their code. If they are doing their job properly. But again it comes down to internal motivation and self management on their part.

Yes, sorry I should have been more explicit. Initial reviews have value (is this guy full of beans or does he know what he's doing), but constant reviews of senior developer code do not. If you've hired anyone you have to constantly do reviews for, you've probably hired the wrong person. I mean they should get what you are looking for after a month or two.
Post reply on HN