Live data from Hacker News

Ask HN: Have you ever worked on a product that was killed by technical debt?

news.ycombinator.com

41–50 of 331 posts

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#41
post #22

Earlier quoted context omitted.

The Firefox rewrite was a success by all measures. Spolsky is wrong on this one, at least with the claims of generality.

Except for one measure: Netscape died as a company. The huge rewrite contributed to killing it. If you don't ship a product (for like 4-6 years?) you're gonna die. Mozilla originally chose the name phoenix, (then firebird to avoid trademark problems, then finally firefox) was chosen because it was a phoenix rising from Netscape's ashes. Its major innovation: It was 'blazing fast' when compared to ie 5.5 / 6. Tabbed b…

I remember how long it took to release a stable version of Mozilla and Mozilla Phoenix. In the meantime, had to recompile newer releases all the time manually. There was no alternative browser on Linux or *NIX for that matter (OK, macOS still had MSIE).

The successor of Netscape Communicator was Mozilla (IIRC it was just called that, later renamed Mozilla SeaMonkey), and the successor of Netscape Navigator was Mozilla Phoenix (later renamed Mozilla Firebird and eventually Mozilla Firefox). Firefox and Thunderbird were once again separate clients.

Mozilla was still considered bloated, but Phoenix was far less bloated which is nice on lower RAM machines, and allowed the start of Web 2.0. It was also the return of doing one thing and doing it right: browsing the WWW. As Netscape Communicator (unlike its predecessor, Netscape Navigator) came with a Usenet client and e-mail client.

Later in development, addons became a thing, and you could add features which were previously part of Netscape Communicator such as calendar, HTML editor, etc. You can also add such features with addons to Mozilla Thunderbird.

Then Google Chrome happened, and people switched to that, but I'm not entirely sure why.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#42
post #27

I have seen a product getting killed by trying to resolve technical debt. The refactor took nine months and in the end didn't work better. I am a big fan of constant refactoring on a small scale but I am very skeptical of large refactoring of a whole project. You may end up with something that's just different but not really better.

I've had the opposite happen every time the team I've been on decided to refactor a large portion (or even the entire code base). Every time, what was a source of constant bugs (i.e., X bugs per week, every week, never lessening), became tractable and moved to stable post the rewrite (X bugs first week, .7x bugs second week, etc, until finally we're encountering the odd bug only once every few months, if at all).

I'm not sure what the differentiator is. I'd be curious if others have ideas. I think part of it is that in both cases it was a small team, who caught the issues early enough that it hadn't gotten too bad yet, but late enough that the right direction to move in was clear.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#43
post #34

I worked for a startup that had basically the right idea but proper execution took so long we ran out of runway. The first iteration of the product was built in an extremely haphazard, cowboy way - and took months, if not years, to refactor into something stable, usable and crash-proof. By the time the product was operational, the company was bankrupt. We simply hemorrhaged money until we bled to death. As someone el…

That may be a 'startup problem'. Do it cheap and coyboy because, runway. Assuming the money will come along later to do it all again. But that happens (lots of money later) only if you get bought out. Not if you have to make it on your own. So any business plan that includes the steps "A miracle occurs" and then "We get bought out" is probably going to suffer that fate?

Even if the miracle occurs and you get bought out and get shittons of money thrown at you, you've already built a company with a "fake it till you make it" culture. Even if you get to hire great new engineers to build your product the right way, your existing team is a ragtag bunch of amateurs who don't know how to build things properly and block any attempts to improve the status quo. I've been there. You can't build castles on the foundation of mud. You'd have to throw it all out and start again - and that's a recipe for disaster.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#44

It happened to me twice. The first time was in a start-up at the beginning of the century, we were developing an electronic health record and we had outsourced the database abstraction layer to a company in Greece. In the beginning things went fine but after a while the development of the DAL went slower and slower and it became unstable as well. Eventually the word came out: the main developer of the DAL framework h…

For the second one, you must have been receiving a lot of traffic for template rendering to be such a bottleneck. Why not upgrade the server?

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#45
post #5
post #2

I've worked in the industry as a developer for 12 years and I can't remember any. I do remember a competitor dying of not releasing their big refactored next version soon enough, and running out of cash. Spolski tells it better than me: https://www.joelonsoftware.com/2000/04/06/things-you-should-...

Spolsky should be required reading in software development classes. I wish he had time to keep it up.

I can't remember which podcast episode it was, but I do remember him mentioning a professor in South Korea had once told him he was using his blog posts for his CS classes.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#46
post #33

Knight Capital lost $465 million in 45 minutes caused (at least in part) by technical debt and poor development practices. Summary: http://pythonsweetness.tumblr.com/post/64740079543/how-to-lo...

I'd say it wasn't due to technical debt, more a start-up like development approach to a company that trades millions within seconds in full automation. It sounds like the deployment process wasn't that complicated for a company of that size, but it was deployed without a single check by a second person.

If you're trading automatically, you'll need a very, very solid deployment and audit process, even if you're just a small company. The reason banks are so slow in deploying software is because most of them lost a few millions at some point due to some bug.

Startups that think they can act faster than banks just haven't had that bug yet. That's also why I'm rather negative on the whole Fintech scene at the moment.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#47
Sort of. I worked on a project that gradually lost customers due to instability and a lack of new features.

It's still around and still being maintained but it's a shadow of what it once was.

I've worked at multiple other companies which have gone through rounds of expensive rewrites.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#48
post #37

Yes and no. The project wasn't killed specifically because "you have technical debt". It was killed because there was no way for anyone to be effective with the combination of poor undocumented code. "We need to change the email message that goes out when someone registers". This took a team of (4?) people 5 calendar days to change. As a contractor, I had to vpn in to one system, then remote desktop over another vpn…

Templates stored across a database is probably the worst thing I've seen repeatedly across projects. Just because a database can store everything doesn't mean it has to. Some people really seem(ed) to have an allergy to plain files for storage. A plain file with OS level caching will beat most (if not all) databases for static content. But doesn't sound as fancy, so it's probably harder to charge a lot of money for i…

Template in one database table I can live with (pros and cons, multiple front-ends, etc). One template broken up in to 12 tables requiring an 100+ line SQL statement with concat()s and HTML interspersed is insane. Had there been an API or utilities with it to manage it, it might have been manageable, but nope - just "write some queries".

Also, just repeated your comment to a friend who said "that's the worst thing you've seen? can i have your job?" :)

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#50
post #2

I've worked in the industry as a developer for 12 years and I can't remember any. I do remember a competitor dying of not releasing their big refactored next version soon enough, and running out of cash. Spolski tells it better than me: https://www.joelonsoftware.com/2000/04/06/things-you-should-...

The Firefox rewrite was a success by all measures. Spolsky is wrong on this one, at least with the claims of generality.

Then again Firefox was itself a strip down of a rewritten Netscape suite. Stripped down in that the suite included not just a browser but also a email client, IRC client and a HTML editor, and the UI was done using JS and XUL markup.

What Firefox devs did was to take the browser part, make it stand alone, and replace much of the XUL UI with native widgets (GTK on _nix).

Post reply on HN