No, but I have worked on projects that were killed by a fear of technical debt.
Ask HN: Have you ever worked on a product that was killed by technical debt?
241–250 of 331 posts
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#242It happened to me twice. The first time was in a start-up at the beginning of the century, we were developing an electronic health record and we had outsourced the database abstraction layer to a company in Greece. In the beginning things went fine but after a while the development of the DAL went slower and slower and it became unstable as well. Eventually the word came out: the main developer of the DAL framework h…
> We tried to take over the framework by ourselves but it was written in Eiffel and the code was a horrible entangled mess. Eventually we rewrote it in Java but, being a start-up, we lost too much precious time already
I wonder, would the result be different if you had access to competent Eiffel developers? How large was the Eiffel codebase?
Eiffel is an interesting language, with a somewhat unique feature-set (I think only Ada is coming close). Design by contract and static typing as core language features - if used right - should greatly help with both stability and ease of refactoring.
How large the codebase was is an important question, also how bad it really was. I saw a similar story - external codebase getting worse and worse from some point on - with Clojure at the center. The code quality was quite ok for a couple of months, then it worsened. At that point and for a couple of following months the codebase was possible to save - a single competent Clojure programmer would make a difference, I think. The project was less than 10k LOC then. However, more than 1.5 years and 60k LOC later, doing anything became nearly impossible for anyone, including original authors.
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#243I am right now, but I can't take a step back and fix any of it because that wouldn't be "Agile". The project won't be declared a failure though, they will just keep adding more and more "resources" aka people to get the same amount of work done.
tip: problem is not agile or scrum or whatever but your PO ;-)
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#244I was CTO of a company that had a two-week outage due to technical debt. I didn't sleep much for any of it. We fixed it, and we'd lost about 30% of our subscriber base in that period. The company took on new funding to survive, invested that in a new set of products, and shuttered the old stuff just to stay afloat. I am currently working in a business where there is a nearly 8-year old Rails app (600+ models, 250+ co…
IMO this is the price to pay for a dynamically typed language. 60K LOC is not much in a static language, you can use tools to refactor it easily or to visualize the control flow.
But with a dynamically typed language? Its a nightmare. You change one thing and cannot possibly know what else could have gone wrong.
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#245How about: All Of Them
Every failed product/project I've worked on in my professional career, which had full intent to ship from the start, was killed by technical debt. It's usually indirect, but it's always the root cause.
It takes many forms:
* Too buggy to ship, due to a creaky old code base being over-stretched to a product with too high reliability/experience expectations.
* Product form factor, efficiency, user experience not good enough to sell well, due to spaghetti code base which couldn't be whittled down to removable pieces. Result: large runtime, more expensive, less efficient hardware.
* Existing old codebase deemed too bad to ship a product, requiring a rewrite-from-scratch, but timescale too long to make any sense -> product killed.
It's difficult to elaborate more while maintaining some discretion about exact companies and projects. The general point is: technical debt isn't just some fuzzy intangible issue — it indirectly creates enormous costs in people and time, can affect the physical form products take on, and impact the user experience. Products always get started without taking this debt into account, but when it's finally realized, it can change basic features, and then it kills them.
Products are designed with faulty assumptions about what existing resources can be applied to them.
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#246I'm currently working on such a product.
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#247Earlier quoted context omitted.
We're an enterprise software shop, which necessarily means we do a lot of custom work, but we're careful to consider what we'll do. My mentor is an old hand that been through multiple exits, and in every meeting we have, he hammers this point. You're either a product shop, or a professional services shop, and if you don't know which one you are (or you believe wrongly) you die. Simple as that. The deeper you get into…
Can you expand on what your mentor defined as a "product shop" vs "professional services shop"?
Sure, but the answer is pretty trivial: If you spend more than half your time on customization, you're a professional services shop.
He also added that if you're a product shop doing less than 70% off-the-shelf, you're probably screwed, while 90% off the shelf is really the ideal (again, enterprise software).
I think the more interesting question is "what counts as professional services?" This gets much trickier, for example when you start building out APIs to make second- or third-party integrations easier, is that "product" or "professional services"? It certainly seems like product building, but if you're doing for a customer's use, it gets real blurry real fast. If you're not using that API internally, you're almost certainly on the professional services side. If you do use it internally, is it rock solid enough that you can support and expose it without that support becoming professional services?
Drawing sharp lines aside, this all probably seems kind of trivial, but the first time I ran through our product design with him and we discussed this, I went back and radically re-thought a lot of our strategy, particularly at the customer interfaces.
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#248Earlier quoted context omitted.
I'd say it wasn't due to technical debt, more a start-up like development approach to a company that trades millions within seconds in full automation. It sounds like the deployment process wasn't that complicated for a company of that size, but it was deployed without a single check by a second person. If you're trading automatically, you'll need a very, very solid deployment and audit process, even if you're just a…
That's not a startup approach. That's an enterprise approach. Believe me, I've fought tons of resistance in automating deployment operations in the enterprise. There's a perception that automation is dangerous, and you need human checkpoints. In practice, I've worked on projects much, much larger than Knight Capital, where the deployment process was driven by a huge spreadsheet, and orchestrated by non-technical over…
Yeah, the amount of resistance towards automation at some larger companies is a total mindfuck.
My first ops job back in 2008 was at a large exchange's NOC where we shut down and clean the application environment every day. Every Friday, we would have to take a backup of the ~20 or so production databases - by hand, in an ancient CDE based UI with a . Right click -> menu -> submenu -> backup database. Very little room for error, and you weren't allowed to do it without somebody else watching you. Throughout the weekend, customers would then run tests against the production databases. Once testing was done, we'd restore the prod databases back to their original state to wipe out test data.
At one point, I asked my boss if it was alright if I automated it after showing him a POC and was rejected because, "We don't trust automation to do it accurately every single time." Mind boggling. In mild fairness, in the 15 or so years they were doing that, I don't think anyone did it wrong.. which is an enormous miracle in itself.
(That was a strange company. My boss was a JW who'd worked there for 30 years regularly tried to convert me and would spend four hours a day on spreadsheets for his church. We'd also manually kick off stock split processing from a ~10" CRT monitor from the early nineties.)
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#249Earlier quoted context omitted.
This sounds less like technical debt, and more like liabilities of over engineering. Possibly feature creep. That is, technical debt is not necessarily tangled over-engineered code. It is more compromises that were made to actually ship and operate in the world. You can see this in the world with devices. Consider, technical debt is the reason you have AC delivered to your house going through as many converters as yo…
Saying that technical debt is only deliberate is an old argument[1], but usage defines meaning and modern usage is that "technical debt" is a catch-all term. It just means bad code we know should be fixed. [1] 2009 - https://martinfowler.com/bliki/TechnicalDebtQuadrant.html
Stretching the debt analogy, you can go bankrupt from payday loans (the "just push it out" tech debt) and from getting too big of a mortgage to build/fix up a house (over-engineered tech debt).
Re: Ask HN: Have you ever worked on a product that was killed by technical debt?
#250Earlier quoted context omitted.
Yeah, that. But even things like cvs, that were modern and hip in 1998, are still floating around 20 years later. I actually thought Subversion would be the last version control system, when it came out. Of course, now it's git. Maybe someday we'll get something better, and git will look decrepit.
I have little doubt that git will be replaced eventually (or perhaps severely modified). It seems like a fad to me. It is indeed very powerful, but its UI is sheer insanity. At least SVN is very straightforward to use and understand. It just doesn't offer the distributed nature that git does, as it relies on a centralized server.
The cool thing about Git is that the underpinnings ultimately boil down to a key-value data store.
I'm not an expert on these innerworkings, but in theory there's nothing stopping someone from creating a new UI that maintains most or all of the same strengths, except that everyone already uses and is used to the current way.
I suspect if you came out with "SuperVCS" that was ultimately just a new UI on Git you'd have more success than releasing the exact same project as some kind of Git enhancement.