Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Have you ever worked on a product that was killed by technical debt?

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 331 posts

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#131

Post by coredog64 »

Bartweiss wrote: post #102
acdha wrote: post #94

Earlier quoted context omitted.

You need a lot more than that to handle CSV in the wild (quoting, Unicode, line termination, etc.) but the real killer I see is when it's edited by humans. The special cases for errors and inconsistencies will add up quickly; in some cases you may be able to reject invalid data but you may not have that option or an easy way to tell whether any particular value is wrong. Excel takes that, adds some fun things like pe…

I know of at least one company whose entire business is handling this stuff. They find growing companies as they hit critical mass and need to move their Excel data into a real database. The product is just "Your data is hideous and was entered by hand without validation or formatting; it'll never convert and it'll be wrong when it does. We can help." They handle all kinds of theory and technical stuff, like normaliz…

I used to use Google Refine (now OpenRefine [0]) for this. It lets you load up the data and then apply rules to see if they are mostly correct. It doesn't get you all the way, but it is better than going blind on manually revising a huge Excel "database".

[0] http://openrefine.org/

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#132

Post by pavel_lishin »

ryandrake wrote: post #107
Bartweiss wrote: post #93

Earlier quoted context omitted.

This is a great example of how technical debt 'kills'. It's not a murder, it's negligence and a slow demise. I went through one of these projects. The tech debt was never as bad as you describe, but it was a small company operating on a short runway. It also taught me an unfortunate lesson about non-technical founders and the dangers of outsourced code. The MVP for the company had been bought off the shelf. It worked…

Seen this a lot. A lot of companies think they are "product" companies, but due to their unwillingness to push back on customers, they become custom engineering shops, bolting on little one-off mods to their project over and over to appease bad customers (or to appease POTENTIAL customers who haven't even bought the product yet). Stop me when you recognize this one: "Hey your product is great, but we really want some…

"Let us use our logo" doesn't seem like an unreasonable request.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#133

Post by true_religion »

piratebroadcast wrote: post #4

I did work on a project that was Rails 3, and for various reasons, it can never be upgraded without basically rebuilding the entire system. They keep hoping for an exit that will simply never happen because who would buy a rotting system like that? I think myself and only a few other devs there understood the situation. Its not like sales or the CEO fully understood.

People buy businesses for the business.... not the software.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#134

Post by pizza234 »

samwilliams wrote: post #90
byroot wrote: post #77

Earlier quoted context omitted.

I don't think I ever seen an Excel / CSV import implementation that wasn't a huge mess.

Do you mean Excel to CSV? Or a CSV importer? I totally agree about Excel importing, but CSV is trivial, no? Here is an Erlang version I happened to write yesterday: lists:map( fun(Row) -> string:tokens(Row, [SepChar]) end, string:tokens(InputStr, "\n") ). EDIT: I know this version won't support escaped separator/newline characters, but I made it for a specific use case in which I knew that would not occur. Adding tha…

CSV is trivial? You may have missed this:

    http://tburette.github.io/blog/2014/05/25/so-you-want-to-write-your-own-CSV-code/
:-)

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#135

Post by kr0 »

My first company was consumed by technical debt. We all bickered about refactoring the database schema and improving the code, but investors wanted more "rooftops" (customers) now.

The company IP and employees were absorbed by the highest bidder and the company lost its funding after the CEO and CTO left.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#136

Post by samwilliams »

byroot wrote: post #129
samwilliams wrote: post #90

Earlier quoted context omitted.

Do you mean Excel to CSV? Or a CSV importer? I totally agree about Excel importing, but CSV is trivial, no? Here is an Erlang version I happened to write yesterday: lists:map( fun(Row) -> string:tokens(Row, [SepChar]) end, string:tokens(InputStr, "\n") ). EDIT: I know this version won't support escaped separator/newline characters, but I made it for a specific use case in which I knew that would not occur. Adding tha…

I'm not talking about parsing. It's a mess in it's own right of course (encoding, line terminators, etc as others mentioned). I'm talking about the actual conversion from tabular data to relational. Most of the applications I've worked on had this in one form or another. So you end up with users downloading an export of their data in CSV, editing it in Excel in various ways, and then reimporting it in the application…

Ah right, yes, I can imagine that would be extremely messy!

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#137

Post by tboyd47 »

byroot wrote: post #77
tboyd47 wrote: post #66

The closest I've come was a Rails project I inherited from a star developer who had just left the company. It was a B2B project that involved importing large Excel spreadsheets of various different formats into a standardized database for itemized review. The code was pretty sloppy, but didn't deviate much from standard Rails idioms. Not many people on the team understood Rails well enough to read it, but I did. Bug…

I don't think I ever seen an Excel / CSV import implementation that wasn't a huge mess.

Ruby has had good ETL libraries for a long time. In my opinion, our product team was too lenient concerning the format of the Excel files. Asking customers to fill out a template spreadsheet to submit to our system, rather than letting them submit any old XLS file they happen to have on their computer, would have gone a long way towards simplifying the problem space.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#138

Post by taeric »

mattmanser wrote: post #127
taeric wrote: post #80

Earlier quoted context omitted.

This sounds less like technical debt, and more like liabilities of over engineering. Possibly feature creep. That is, technical debt is not necessarily tangled over-engineered code. It is more compromises that were made to actually ship and operate in the world. You can see this in the world with devices. Consider, technical debt is the reason you have AC delivered to your house going through as many converters as yo…

Saying that technical debt is only deliberate is an old argument[1], but usage defines meaning and modern usage is that "technical debt" is a catch-all term. It just means bad code we know should be fixed. [1] 2009 - https://martinfowler.com/bliki/TechnicalDebtQuadrant.html

That seems to indicate it is a worthless term, then.

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#139

Post by maxxxxx »

lostcolony wrote: post #121
maxxxxx wrote: post #56

Earlier quoted context omitted.

I was talking about enterprise projects that had had years of development, always changing personnel and had to follow complex and changing business rules. These tend to be ugly and difficult to work with after a few years of development. In my view the only way to deal with these is to break them down into smaller components and then refactor. But that turns then into a political issue because the managers (and a lo…

Yeah, I could see that. I've been in those environments too; I have no data points from that, because getting the okay to refactor was so hard it never happened while I was on the team (I'd been on projects that claimed to have refactored the code, but then it was mixed as to whether people claimed it was a success or a waste).

I always tell the younger guys not to try to get an explicit OK to refactor but just add 20% to all estimates and use that for continuous refactoring without asking. It's just a regular part of professional work like writing code, pull requests and testing. This also has the advantage that refactors are relatively small so you can rollback if it turns out that the idea for the refactor was wrong (yes, this happens:-) ).

Re: Ask HN: Have you ever worked on a product that was killed by technical debt?

#140

Post by halis »

I have seen projects that were all but dead from technical debt, that management simply refused to believe were dead.

I once worked on a system that was so messed up that fixing any bug would create 2 more. At one point, the entire team was only working bug fixes for 6 months straight and in the end, we had more bugs than we started with.

We tried to refactor the code several times but it was just so fucked up.

This was the stuff of nightmares. Most of the module consisted of one class with 50,000+ lines of VB.NET code in a single file.

Global mutating state referenced in functions everywhere. Functions that were 500-1000 lines long (today I have ESLint limit functions to 10 lines).

After working on it for over a year, I recommended to my boss that they initiate a complete re-write and made it clear that in my estimation there was NO saving this code base.

I got moved to working on single page applications in JavaScript, but when I quit that job and moved on roughly a year and a half later, that code base was still in production, generating ever more bugs for that team.

Post reply on HN