Undebt: How We Refactored 3M Lines of Code

#1

Post by vivagn » Wed, Aug 24, 2016, 5:13 AM UTC

Undebt: How We Refactored 3M Lines of Code
engineeringblog.yelp.com

Re: Undebt: How We Refactored 3M Lines of Code

#2

Post by wry_discontent » Tue, Aug 23, 2016, 5:33 PM UTC

It seems to me this is only going to handle the most trivial kind of technical debt. This kind of tool can't manage the way you organized your codebase, for instance. There's more to refactoring than find-and-replace.

Re: Undebt: How We Refactored 3M Lines of Code

#3

Post by sfrailsdev » Tue, Aug 23, 2016, 11:17 PM UTC

wry_discontent wrote: post #2

It seems to me this is only going to handle the most trivial kind of technical debt. This kind of tool can't manage the way you organized your codebase, for instance. There's more to refactoring than find-and-replace.

Well even if it just gave you some reasonable abstraction over regex, at this kind of scale it seems useful. Plus I bet they had excellent test coverage, which really makes refactoring much easier.

Re: Undebt: How We Refactored 3M Lines of Code

#4

Post by denfromufa » Wed, Aug 24, 2016, 5:58 AM UTC

Why pyparsing, not ply or regex?

Re: Undebt: How We Refactored 3M Lines of Code

#5

Post by wRastel27 » Wed, Aug 24, 2016, 5:58 AM UTC

"...time that could be better spent working on new features and shipping new code"

Can we please stop putting forth this idea that features >>> reliable product? The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature. I look forward to the day where the executive team comes to the developers and ask why they are working on features instead of cutting down technical debt.

Re: Undebt: How We Refactored 3M Lines of Code

#6

Post by ajdlinux » Wed, Aug 24, 2016, 6:01 AM UTC

Related: http://coccinelle.lip6.fr/

Coccinelle is used extensively by Linux kernel developers for a whole tonne of things like this.

Re: Undebt: How We Refactored 3M Lines of Code

#7

Post by ajdlinux » Wed, Aug 24, 2016, 6:03 AM UTC

wRastel27 wrote: post #5

"...time that could be better spent working on new features and shipping new code" Can we please stop putting forth this idea that features >>> reliable product? The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature. I look forward to the day where the executive team comes to the developers and ask why they are wor…

"The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature."

Not always true (though it often is!), but particularly not always true in the timespan that the company needs to get sales in...

Re: Undebt: How We Refactored 3M Lines of Code

#8

Post by iMerNibor » Wed, Aug 24, 2016, 6:05 AM UTC

wRastel27 wrote: post #5

"...time that could be better spent working on new features and shipping new code" Can we please stop putting forth this idea that features >>> reliable product? The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature. I look forward to the day where the executive team comes to the developers and ask why they are wor…

technical debt will not always manifest itself as instability/bad product.

It might just affect development time of things interacting with the bad, old code - or make bugs harder to fix Changes to old nasty code will probably be hacky under the mentality "we're going to refactor this anyways, no point in writing clean code /now/"

Most old code (i've worked with) will function fine - it has for a while after all.

Re: Undebt: How We Refactored 3M Lines of Code

#9

Post by necubi » Wed, Aug 24, 2016, 6:12 AM UTC

For Java, IntelliJ has a built-in version of this called "structural search and replace" [0]. This is incredibly useful when a library changes an API or you need to refactor a lot of similar code.

This feels relatively safe in Java because tooling can staticly know a lot about your code (and can know for sure that a particular call site is the method or class you're targeting). I've be terrified to do it in python without a very thorough test suite.

[0] https://www.jetbrains.com/help/idea/2016.2/structural-search...

Re: Undebt: How We Refactored 3M Lines of Code

#10

Post by bigiain » Wed, Aug 24, 2016, 6:19 AM UTC

wRastel27 wrote: post #5

"...time that could be better spent working on new features and shipping new code" Can we please stop putting forth this idea that features >>> reliable product? The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature. I look forward to the day where the executive team comes to the developers and ask why they are wor…

There is an old Joel On Software blog post I was reading recently that talked about a methodology at Microsoft they called "Zero defects", where fixing known bug _alway_ had priority over working on new features.

Here - pont 5 in this post: http://www.joelonsoftware.com/articles/fog0000000043.html