This smells as being a need that comes as a consequence of using a dynamically-typed language. Because the example given seems to be just getting rid of the usage of a certain method, to replace with a new one. In a statically typed language, e.g. C#, you just mark the old method with an [Obsolete] attribute and go fix all the warnings. (Granted, a tool that replaces all these usages is also useful, but to me, there are much more complex ways of technical debt than just obsolete methods.)
Undebt: How We Refactored 3M Lines of Code
51–60 of 143 posts
Re: Undebt: How We Refactored 3M Lines of Code
#52Having an intern write a good blog post for the engineering blog is a great recruiting move.
Or a horrible statement at the level of company interest in tech debt - a problem given to the interns.
Re: Undebt: How We Refactored 3M Lines of Code
#53This smells as being a need that comes as a consequence of using a dynamically-typed language. Because the example given seems to be just getting rid of the usage of a certain method, to replace with a new one. In a statically typed language, e.g. C#, you just mark the old method with an [Obsolete] attribute and go fix all the warnings. (Granted, a tool that replaces all these usages is also useful, but to me, there…
In VS2015, showing all references to methods, props, classes, etc. is just one click away. Unfortunately it's not available in the community version.
https://msdn.microsoft.com/en-us/library/dn269218.aspx?f=255...
Re: Undebt: How We Refactored 3M Lines of Code
#54I would also be interested the thought process in deciding what functionality to refactor. Did you review the code and identify areas before unleashing your tool on it?
With 3M lines of code gone, it must be terrifying to feel that it may have broken something. How did you ensure that it is still working as before?
Edit: Grammatic corrections.
Re: Undebt: How We Refactored 3M Lines of Code
#55Earlier quoted context omitted.
That's by far the best way to work. It's hard to explain to someone who hasn't experienced it how much easier it is to develop in a bug free code base.
I challenge that; I don't think a "bug free code base" actually exists. Joshua Bloch has a great article about this which I think may be of interest to other readers: https://research.googleblog.com/2006/06/extra-extra-read-all... . To paraphrase: We programmers need all the help we can get, and we should never assume otherwise. Careful design is great. Testing is great. Formal methods are great. Code reviews are gre…
That bug exists because it's written in a language where + is permitted to silently do surprising things, for reasons that made sense as a performance optimization for general-purpose computers in the '70s and embedded systems in the '90s (the original target of Java) but do not make sense for general-purpose computers today.
Better languages are possible. Provably correct software is possible. We really can eliminate bugs.
Re: Undebt: How We Refactored 3M Lines of Code
#56I would also be interested the thought process in deciding what functionality to refactor. Did you review the code and identify areas before unleashing your tool on it? With 3M lines of code gone, it must be terrifying to feel that it may have broken something. How did you ensure that it is still working as before? Edit: Grammatic corrections.
I had to deal with many sizable codebases of legacy code over the years and even answered a similar question once on StackExchange. Apparently people liked the answer: http://programmers.stackexchange.com/questions/155488/ive-in...
The question was within a somewhat specific context (team of scientists, visual programming environment, etc...) but I use the same approach for all projects. Never reached 3M LoC for a single projet or system component though. More like 1.5M. Anyways, that process works for me. Maybe it does for others.
Re: Undebt: How We Refactored 3M Lines of Code
#57Was the 3M LOC refactoring for yelp.com? The article doesn't say. How could possibly a review site have 3M LOC?
If you pay coders to code, they will give you code, and if you track performance based on their output, the code remains to justify their cost. And it's just hard to delete good work. So everything grows in businesses that hire tons of (good) coders. Pay them to refactor it and they'll do that too :)
Re: Undebt: How We Refactored 3M Lines of Code
#58How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…
I think you underestimate the complexity of things that happen under the hood or simply out of sight: back-office apps, integration systems for data import/export, backups, alerting.
I'm not so familiar with Yelp, but I'd bet they're interfacing with a crap load of additional stuff, including probably in-house tools to follow leads, facilitate reviews, handle their ad programs, etc...
I've commonly seen codebases explode the 1M LoC mark. Not necessarily for a single component, but if you have multiple systems interfacing with each other it's really quite common for business applications. That's obviously excluding libs.
Frameworks are a bit responsible for this in my opinion (note: not saying frameworks are bad, but it's a side-effect), as you'd often either have some additional configuration, boilerplate code, or generated code.
Also, it your application is long-lived (think decades), it's even less surprising: new engineers come and go, and it gets harder and harder to touch the things that were maintained by the previous key-holders... So you had a new stone here and there, polish a turd here and there, but you don't really untangle the mess that sits right at the middle, because it's just way too dangerous (or so you think). And it goes on and on. And it's aggravated by the fact that, as the project grows larger, the barrier for entry for new developers get higher: it takes longer and longer to understand the system/platform in depth, and many never even try to get there.
Processes and security concerns affect this too: you're often only allowed to fix something which has a ticket assigned to it, originating from a business user. Touching anything else is a big no-no, as it would mean QA has to re-test all the things that could be impacted. Of course we can argue whether that's actually the case and how proper testing would mitigate this, but you see my point...
Re: Undebt: How We Refactored 3M Lines of Code
#59How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…
I don't know exactly what Yelp does, but assuming that they have listings for restaurants that aren't their customers, one cause could be that they take in data from lots of sources. Ideally, that is all in the same format, with an enforced data scheme. However, if you require that, you'll notice that very little data manages to make it through your entry port. Few suppliers will want to bend their system for you to…
> On top of that, once you operate world-wide, you'll learn the joy of differences in addresses. Does a country have states? Zip codes? If so, where does one specify them in an address? If you want to localize that in your app (in yelp's case, people may want to show an address to a taxi driver. For that, it would help if the address followed local conventions) line count skyrockets.
This is not the first time this issue has been faced. Why would they reinvent the wheel instead of just using libraries and conventions?
Re: Undebt: How We Refactored 3M Lines of Code
#60For Java, IntelliJ has a built-in version of this called "structural search and replace" [0]. This is incredibly useful when a library changes an API or you need to refactor a lot of similar code. This feels relatively safe in Java because tooling can staticly know a lot about your code (and can know for sure that a particular call site is the method or class you're targeting). I've be terrified to do it in python wi…
IMO the main reason Python standard library is so wildly inconsistent. They don't really have the tools to migrate stuff painlessly and the 'batteries included' approach with weak versioning means you can't change stuff without breaking everyone who upgrades a python version.
Even if they changed the standard library from version to version... the result would be that people would stop using the standard library, migration tools or no tools. Nobody really wants to deal with being pinned to a specific minor version and no older/newer - especially libraries.