[deleted]
Undebt: How We Refactored 3M Lines of Code
41–50 of 143 posts
Re: Undebt: How We Refactored 3M Lines of Code
#42For Java, IntelliJ has a built-in version of this called "structural search and replace" [0]. This is incredibly useful when a library changes an API or you need to refactor a lot of similar code. This feels relatively safe in Java because tooling can staticly know a lot about your code (and can know for sure that a particular call site is the method or class you're targeting). I've be terrified to do it in python wi…
IMO the main reason Python standard library is so wildly inconsistent. They don't really have the tools to migrate stuff painlessly and the 'batteries included' approach with weak versioning means you can't change stuff without breaking everyone who upgrades a python version.
Re: Undebt: How We Refactored 3M Lines of Code
#43Earlier quoted context omitted.
There is an old Joel On Software blog post I was reading recently that talked about a methodology at Microsoft they called "Zero defects", where fixing known bug _alway_ had priority over working on new features. Here - pont 5 in this post: http://www.joelonsoftware.com/articles/fog0000000043.html
That's by far the best way to work. It's hard to explain to someone who hasn't experienced it how much easier it is to develop in a bug free code base.
I challenge that; I don't think a "bug free code base" actually exists. Joshua Bloch has a great article about this which I think may be of interest to other readers: https://research.googleblog.com/2006/06/extra-extra-read-all....
To paraphrase: We programmers need all the help we can get, and we should never assume otherwise. Careful design is great. Testing is great. Formal methods are great. Code reviews are great. Static analysis is great. But none of these things alone are sufficient to eliminate bugs: They will always be with us. A bug can exist for half a century despite our best efforts to exterminate it. We must program carefully, defensively, and remain ever vigilant.
Re: Undebt: How We Refactored 3M Lines of Code
#44Earlier quoted context omitted.
That's by far the best way to work. It's hard to explain to someone who hasn't experienced it how much easier it is to develop in a bug free code base.
I challenge that; I don't think a "bug free code base" actually exists. Joshua Bloch has a great article about this which I think may be of interest to other readers: https://research.googleblog.com/2006/06/extra-extra-read-all... . To paraphrase: We programmers need all the help we can get, and we should never assume otherwise. Careful design is great. Testing is great. Formal methods are great. Code reviews are gre…
I don't think Spolsky was saying their product is bug free, just that they won't tolerate having known bugs.
Re: Undebt: How We Refactored 3M Lines of Code
#45Earlier quoted context omitted.
I've worked at a couple of places that went this way (joined after they already had millions of lines). It was code duplication. One example I saw was a many thousand line css file included 4 times in a row. They were all different generations and some were customer specific so it was an impossible task to try to consolidate. That was a common theme. Enterprise stuff where one customer needs slightly different behavi…
Oh man, I would love to work at a place like that. If only there was a "Sandi Metz" specialization in software development. The worst part might simply be managing the egos of people whose work is refactored.
You can find these places everywhere. Look for jobs where the software department isn't the core business and you'll find small groups of devs bogged down trying to implement another customization among code that looks like
switch(clientId){
case 42: //todo: Make sure database agrees
...
}
Because the "risk" of actually changing code for any client where things already "just work" is a testing risk, and testing is nearly always the bottleneck at these places in my experience, if they've even managed to hire a dedicated tester yet.
Of course if you apply the Joel Test you're likely avoiding these places. But they can be rewarding places to work.
Re: Undebt: How We Refactored 3M Lines of Code
#46Earlier quoted context omitted.
There is an old Joel On Software blog post I was reading recently that talked about a methodology at Microsoft they called "Zero defects", where fixing known bug _alway_ had priority over working on new features. Here - pont 5 in this post: http://www.joelonsoftware.com/articles/fog0000000043.html
I want to believe we can someday gave the same policy for bad abstractions.
Bad abstractions are what really makes code difficult to maintain in my opinion.
Re: Undebt: How We Refactored 3M Lines of Code
#47It would be nice if some research institution would pay for the rehabilitation of some huge, bloated, ancient, but relatively unimportant app. Ideally by independent teams in parallel. Just to get some real data on what works, rather than anecdotes from veterans.
Your comment opened my eyes. Considering the importance of software development in today's world ( and tomorrow's), the fact that those best practices or code management technics are found mostly in blogs, instead of scientific papers with proper experiments, tells a lot.
Re: Undebt: How We Refactored 3M Lines of Code
#48How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…
I don't know exactly what Yelp does, but assuming that they have listings for restaurants that aren't their customers, one cause could be that they take in data from lots of sources. Ideally, that is all in the same format, with an enforced data scheme.
However, if you require that, you'll notice that very little data manages to make it through your entry port. Few suppliers will want to bend their system for you to comply with your rules, and those who do will find inconsistencies in your specs or just ignore them (different encoding, not sending state abbreviations separately from city names, forget to encode HTML entities, encode them twice or thrice (my personal record find here is four levels of HTML encoding in data stored in an excel file), etc)
So, you end up with quick and dirty hacks that convert the suppliers format (or, rather, what you _think_ it is, as the supplier won't be able to tell what their format is, either) to yours, fix some egregious errors, etc.
Hundreds of data suppliers, code with tables mapping their codes to yours, or that map known errors in their input to corrections, 'smart' programmers who notice that they can replace that 10,000 entry table with corrections with 20 lines of code and a 1,000 entry table, except that, a few months on, that 20 lines have ballooned into 1,000 that nobody understands, so that this code can't be used for handling the data from a new supplier anymore, and the line count starts increasing rapidly.
On top of that, once you operate world-wide, you'll learn the joy of differences in addresses. Does a country have states? Zip codes? If so, where does one specify them in an address? If you want to localize that in your app (in yelp's case, people may want to show an address to a taxi driver. For that, it would help if the address followed local conventions) line count skyrockets.
Re: Undebt: How We Refactored 3M Lines of Code
#49The fact that you need a find and replace tool to make sensible changes to your codebase indicates that it's hugely out of control already.
Re: Undebt: How We Refactored 3M Lines of Code
#50For Java, IntelliJ has a built-in version of this called "structural search and replace" [0]. This is incredibly useful when a library changes an API or you need to refactor a lot of similar code. This feels relatively safe in Java because tooling can staticly know a lot about your code (and can know for sure that a particular call site is the method or class you're targeting). I've be terrified to do it in python wi…
IMO the main reason Python standard library is so wildly inconsistent. They don't really have the tools to migrate stuff painlessly and the 'batteries included' approach with weak versioning means you can't change stuff without breaking everyone who upgrades a python version.
Even with a consistent standard library, you can't really be confident that you've properly refactored all usages of a class or function unless you have 100% test coverage -- and even if you did, it would be difficult to automate the refactoring the way you could with a static language.