Undebt: How We Refactored 3M Lines of Code • Hacker News Board

Re: Undebt: How We Refactored 3M Lines of Code

#71

Post by olau » Wed, Aug 24, 2016, 9:58 AM UTC

rubber_duck wrote: post #42

necubi wrote: post #9

For Java, IntelliJ has a built-in version of this called "structural search and replace" [0]. This is incredibly useful when a library changes an API or you need to refactor a lot of similar code. This feels relatively safe in Java because tooling can staticly know a lot about your code (and can know for sure that a particular call site is the method or class you're targeting). I've be terrified to do it in python wi…

IMO the main reason Python standard library is so wildly inconsistent. They don't really have the tools to migrate stuff painlessly and the 'batteries included' approach with weak versioning means you can't change stuff without breaking everyone who upgrades a python version.

I don't think your theory is right (I simply don't think it has been given enough polish), but even so, the other side of the coin is that you can relatively easily hack in temporary migration paths. For instance, a function can examine the parameters it is given and convert them to the latest API, spewing out a warning.

Backwards compatibility is mostly an attitude problem.

Re: Undebt: How We Refactored 3M Lines of Code

#72

Post by codedokode » Wed, Aug 24, 2016, 10:04 AM UTC

Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

They probably only add code and never refactor or delete it (and by the way I don't think it is always bad idea if the code is organized). So a large team can write a lot in several years.

This code might also include HTML templates and CSS or JS files that can easily grow large.

Re: Undebt: How We Refactored 3M Lines of Code

#73

Post by DanWaterworth » Wed, Aug 24, 2016, 10:06 AM UTC

chimprich wrote: post #61

lmm wrote: post #55

Earlier quoted context omitted.

That bug exists because it's written in a language where + is permitted to silently do surprising things, for reasons that made sense as a performance optimization for general-purpose computers in the '70s and embedded systems in the '90s (the original target of Java) but do not make sense for general-purpose computers today. Better languages are possible. Provably correct software is possible. We really can eliminat…

> Provably correct software is possible. Ah cool, so you've solved the halting problem then?

You don't need to solve the halting problem to write provably correct software (that is, software that conforms to a formal specification). The halting problem (and Rice's theorem) come into play when you are trying to deduce things about arbitrary programs written in turing complete languages.

It's like the difference between being able to express proofs in maths, versus having a method of deciding the validity of arbitrary propositions. We can certainly write programs that we can we can prove things about, but given an arbitrary program, we can't always tell whether it will do the right thing.

Re: Undebt: How We Refactored 3M Lines of Code

#74

Post by DanWaterworth » Wed, Aug 24, 2016, 10:09 AM UTC

weavie wrote: post #66

chimprich wrote: post #61

Earlier quoted context omitted.

> Provably correct software is possible. Ah cool, so you've solved the halting problem then?

They didn't say it had to be Turing complete.

It's not about turing completeness. Deducing arbitrary properties of programming written in primitive recursive languages is also undecidable. This is why type inference in dependently typed languages is undecidable.

Re: Undebt: How We Refactored 3M Lines of Code

#75

Post by lmm » Wed, Aug 24, 2016, 10:09 AM UTC

chimprich wrote: post #61

lmm wrote: post #55

Earlier quoted context omitted.

That bug exists because it's written in a language where + is permitted to silently do surprising things, for reasons that made sense as a performance optimization for general-purpose computers in the '70s and embedded systems in the '90s (the original target of Java) but do not make sense for general-purpose computers today. Better languages are possible. Provably correct software is possible. We really can eliminat…

> Provably correct software is possible. Ah cool, so you've solved the halting problem then?

It was "solved" in 1940 (proven in 1967) by the simply typed lambda calculus.

Re: Undebt: How We Refactored 3M Lines of Code

#76

Post by bennofs » Wed, Aug 24, 2016, 10:13 AM UTC

lmm wrote: post #55

wcdolphin wrote: post #43

Earlier quoted context omitted.

I challenge that; I don't think a "bug free code base" actually exists. Joshua Bloch has a great article about this which I think may be of interest to other readers: https://research.googleblog.com/2006/06/extra-extra-read-all... . To paraphrase: We programmers need all the help we can get, and we should never assume otherwise. Careful design is great. Testing is great. Formal methods are great. Code reviews are gre…

That bug exists because it's written in a language where + is permitted to silently do surprising things, for reasons that made sense as a performance optimization for general-purpose computers in the '70s and embedded systems in the '90s (the original target of Java) but do not make sense for general-purpose computers today. Better languages are possible. Provably correct software is possible. We really can eliminat…

> Provably correct software is possible.

Even then, the software is provably correct according to some specification, which can still have bugs. No technology can fully prevent logic bugs.

Re: Undebt: How We Refactored 3M Lines of Code

#77

Post by bennofs » Wed, Aug 24, 2016, 10:14 AM UTC

lmm wrote: post #55

wcdolphin wrote: post #43

Earlier quoted context omitted.

I challenge that; I don't think a "bug free code base" actually exists. Joshua Bloch has a great article about this which I think may be of interest to other readers: https://research.googleblog.com/2006/06/extra-extra-read-all... . To paraphrase: We programmers need all the help we can get, and we should never assume otherwise. Careful design is great. Testing is great. Formal methods are great. Code reviews are gre…

That bug exists because it's written in a language where + is permitted to silently do surprising things, for reasons that made sense as a performance optimization for general-purpose computers in the '70s and embedded systems in the '90s (the original target of Java) but do not make sense for general-purpose computers today. Better languages are possible. Provably correct software is possible. We really can eliminat…

> Provably correct software is possible.

Even then, the software is provably correct according to some specification, which can still have bugs. No technology can fully prevent logic bugs.

Re: Undebt: How We Refactored 3M Lines of Code

#78

Post by DanWaterworth » Wed, Aug 24, 2016, 10:21 AM UTC

bennofs wrote: post #76

lmm wrote: post #55

Earlier quoted context omitted.

That bug exists because it's written in a language where + is permitted to silently do surprising things, for reasons that made sense as a performance optimization for general-purpose computers in the '70s and embedded systems in the '90s (the original target of Java) but do not make sense for general-purpose computers today. Better languages are possible. Provably correct software is possible. We really can eliminat…

> Provably correct software is possible. Even then, the software is provably correct according to some specification, which can still have bugs. No technology can fully prevent logic bugs.

You're technically correct, but in practice, proving that software does confirm to a formal specification is, empirically, a good way of drastically reducing defects.

Re: Undebt: How We Refactored 3M Lines of Code

#79

Post by eru » Wed, Aug 24, 2016, 10:46 AM UTC

niftich wrote: post #21

Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

I maintain a line-of-business webapp that could be mistaken for a typical CRUD app, but actually has a lot of business logic enforced in code. That's stuff you don't have to hardcode -- you can pull it out into a 'rules engine' (at the expense of an additional runtime dependency) or push it further down into, say, database stored procedures (I can hear some of you shudder). But for us, the rules rarely change, or cha…

I wouldn't draw the distinction at 'hardcode' or some alternative. Rather the question is which language expresses the business rules well enough (or even best).

If your application language also does a decent job of expressing the business logic, that's great.

Re: Undebt: How We Refactored 3M Lines of Code

#80

Post by TeeWEE » Wed, Aug 24, 2016, 10:52 AM UTC

crdoconnor wrote: post #67

* Newline at EOF * Double quoted docstring * Remove unused imports. These things are largely cosmetic.

Indeed what i was thinking. A big refactoring is often structural, and changes the program in a bigger way. In Java you can actually easly move classes around by drag-n-drop and they code will refactor. In python this is impossible.