Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Undebt: How We Refactored 3M Lines of Code

engineeringblog.yelp.com

Post reply on HN
Print view View original

21–30 of 143 posts

Re: Undebt: How We Refactored 3M Lines of Code

#21

Post by niftich »

Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

I maintain a line-of-business webapp that could be mistaken for a typical CRUD app, but actually has a lot of business logic enforced in code.

That's stuff you don't have to hardcode -- you can pull it out into a 'rules engine' (at the expense of an additional runtime dependency) or push it further down into, say, database stored procedures (I can hear some of you shudder). But for us, the rules rarely change, or change at a pace that's acceptable to keep up with.

Also, there are a lot of views and specialized interfaces tailored for particular workflows. In several cases, the data underneath of them is the same, but there are different UIs -- that adds LOCs considerably.

Re: Undebt: How We Refactored 3M Lines of Code

#23

Post by aidos »

hvidgaard wrote: post #18
Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

When you don't spend the time to refactor and groom the codebase, actively seek to reduce the complexity, it grows. And it grows exponentially. Once you reach the point where you're afraid to change something because it might break something unrelated, you just add new code all the time.

I've worked at a couple of places that went this way (joined after they already had millions of lines).

It was code duplication. One example I saw was a many thousand line css file included 4 times in a row. They were all different generations and some were customer specific so it was an impossible task to try to consolidate.

That was a common theme. Enterprise stuff where one customer needs slightly different behaviour so code is duplicated (maybe thousands of lines).

The first time you take that road, you've doomed yourself. With a good / small team they can manage it- but imagine when you've got 20 devs sitting there of varying experience. The precedent is set and the solution for anything mildly complex will forever be, make a copy.

Re: Undebt: How We Refactored 3M Lines of Code

#25

Post by epidemian »

bigiain wrote: post #10
wRastel27 wrote: post #5

"...time that could be better spent working on new features and shipping new code" Can we please stop putting forth this idea that features >>> reliable product? The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature. I look forward to the day where the executive team comes to the developers and ask why they are wor…

There is an old Joel On Software blog post I was reading recently that talked about a methodology at Microsoft they called "Zero defects", where fixing known bug _alway_ had priority over working on new features. Here - pont 5 in this post: http://www.joelonsoftware.com/articles/fog0000000043.html

Interesting. I recently read an article[1] where John Romero talks about the culture of early id Software and he mentions something similar:

> As soon as you see a bug, you fix it. Do not continue on. If you don’t fix your bugs your new code will be built on a buggy codebase and ensure an unstable foundation.

Looking back to the codebases i've worked with, this advice seems extremely wise.

[1] http://www.gamasutra.com/view/news/279357/Programming_princi...

Re: Undebt: How We Refactored 3M Lines of Code

#27

Post by throwaway_exer »

raverbashing wrote: post #19
spapas82 wrote: post #15

Was the 3M LOC refactoring for yelp.com? The article doesn't say. How could possibly a review site have 3M LOC?

The front-facing system is the smallest part The admin and back-office system is deeper. Beyond the reviews you also have: events, mailing list management, profile management, messages, i18n, search, etc Example: http://engineeringblog.yelp.com/2015/10/how-we-use-deep-lear... Edit: still, 3Mi lines is massive. However, I think there's something that contributes significantly: HTML and CSS

i18n generally just adds property files and maybe localized CSS, not code.

Re: Undebt: How We Refactored 3M Lines of Code

#28

Post by ggurgone »

vincnetas wrote: post #17
Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

This could be including backend tools not seen by simple user...

definitely also tests, deployment and generic tools

Re: Undebt: How We Refactored 3M Lines of Code

#29

Post by encoderer »

Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

Truly that big. Imagine even 25 full time developers working on a website for several years.

If you look at Yelp and see a simple crud app you're not looking hard enough.

Re: Undebt: How We Refactored 3M Lines of Code

#30

Post by rhizome »

aidos wrote: post #23
hvidgaard wrote: post #18

Earlier quoted context omitted.

When you don't spend the time to refactor and groom the codebase, actively seek to reduce the complexity, it grows. And it grows exponentially. Once you reach the point where you're afraid to change something because it might break something unrelated, you just add new code all the time.

I've worked at a couple of places that went this way (joined after they already had millions of lines). It was code duplication. One example I saw was a many thousand line css file included 4 times in a row. They were all different generations and some were customer specific so it was an impossible task to try to consolidate. That was a common theme. Enterprise stuff where one customer needs slightly different behavi…

Oh man, I would love to work at a place like that. If only there was a "Sandi Metz" specialization in software development. The worst part might simply be managing the egos of people whose work is refactored.
Post reply on HN