Undebt: How We Refactored 3M Lines of Code • Hacker News Board

Re: Undebt: How We Refactored 3M Lines of Code

#121

Post by coredog64 » Wed, Aug 24, 2016, 4:05 PM UTC

sidlls wrote: post #108

skrebbel wrote: post #83

Earlier quoted context omitted.

I don't follow. What does it tell?

Have you heard of the term/phrase "bro science" with respect to weight lifting? I'd draw an analogy between them. Clarification: what I mean is there is a lot of ad-hoc stuff in these blogs that may be narrowly true or applicable to a given project or subset of a domain, but generally there is either significant evidence against some of these blogs' contents or the contents themselves have absolutely no formal, rigor…

I switched to Scrum, check out my GAINZ!

Re: Undebt: How We Refactored 3M Lines of Code

#122

Post by pjmorris » Wed, Aug 24, 2016, 4:08 PM UTC

sotojuan wrote: post #97

epidemian wrote: post #25

Earlier quoted context omitted.

Interesting. I recently read an article[1] where John Romero talks about the culture of early id Software and he mentions something similar: > As soon as you see a bug, you fix it. Do not continue on. If you don’t fix your bugs your new code will be built on a buggy codebase and ensure an unstable foundation. Looking back to the codebases i've worked with, this advice seems extremely wise. [1] http://www.gamasutra.co…

I guess he forgot about this when making Daikatana? :-) Good advice nonetheless. It's a bit far-fetched, but reminds me of the "if you can do it in less than fives minutes, do it now" thing.

Obligatory mention of 'Masters of Doom'. I think Romero may have been a better programmer than first-time large project manager, but he wouldn't be the only one :)

Re: Undebt: How We Refactored 3M Lines of Code

#123

Post by bpicolo » Wed, Aug 24, 2016, 4:13 PM UTC

vertex-four wrote: post #63

miguelrochefort wrote: post #59

Earlier quoted context omitted.

> On top of that, once you operate world-wide, you'll learn the joy of differences in addresses. Does a country have states? Zip codes? If so, where does one specify them in an address? If you want to localize that in your app (in yelp's case, people may want to show an address to a taxi driver. For that, it would help if the address followed local conventions) line count skyrockets. This is not the first time this i…

Yelp would likely wind up forking such a library, if they used one, when it shows bugs.

There's a layer on top of a number of i18n tools at Yelp, yep.

Re: Undebt: How We Refactored 3M Lines of Code

#124

Post by coredog64 » Wed, Aug 24, 2016, 4:14 PM UTC

denfromufa wrote: post #4

Why pyparsing, not ply or regex?

Jamie Zawinski wrote on Tue, 12 Aug 1997 13:16:22 -0700:

Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.

Re: Undebt: How We Refactored 3M Lines of Code

#125

Post by bpicolo » Wed, Aug 24, 2016, 4:21 PM UTC

ksec wrote: post #69

Interesting, I have always ( wrongly ) remember yelp as a Ruby Rails shop. Does anyone know the stack behind yelp?

It is mostly Python, both the monolith and SoA. A few services are Java, e.g. for talking to Lucene. The github site actually gives a reasonably complete view of the stack via the various tools we have for integrating with various parts of the stack. https://yelp.github.io/

Re: Undebt: How We Refactored 3M Lines of Code

#126

Post by meddlepal » Wed, Aug 24, 2016, 4:22 PM UTC

Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

Previous gig I worked at had a decade old codebase for just the web app that was a bit over 2.5 million according to one of the chief engineers. That wasn't even the complete product just a small piece.

Re: Undebt: How We Refactored 3M Lines of Code

#127

Post by manacit » Wed, Aug 24, 2016, 4:25 PM UTC

huherto wrote: post #103

niftich wrote: post #21

Earlier quoted context omitted.

I maintain a line-of-business webapp that could be mistaken for a typical CRUD app, but actually has a lot of business logic enforced in code. That's stuff you don't have to hardcode -- you can pull it out into a 'rules engine' (at the expense of an additional runtime dependency) or push it further down into, say, database stored procedures (I can hear some of you shudder). But for us, the rules rarely change, or cha…

I don't get the rules engine thing. It is still code. I rather have them hardcoded with proper source control and do frequent releases. May be when you have rules that change every hour back and forth.

The difference is, I think, that you might want to be able to ad-hoc configure the rules without changing code. Going beyond that, you might want to allow people who are not developers to add/remove/change rules around business logic. Building all of this up can be quite an effort depending on how complicated the rules get.

Re: Undebt: How We Refactored 3M Lines of Code

#128

Post by initram » Wed, Aug 24, 2016, 5:15 PM UTC

bmj wrote: post #84

r00fus wrote: post #12

Earlier quoted context omitted.

> I look forward to the day where the executive team comes to the developers and ask why they are working on features instead of cutting down technical debt. I had a Philippino friend who told me of his early experience in a Japanese development outfit - the team got praised by management (maybe not CEO) on performance improvements and code reduction. At the time I thought those priorities would never have flown in t…

Do you know what sort of project/product the team produced? I find that to be a key indicator of what sort of work management expects. If are building something that will be sold, particularly in a competitive market, management will almost always believe that new features trump reducing technical debt. To a product manager, even four weeks of technical debt reduction sounds like "no new features for a month."

>To a product manager, even four weeks of technical debt reduction sounds like "no new features for a month."

Yeah, but if you can't implement new features in under a month because the code is such a mess, then maybe spending a month cleaning it up will allow you to do more features in a shorter amount of time, thus increasing revenue faster than just hacking more features into the existing crappy codebase. Obviously it depends on the codebase, but you have to look at both long and short term goals to make good decisions.

Re: Undebt: How We Refactored 3M Lines of Code

#129

Post by sheeshkebab » Wed, Aug 24, 2016, 5:26 PM UTC

Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

I'm looking at one right now with 2m+ lines. It's more surprising that they don't have closer to 10m to 20m lines by now (3m feels smallish for their scale).

Also it's easy for business apps that have been around for 10+ years and continually worked on during that time to get to 1m-2m+ lines of code... and jump from 2m to 3m is easy too.

Re: Undebt: How We Refactored 3M Lines of Code

#130

Post by itchyouch » Wed, Aug 24, 2016, 5:29 PM UTC

Kendrick2 wrote: post #14

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting thir…

We have an in-house configuration system that generates configurations for about ~14k heterogeneously configured applications (1-3x 3-20 line config file per app) and that system is has about 451k lines in it.

Of course not all lines are code. About 140k lines are flat file database of configurations, and the other ~300k lines is code/templates.

300k lines of code to manage 14k applications. lol.

------------------

Actually did a more thorough analysis cleaning out comments and what not. The database is more manageable ~86k lines and template generation code is around ~47k lines. There's a crap ton of whitespace and comments.