Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: I have to analyze 100M lines of Java – where do I start?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 126 posts

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#101

Post by fleitz »

sergiosgc wrote: post #26

For rewrite from scratch projects, I always start by identifying the use cases covered by the application. You don't need the code for that. Just run the application and identify what it is that it does. Then, work backwards. For each use case, use the existing code as specification of the use case behavior. At 100 million lines, I'd suspect this is either an extremely large project, where a rewrite from scratch is i…

Yup, start the profiler up, use the application, check the call graph.

At 100 MLOC the code base is probably a complete mess.

Also, simian is probably your friend as it will identify large chunks of duplicate code.

As well source control can be your friend, the older the source is the more likely it is to contain useful code. The files with the most changes will usually be where the bugs are.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#102

Post by k__ »

burnte wrote: post #99
goshx wrote: post #91

10 years ago I worked on a large project to re-write a code base written in C. Our approach was to forget about the code and document everything it did from the user's perspective. Once everything was mapped out we decided on what we were going to keep, modify or remove, and then started building everything from scratch. You can always go back to the original code to see how a particular feature was implemented and p…

Having done projects like this before as well, this is the best method. Knowing WHAT needs to be done today and tomorrow is far more important that knowing HOW it was done before. The how is only important once you know what you need to do.

lol, so true...

Funny thing is, often the users tell you "I used it for XYZ" and it was...

...never been written for XYZ

...never DID XYZ, all the results/numbers were trash, but no one noticed

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#103

Post by weinzierl »

Large projects tend to accumulate lots of unused cruft. Coverage tools like emma/jacoco but I have successfully used UCDetector [1]. It's not bullet proof but it helped me lot when analyzing code to remove the unused parts, even if there are false positives sometimes. [1] http://www.ucdetector.org/

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#104

Post by pacofvf »

andrewljohnson wrote: post #46

The code length has to be overstated, by including libraries, generated files, or data files. Is any real code that long? I bet the core java code the team actually wrote is 2 orders of magnitude smaller.

Yeah, the LOCs in one of the biggest banks in the world is close to 10M (including: Trading[eq & ficc] , Credit, Mortgages, Wholesale, Wealth Management, Quant, Risk, and also UIs, tests , reports , recons, interfaces) , I don't think a task like that was given to single person, usually with that base of code a big IT consulting company is hired.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#105

Post by stronglikedan »

sergiosgc wrote: post #26

For rewrite from scratch projects, I always start by identifying the use cases covered by the application. You don't need the code for that. Just run the application and identify what it is that it does. Then, work backwards. For each use case, use the existing code as specification of the use case behavior. At 100 million lines, I'd suspect this is either an extremely large project, where a rewrite from scratch is i…

I could be wrong, but I think the "M" means thousands in this case. I know it means thousands when dealing with inventory UOMs. 100 million lines just seems like a lot, even for a code generator, and not something that would be turned over to a single team.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#106

Post by _em_ »

goshx wrote: post #91

10 years ago I worked on a large project to re-write a code base written in C. Our approach was to forget about the code and document everything it did from the user's perspective. Once everything was mapped out we decided on what we were going to keep, modify or remove, and then started building everything from scratch. You can always go back to the original code to see how a particular feature was implemented and p…

I will totally second that approach. Rather spending time on reading code line by line, you should have your team spend time (as a tester) to figure out use-cases. One by one. The end result of those use-cases should be a user-case/requirement document which you can feed into developers cycles to start building from scratch.

You can break the steps in more agile method and have 3 sub-teams.

1. Figuring out use cases 2. Developers 3. Testers

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#107

Post by astockwell »

goshx wrote: post #91

10 years ago I worked on a large project to re-write a code base written in C. Our approach was to forget about the code and document everything it did from the user's perspective. Once everything was mapped out we decided on what we were going to keep, modify or remove, and then started building everything from scratch. You can always go back to the original code to see how a particular feature was implemented and p…

This. Also from a business- (and CYA-) perspective, this gives you a punchlist of functionality to give to executive management, which can be used for anything from doing a proper scoping exercise to actually giving you a metric to show progress against.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#108

Post by smm2000 »

jerven wrote: post #81
jacquesm wrote: post #76

Earlier quoted context omitted.

Code complexity does not increase linearly. 100 M lines is stupendous.

Not really, my project is 3 FTE for 8 years. Double it to 16 years and then multiple the number of developers by 100. Consider a large enterprise having 300 developers in multiple teams, I am not at all surprised that they can manage to write 100 million lines or so. Also I think this is really a system of systems, and in my experience probably has large parts developed by the lowest bidding firms. Which when softwar…

Productivity in LOC/developer/day drops dramatically with increase of the team and code size. Every change requires extensive testing, figuring out interdependency, etc. If codebase is bad, it can take multiple days to make 10 lines change.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#109

Post by pan69 »

goshx wrote: post #91

10 years ago I worked on a large project to re-write a code base written in C. Our approach was to forget about the code and document everything it did from the user's perspective. Once everything was mapped out we decided on what we were going to keep, modify or remove, and then started building everything from scratch. You can always go back to the original code to see how a particular feature was implemented and p…

Indeed. Code is useless without the people who wrote it being around. It's not the code that's valuable, it's the experience of the programmers who it that's valuable.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#110

Post by crpatino »

stronglikedan wrote: post #105
sergiosgc wrote: post #26

For rewrite from scratch projects, I always start by identifying the use cases covered by the application. You don't need the code for that. Just run the application and identify what it is that it does. Then, work backwards. For each use case, use the existing code as specification of the use case behavior. At 100 million lines, I'd suspect this is either an extremely large project, where a rewrite from scratch is i…

I could be wrong, but I think the "M" means thousands in this case. I know it means thousands when dealing with inventory UOMs. 100 million lines just seems like a lot, even for a code generator, and not something that would be turned over to a single team.

I don't think there are that many companies with ~10^8 LOC code bases in the world, let alone for a single product[1]:

Facebook(webapp): ~310^7 LOC Linux Kernel: ~210^7 LOC Windows XP: ~5*10^7 LOC My impression is that 10 million LOC codebase is relatively common... but past that the size of the organization (company or volunteers) needed becomes a major sorting criteria.

[1] http://www.wired.com/2013/04/facebook-windows/

Post reply on HN