Live data from Hacker News

Ask HN: I have to analyze 100M lines of Java – where do I start?

news.ycombinator.com

41–50 of 126 posts

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#41
post #35

Earlier quoted context omitted.

It's code that's been developed and it's been running for decades, I'm afraid.

developed in java and running for decades? how long has the language been around?

I started using Java in early '95 (having begged a copy from someone at Sun) and I seemed to be one of the first people writing stuff outside of Sun.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#42
1) determine the use cases of the different applications involved (ie, what is each one used for, how does it fit into the company's workflow)

2) treat each app as a black box, understand the major data flows involved (which data sources is it interacting with? is it doing reads or writes? which tables)

3) treat each app as a black box, and try to understand the interactions between each app and any external components (other apps, web services, etc)

4) identify the overall architecture, determine the class hierarchy for each app, identify the major classes and functionality

By this stage you should be ready for a rewrite. At no point do you need to go into the code in any great depth.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#43
1. Configure Jenkins builds 2. Add PMD - code analyzing 3. Add Sonar - code analyzing (they has different rules than PMD) 4. Use Archeology 3d > https://github.com/pslusarz/archeology3d to visualizing your code stats.

But before you start just pray to Omnissiah (http://warhammer40k.wikia.com/wiki/Machine_God).

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#44
Do you and your team have experience of Java development?

Your question sounds like something that someone with either no real experience and/or no experience of an object-oriented language would ask.

100 million lines is a lot of code. Why do you need to "parse it to extra the AST"? That's crazy.

Do you have the original design documents and architectural documentation? If you do, read it.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#45

What type of application has 100 million loc's? Windows 7 has 40 million lines of codes so I'm wondering why type of application/software it is.

I have a quite strong feeling that certain complex automated systems within financial services / insurances domain could reach those LOC levels. Including all the frontend side, internal backend logic, possible web services, internal tools, tests, tens to hundreds of interfaces to different kinds of external services, report generation, libraries, etc.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#47

With a codebase like that, it's better to look at it through the users' eyes, rather than trying to reverse engineer the business from the code. Things that look like bugs in the code may actually be features for the users, or may have been absorbed so long ago that they've fundamentally changed the nature of the business. You don't need to understand the whole codebase. It will take years. Best to focus on what the…

Any human line-by-line/application-by-application analysis is (for this particular discussion) out of the scope.

The size of the thing and the way we thought we were going to work is quite different.

For instance, suppose we produce AST for all the routines/pieces of logic/you_name_it we wanted to then find similar patterns or clusters that would give us hint to then work on a "pareto-like" way.

As already stated it's not ONE project, it's an old (but still running), poorly-documented codebase produce in decades around this big firm we work for.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#48

What type of application has 100 million loc's? Windows 7 has 40 million lines of codes so I'm wondering why type of application/software it is.

I have a quite strong feeling that certain complex automated systems within financial services / insurances domain could reach those LOC levels. Including all the frontend side, internal backend logic, possible web services, internal tools, tests, tens to hundreds of interfaces to different kinds of external services, report generation, libraries, etc.

BINGO!

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#49

Just curious: what application needs 100M lines of anything?

It's not just one application. It's many of them.

I agree with kubiiii.

Since a rewrite is in the card (hopefully) there is something wrong with the entire system.

* Identify where the applications interact with each other.

* Identify the most problematic applications.

* Rewrite those (starting with the smallest) while trying to keep the interfaces between applications constant.

And involve end users as much as possible.

Re: Ask HN: I have to analyze 100M lines of Java – where do I start?

#50

Earlier quoted context omitted.

How does that stop you from making a callgraph? (On the off chance that you don't know what that is: http://en.wikipedia.org/wiki/Call_graph )

It doesn't. I was just answering about the code generator thing you mentioned.

So ... is (some of) this 100M lines the output of a code generator which has since itself been lost? If so, then it seems like a good idea to take a good hard second look for it: scour every nook and cranny for source code, object code, documentation or any surviving evidence (old developer emails?) of how it worked.
Post reply on HN