71–80 of 126 posts
Earlier quoted context omitted.
BINGO all you want but if you're at liberty to disclose such things (or to confirm them) you should have included it in your original write up.
Sure, 'cause if he/she had, you would already have found a solution! No need to be rude nor arrogant, even more if pointlessly so.
Earlier quoted context omitted.
I disagree an AST would help with a project this size, it's just unmanageable. You'd be better off to start with just the build scripts and build tools. ASTs are great for increasing understanding of much smaller projects but for something this size you'd likely end up with very little to show for your effort except the crashlogs of your tools. You need to go 'coarse' before you can go 'fine' on something this magnit…
> something this magnitude Yeah, I started commenting before the realization of how HUGE this thing would be hit me, sorry :)
I think your sorry would be better directed at this guy:
Earlier quoted context omitted.
Am I missing something or are you saying that making an AST like a compiler will help you understand a huge codebase better and faster?
"making an AST like a compiler" will give you a semantic - as contrasted with textual - understanding of the code. Especially with how much data Java encodes in its source, this makes it a very good base for running automated analysis and visualisation tools, perhaps written by yourself. In general having AST is always better than having a plain text file, unless you want to read it. But then you can easily dump AST…
I have written some Scheme and I still can't say I need to screw around with ASTs. May be I will be enlightened some day?
A UML diagram for this level of hugeness would be a really useful thing according to me, much much better than an AST.
Before rebuilding anything piece of software from scratch, I would give a serious look at this amazing bunch of wisdom: http://www.joelonsoftware.com/articles/fog0000000069.html
Earlier quoted context omitted.
Any human line-by-line/application-by-application analysis is (for this particular discussion) out of the scope. The size of the thing and the way we thought we were going to work is quite different. For instance, suppose we produce AST for all the routines/pieces of logic/you_name_it we wanted to then find similar patterns or clusters that would give us hint to then work on a "pareto-like" way. As already stated it'…
Don't try to figure out how the code does what does yet. Figure out what systems exists inside it: 1. What kind of modules? 2. Which servers/hardware? 3. Which databases/datastores? 4. What systems talk to what? 5. What test systems exist or existed? 6. Which api/frameworks where used? 7. Who is currently working on them/maintaining it? 8. Is anyone left who used to? 9. Why is a rewrite on the table? 10. Is there any…
100 M lines is stupendous.
why would you need the ast? if given a large chunk of code to maintain i'll usually run doyxgen on it to generate the xmlish kind of chart that it makes. at least it gives me a roadmap to start, but it's not super great.
Earlier quoted context omitted.
"making an AST like a compiler" will give you a semantic - as contrasted with textual - understanding of the code. Especially with how much data Java encodes in its source, this makes it a very good base for running automated analysis and visualisation tools, perhaps written by yourself. In general having AST is always better than having a plain text file, unless you want to read it. But then you can easily dump AST…
Yeah I only came to know about AST around a year back. I guess I am not just that competent but I can not imagine someone trying to make sense of a codebase using ASTs. Mostly it's read the code/modify the code/debug the code for me. I have written some Scheme and I still can't say I need to screw around with ASTs. May be I will be enlightened some day? A UML diagram for this level of hugeness would be a really usefu…
As for this:
> A UML diagram for this level of hugeness would be a really useful thing
we actually agree 100% here. What I mean is that having AST is meaningless by itself, but you need AST if you want to generate UML diagram from the code. Or generate a callgraph. Or find similarities or duplication in the code. Or indeed perform any kind of automatic code transformation.
So extracting AST is a first step to developing your own tools for working with a codebase. And with a codebase of this size you just have to write your own tools, adapted to the nature of this particular codebase. So while "trying to make sense using ASTs" really is a bit hard to imagine, trying to make sense of a codebase using all the tools AST enables you to write is what I had in mind.
Callgraph. Then document the larger chunks, working your way down. It's like having a map versus having no map at all. And 100M lines? Are you sure there is no code generator at work here?
It's code that's been developed and it's been running for decades, I'm afraid.
Presumably somebody didn't write 100M lines of code on Day 1.
The code length has to be overstated, by including libraries, generated files, or data files. Is any real code that long? I bet the core java code the team actually wrote is 2 orders of magnitude smaller.