"making an AST like a compiler" will give you a semantic - as contrasted with textual - understanding of the code. Especially with how much data Java encodes in its source, this makes it a very good base for running automated analysis and visualisation tools, perhaps written by yourself.
In general having AST is always better than having a plain text file, unless you want to read it. But then you can easily dump AST back to text whenever you want.
Yeah, making AST will help you analyse your codebase programmatically which in turn will let you understand the codebase better and faster. This is some very basic programming knowledge, I think. Or is it not? Some comments here don't know what AST even is - is this the state of PL knowledge in the mainstream? Lisp and Smalltalk people would be very, very sad if it was so.
I disagree an AST would help with a project this size, it's just unmanageable.
You'd be better off to start with just the build scripts and build tools.
ASTs are great for increasing understanding of much smaller projects but for something this size you'd likely end up with very little to show for your effort except the crashlogs of your tools.
You need to go 'coarse' before you can go 'fine' on something this magnitude.
This is not a 3 week project, just mapping the thing properly will take (man)years.
Yeah I only came to know about AST around a year back. I guess I am not just that competent but I can not imagine someone trying to make sense of a codebase using ASTs. Mostly it's read the code/modify the code/debug the code for me.
I have written some Scheme and I still can't say I need to screw around with ASTs. May be I will be enlightened some day?
A UML diagram for this level of hugeness would be a really useful thing according to me, much much better than an AST.
I probably overreacted, sorry for this - working on programming languages and programming tools is not what most of programmers do (obviously) and that's the only domain where you need to know about what you can use AST for.
As for this:
> A UML diagram for this level of hugeness would be a really useful thing
we actually agree 100% here. What I mean is that having AST is meaningless by itself, but you need AST if you want to generate UML diagram from the code. Or generate a callgraph. Or find similarities or duplication in the code. Or indeed perform any kind of automatic code transformation.
So extracting AST is a first step to developing your own tools for working with a codebase. And with a codebase of this size you just have to write your own tools, adapted to the nature of this particular codebase. So while "trying to make sense using ASTs" really is a bit hard to imagine, trying to make sense of a codebase using all the tools AST enables you to write is what I had in mind.
In general having AST is always better than having a plain text file, unless you want to read it. But then you can easily dump AST back to text whenever you want.
Yeah, making AST will help you analyse your codebase programmatically which in turn will let you understand the codebase better and faster. This is some very basic programming knowledge, I think. Or is it not? Some comments here don't know what AST even is - is this the state of PL knowledge in the mainstream? Lisp and Smalltalk people would be very, very sad if it was so.