6

I am using javac from tools.jar (i.e. JavaCompiler) for parsing java files. I parse the sources using the implementation of TreePathScanner. So far, everything seems fine, as I can parse imports, package name, class name, method name, instructions...

But i do have problem with inline comments - I can not simply make them appear in the created AST tree, or visited them. I am able, however, to read javadoc comments for classes, methods etc, but no inline comments.

How to read inline comments in the best way? I am looking at the netbeans source code (as it is using javac for parsing, too), but i can not find easily anything about it.

My desperate solution would be using statements position of the source file, and then manually parse for comment everything whats between two statement. Or similar thing, but between two tree nodes.

Does anyone know better solution? Thank you!

Improve this question
2
  • I see that com.sun.tools.javac.main.JavaCompiler has some keepComments flag set to false by default. Is that what you are trying to use? Did you try changing it? Commented Feb 2, 2013 at 1:17
  • yes, i already had this set to true, didn't notice any difference. Commented Feb 2, 2013 at 8:37

2 Answers 2

Reset to default
4

You can't. The compiler throws them away. Compilers always do that. The Java compiler doesn't throw away Javadoc comments only because Javadoc uses the compiler to find them and the Javadoc guys got together with the compiler guys.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This is true. I've checked the source for javac and in class Scanner you can see the following method: protected void processComment(CommentStyle style) that just log debug message. I will try to see if I can override it.
I managed to run parser not using JavaCompiler, but just Scanner and Parser classes; and was able to override the processComment method, but... this method just gives the information that comment has been processed and comments type and nothing more (like value, position etc).
btw, it seems that eclipse AST compiler preserves comments.
1

A key difference between a "compiler parser" and a "reengineering parser" has to do with what information is captured about the layout, comments, and formats of literals. As others have observed, most compilers throw all this information away, as it isn't germane to compiling down to low-level code.

Similarly, classic parser generators (such as JavaCC, ANTLR, etc.) offer very little in support of capturring/regenerating this information.

Reengineering parsers, in contrast, are used to analyze code and comments, sometimes even to revise the code without losing (or appropriately revising comments). For code analysis with comments, you can't throw away the comments :-} For code modification, if you regenerate changed code based on original, it is nice if the changed code preserves code layout, comments, and literal "formats" (e.g., regnerating a hex literal as a decimal value is legal and equivalent, but makes the original authors pretty unhappy). To do this, reengineering parsers need special lexers to capture all this data, and parsing machinery that doesn't throw it away.

Our DMS Software Reengineering Toolkit includes, well, a reengineering parser as generic machinery; DMS-based parsers exist for a wide variety of languages (including OP's interest in Java). DMS captures all the comments/layout/formatting information. Analysis tools have access to it all.

TXL and Stratego provide some support for this, too.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.