【发布时间】:2016-07-24 12:30:46
【问题描述】:
我的java应用程序要解析很多java源代码,我正在使用这个解析器https://github.com/javaparser/javaparser。
这是我调用另一个解析单个文件的函数的函数:
private static void createSimplifiedAST(String userFolder, String userProjectFolder, ArrayList userProjectFiles){
File projectDir = new File(usersFolderPath + userFolder + "/" + userProjectFolder);
// explore all java files inside a directory (user's project folder)
new DirExplorer((level, path, file) -> path.endsWith(".java"), (level, path, file) -> {
System.out.println(path);
System.out.println(Strings.repeat("=", path.length()));
// create root of simplified AST of user's single file
TreeNode<String> projectSingleFile = new ArrayMultiTreeNode<String>("projectFileTreeRoot");
// create simplified AST of user's single file
declarationNumber = 0;
forNumber = 0;
forEachNumber = 0;
whileNumber = 0;
try {
iterateASTNodes(file, projectSingleFile);
} catch (ParseException ex) {
Logger.getLogger(JavaSourceCodeParser.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(JavaSourceCodeParser.class.getName()).log(Level.SEVERE, null, ex);
}
// prints single file tree
System.out.println(projectSingleFile);
// save simplified ASTs to txts
saveASTToTxt(userFolder, userProjectFolder, file, projectSingleFile);
// add simplified AST of single file to user's project files array list
userProjectFiles.add(projectSingleFile);
}).explore(projectDir);
}
这是解析单个文件的函数:
private static void iterateASTNodes(File file, TreeNode<String> treeNode) throws ParseException, IOException{
try {
new NodeIterator(new NodeIterator.NodeHandler() {
@Override
public boolean handle(Node node) {
if (node instanceof VariableDeclarator) {
treeNode.add(new ArrayMultiTreeNode<>("declaration" + declarationNumber));
declarationNumber++;
return false;
} else if (node instanceof ForStmt) {
treeNode.add(new ArrayMultiTreeNode<>("for" + forNumber));
forNumber++;
iterateASTNodesWrapper(node, treeNode.find("for" + (forNumber - 1)));
return false;
} else if (node instanceof ForeachStmt) {
treeNode.add(new ArrayMultiTreeNode<>("foreach" + forEachNumber));
forEachNumber++;
iterateASTNodesWrapper(node, treeNode.find("foreach" + (forEachNumber - 1)));
return false;
} else if (node instanceof WhileStmt) {
treeNode.add(new ArrayMultiTreeNode<>("while" + whileNumber));
whileNumber++;
iterateASTNodesWrapper(node, treeNode.find("while" + (whileNumber - 1)));
return false;
} else {
return true;
}
}
}).explore(JavaParser.parse(file));
} catch (ParseException | IOException ex) {
System.err.println(" ERROR: lexical error!");
}
}
这是 NodeIterator 类:
public class NodeIterator {
public interface NodeHandler {
boolean handle(Node node);
}
private NodeHandler nodeHandler;
public NodeIterator(NodeHandler nodeHandler) {
this.nodeHandler = nodeHandler;
}
public void explore(Node node) {
if (nodeHandler.handle(node)) {
for (Node child : node.getChildrenNodes()) {
explore(child);
}
}
}
}
还有 DirExplorer 类:
public class DirExplorer {
public interface FileHandler {
void handle(int level, String path, File file);
}
public interface Filter {
boolean interested(int level, String path, File file);
}
private FileHandler fileHandler;
private Filter filter;
public DirExplorer(Filter filter, FileHandler fileHandler) {
this.filter = filter;
this.fileHandler = fileHandler;
}
public void explore(File root) {
explore(0, "", root);
}
private void explore(int level, String path, File file) {
try {
if (file.isDirectory()) {
for (File child : file.listFiles()) {
explore(level + 1, path + "/" + child.getName(), child);
}
} else {
if (filter.interested(level, path, file)) {
fileHandler.handle(level, path, file);
}
}
} catch (Exception e) {
System.out.println(" ERROR: something went wrong!\n");
}
}
}
当我运行程序时,有时它会在解析此文件时停止:
public class Test {
/**<caret>
public void foo() {
}
}
我知道它会因为词法错误而停止,这是:
Exception in thread "main" com.github.javaparser.TokenMgrError: Lexical error at line 6, column 2. Encountered: <EOF> after : ""
at com.github.javaparser.ASTParserTokenManager.getNextToken(ASTParserTokenManager.java:2480)
at com.github.javaparser.ASTParser.jj_ntk(ASTParser.java:9154)
at com.github.javaparser.ASTParser.ClassOrInterfaceBody(ASTParser.java:829)
at com.github.javaparser.ASTParser.ClassOrInterfaceDeclaration(ASTParser.java:470)
at com.github.javaparser.ASTParser.TypeDeclaration(ASTParser.java:398)
at com.github.javaparser.ASTParser.CompilationUnit(ASTParser.java:203)
at com.github.javaparser.JavaParser.parse(JavaParser.java:111)
at com.github.javaparser.JavaParser.parse(JavaParser.java:158)
at com.github.javaparser.JavaParser.parse(JavaParser.java:177)
at javasourcecodeparser.JavaSourceCodeParser.iterateASTNodes(JavaSourceCodeParser.java:89)
at javasourcecodeparser.JavaSourceCodeParser.lambda$createSimplifiedAST$1(JavaSourceCodeParser.java:178)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:35)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:31)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:31)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:31)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:31)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:31)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:31)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:31)
at javasourcecodeparser.DirExplorer.explore(DirExplorer.java:24)
at javasourcecodeparser.JavaSourceCodeParser.createSimplifiedAST(JavaSourceCodeParser.java:194)
at javasourcecodeparser.JavaSourceCodeParser.main(JavaSourceCodeParser.java:225)
现在,这个错误停止了整个程序,我无法解析其他“好”的代码。
我想要的是程序必须跳过有词法错误的文件并继续解析其他文件。
有什么想法吗?可能会尝试编译每个文件并仅在编译成功时对其进行解析?
【问题讨论】:
-
要求我们推荐或查找书籍、工具、软件库、教程或其他场外资源的问题不属于 Stack溢出,因为它们往往会吸引固执己见的答案和垃圾邮件。相反,请描述问题以及迄今为止为解决该问题所做的工作。
标签: java parsing compilation compiler-errors