xtext 处理左递归语法答案

【问题标题】：xtext dealing with left recursion grammarxtext 处理左递归语法
【发布时间】：2013-08-03 03:07:36
【问题描述】：

我正在使用 xtext 2.4 并希望同时支持 map 和 set，我的语法看起来像这样

<term> ::- <collection>
<collection> ::- <map> | <set>
<map> ::- '{' (<term> ':' <term> (',' <term> ':' <term> )*)? '}'
<set> ::- '{' (<term> (',' <term>)* )+ '}'

所以可以简化为

<term> ::- '{' (<term> ':' <term> (',' <term> ':' <term> )*)? '}' |
           '{' (<term> (',' <term>)* )+ '}'

antlr 抱怨这个语法开始左递归，我不知道为什么，因为它在右手边有'{'。

即使我想排除相同的部分，我也不知道如何在 xtext 中执行此操作，因为 EObject 现在被拆分为两个生产规则，并且解析后的数据无法存储在同一个对象中。

有什么帮助吗？

================================================ ============================== 我写的对应的xtext语法是

grammar org.xtext.problem.Term with org.eclipse.xtext.common.Terminals

generate term "http://www.xtext.org/problem/Term"

Term:
    term = Collection
;

Collection:
    MyMap | MySet
; 

MyMap:
    {MyMap} '{'( keys+= Term ':' values += Term ( ',' keys+=Term ':' values +=Term)*  )?'}'
;

MySet:
    {MySet} '{'( values += Term ( ',' values +=Term)*  )?'}'
;

错误消息（以 error(211) 开头）：

0    [main] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Registering platform uri '/Users/jcwu/opensource/xtext-problemistic'
116  [main] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Adding generated EPackage 'org.eclipse.xtext.xbase.XbasePackage'
454  [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.eclipse.org/Xtext/Xbase/XAnnotations' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
459  [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.eclipse.org/xtext/xbase/Xtype' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
479  [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.eclipse.org/xtext/xbase/Xbase' from 'platform:/resource/org.eclipse.xtext.xbase/model/Xbase.genmodel'
479  [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.eclipse.org/xtext/common/JavaVMTypes' from 'platform:/resource/org.eclipse.xtext.common.types/model/JavaVMTypes.genmodel'
1660 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/jcwu/opensource/xtext-problemistic/org.xtext.problem.term/../org.xtext.problem.term/src-gen
1665 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/jcwu/opensource/xtext-problemistic/org.xtext.problem.term/../org.xtext.problem.term.ui/src-gen
1666 [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/jcwu/opensource/xtext-problemistic/org.xtext.problem.term/../org.xtext.problem.term.tests/src-gen
2033 [main] INFO  ipse.xtext.generator.LanguageConfig  - generating infrastructure for org.xtext.problem.Term with fragments : ImplicitRuntimeFragment, ImplicitUiFragment, GrammarAccessFragment, EcoreGeneratorFragment, SerializerFragment, ResourceFactoryFragment, XtextAntlrGeneratorFragment, ValidatorFragment, ImportNamespacesScopingFragment, QualifiedNamesFragment, BuilderIntegrationFragment, GeneratorFragment, FormatterFragment, LabelProviderFragment, OutlineTreeProviderFragment, QuickOutlineFragment, QuickfixProviderFragment, ContentAssistFragment, XtextAntlrUiGeneratorFragment, Junit4Fragment, RefactorElementNameFragment, TypesGeneratorFragment, XbaseGeneratorFragment, CodetemplatesGeneratorFragment, CompareFragment
4115 [main] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://www.xtext.org/problem/Term' from 'platform:/resource/org.xtext.problem.term/src-gen/org/xtext/problem/Term.genmodel'
error(211): ../org.xtext.problem.term/src-gen/org/xtext/problem/parser/antlr/internal/InternalTerm.g:119:1: [fatal] rule ruleCollection has non-LL(*) decision due to recursive rule invocations reachable from alts 1,2.  Resolve by left-factoring or using syntactic predicates or using backtrack=true option.
5918 [main] ERROR enerator.CompositeGeneratorFragment  - java.io.FileNotFoundException: ../org.xtext.problem.term/src-gen/org/xtext/problem/parser/antlr/internal/InternalTermLexer.java (No such file or directory)
org.eclipse.emf.common.util.WrappedException: java.io.FileNotFoundException: ../org.xtext.problem.term/src-gen/org/xtext/problem/parser/antlr/internal/InternalTermLexer.java (No such file or directory)
    at org.eclipse.xtext.util.Files.readFileIntoString(Files.java:129)
    at org.eclipse.xtext.generator.parser.antlr.AbstractAntlrGeneratorFragment.suppressWarningsImpl(AbstractAntlrGeneratorFragment.java:132)
    at org.eclipse.xtext.generator.parser.antlr.AbstractAntlrGeneratorFragment.suppressWarnings(AbstractAntlrGeneratorFragment.java:142)
    at org.eclipse.xtext.generator.parser.antlr.AbstractAntlrGeneratorFragment.suppressWarnings(AbstractAntlrGeneratorFragment.java:138)
    at org.eclipse.xtext.generator.parser.antlr.XtextAntlrGeneratorFragment.generate(XtextAntlrGeneratorFragment.java:49)
    at org.eclipse.xtext.generator.CompositeGeneratorFragment.generate(CompositeGeneratorFragment.java:92)
    at org.eclipse.xtext.generator.LanguageConfig.generate(LanguageConfig.java:113)
    at org.eclipse.xtext.generator.Generator.generate(Generator.java:361)
    at org.eclipse.xtext.generator.Generator.invokeInternal(Generator.java:128)
    at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:126)
    at org.eclipse.emf.mwe.core.lib.Mwe2Bridge.invoke(Mwe2Bridge.java:34)
    at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:201)
    at org.eclipse.emf.mwe2.runtime.workflow.AbstractCompositeWorkflowComponent.invoke(AbstractCompositeWorkflowComponent.java:35)
    at org.eclipse.emf.mwe2.runtime.workflow.Workflow.run(Workflow.java:19)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:102)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:62)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:52)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:74)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:35)
Caused by: java.io.FileNotFoundException: ../org.xtext.problem.term/src-gen/org/xtext/problem/parser/antlr/internal/InternalTermLexer.java (No such file or directory)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at java.io.FileInputStream.<init>(FileInputStream.java:97)
    at org.eclipse.xtext.util.Files.readFileIntoString(Files.java:126)
    ... 18 more
error(211): ../org.xtext.problem.term.ui/src-gen/org/xtext/problem/ui/contentassist/antlr/internal/InternalTerm.g:176:1: [fatal] rule rule__Collection__Alternatives has non-LL(*) decision due to recursive rule invocations reachable from alts 1,2.  Resolve by left-factoring or using syntactic predicates or using backtrack=true option.
6617 [main] ERROR enerator.CompositeGeneratorFragment  - java.io.FileNotFoundException: ../org.xtext.problem.term.ui/src-gen/org/xtext/problem/ui/contentassist/antlr/internal/InternalTermLexer.java (No such file or directory)
org.eclipse.emf.common.util.WrappedException: java.io.FileNotFoundException: ../org.xtext.problem.term.ui/src-gen/org/xtext/problem/ui/contentassist/antlr/internal/InternalTermLexer.java (No such file or directory)
    at org.eclipse.xtext.util.Files.readFileIntoString(Files.java:129)
    at org.eclipse.xtext.generator.parser.antlr.AbstractAntlrGeneratorFragment.suppressWarningsImpl(AbstractAntlrGeneratorFragment.java:132)
    at org.eclipse.xtext.generator.parser.antlr.AbstractAntlrGeneratorFragment.suppressWarnings(AbstractAntlrGeneratorFragment.java:142)
    at org.eclipse.xtext.generator.parser.antlr.AbstractAntlrGeneratorFragment.suppressWarnings(AbstractAntlrGeneratorFragment.java:138)
    at org.eclipse.xtext.generator.parser.antlr.XtextAntlrUiGeneratorFragment.generate(XtextAntlrUiGeneratorFragment.java:53)
    at org.eclipse.xtext.generator.CompositeGeneratorFragment.generate(CompositeGeneratorFragment.java:92)
    at org.eclipse.xtext.generator.LanguageConfig.generate(LanguageConfig.java:113)
    at org.eclipse.xtext.generator.Generator.generate(Generator.java:361)
    at org.eclipse.xtext.generator.Generator.invokeInternal(Generator.java:128)
    at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:126)
    at org.eclipse.emf.mwe.core.lib.Mwe2Bridge.invoke(Mwe2Bridge.java:34)
    at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:201)
    at org.eclipse.emf.mwe2.runtime.workflow.AbstractCompositeWorkflowComponent.invoke(AbstractCompositeWorkflowComponent.java:35)
    at org.eclipse.emf.mwe2.runtime.workflow.Workflow.run(Workflow.java:19)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:102)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:62)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:52)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:74)
    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:35)
Caused by: java.io.FileNotFoundException: ../org.xtext.problem.term.ui/src-gen/org/xtext/problem/ui/contentassist/antlr/internal/InternalTermLexer.java (No such file or directory)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at java.io.FileInputStream.<init>(FileInputStream.java:97)
    at org.eclipse.xtext.util.Files.readFileIntoString(Files.java:126)
    ... 18 more
6650 [main] INFO  text.generator.junit.Junit4Fragment  - generating Junit4 Test support classes
6682 [main] INFO  text.generator.junit.Junit4Fragment  - generating Compare Framework infrastructure
7118 [main] INFO  .emf.mwe2.runtime.workflow.Workflow  - Done.

【问题讨论】：

我不明白为什么要使用 Xtext 时要编写 ANTLR 语法。 Xtext 语法是完全不同的，并且对于您可以将什么放入一个规则和什么不能放入其中也有不同的要求。
@A.H. Xtext 使用 ANTLR 来解析它自己的 DSL。 Ray Wu 可能在他的控制台/日志上看到了 ANTLR 风格的错误消息。
RayWu，左递归规则可能起源于其他地方（您发布的确实不是左递归）。您能否发布一个演示该问题的小型 Xtext 语法？
@BartKiers：我知道内部使用了 ANTLR。但这与 Xtext 用户和sscce.org 示例无关：-)
@A.H.，啊，好吧。我对 Xtext 知之甚少，但认为更完整的 Xtext 语法/文件会有所帮助...

标签： eclipse parsing compiler-construction xtext

【解决方案1】：

适用于您的问题的 Xtext 语法是这样的：

Model: term += Term*;

Term: c=Collection;

Collection: ( => Map | Set );

Map: '{' {Map} ( entries += MapEntry ( ',' entries += MapEntry )* )? '}';
MapEntry: key=Term ':' value=Term;

Set: '{' ( values += Term ( ',' values += Term )* )+ '}';

注意事项：

Collection 规则中的 => 语法谓词。这将 Xtext/ANTLR 引导到正确的方向。 (Docu here)
{Map}“简单动作”(Docu here) 会创建一个Map，即使内容为空。
额外的MapEntry 规则也是必需的，否则您将无法保存键/值对。
您的语法允许Set 中的{}{}{} 和{}, {}, {}。这可能是也可能不是您想要的。

所以在我看来，关于 LL 递归的 Xtext/ANTLR 错误消息在这种情况下并不是最合适的。这不是关于 LL 递归，而是关于语法中的歧义，有时可以通过应用 => 来解决。有关详细信息，请参阅链接的文档。

仅供参考：语法可以解析像这样的简单嵌套的东西：

// Maps
{}
{ {} : {} }
{ {} : {}, {} : {}}

// Sets
{ {} }
{ {} {} {} {} }
{ {}, {}, {}, {} }

// nested / mixed
{ { { {}:{} } } : {}, {} : { {}:{} }}
{ { { {}:{} } } : {}, {} : { {}:{ {}{}{} } }}

【讨论】：

{}{}{} 问题由 term += Term* 引入，但我确实在其他规则中处理。我认为句法谓词是我的问题的关键。让我深入研究一下。（接受为答案）
能否更新您的文档链接？
@FrontMobe 完成