I’ve noticed the following issue in Spoofax, but I think it is a bug in stratego:

When an optional literal is present in a grammar, there is a mismatch between the number of arguments of the corresponding constructor Stratego thinks there are at compile/editing time, and the number of arguments this constructor actually has at runtime.

How to reproduce (in Spoofax):

• open the tutorial com.pany.entities project
• open test/example.ent
• notice how there is one info message ‘This is just an example program in the “entities” language…’
• change EntityLang.sdf like this (add the question mark): "module"? ID Definition* -> Start
• build
• notice how the info message in example.ent disappeared (this implies Module(x, _) (check.str) does not match anymore)
• notice how both the stratego editor and the stratego compiler still consider Module/2 existing (changing to Module/3 results in a compile error), while at runtime only Module/3 is present (with the first argument always equal to None, this can be seen by inspecting the AST)

Clearly either Stratego is wrong at runtime, or it is wrong at compile/editing time.

Submitted by Tobi Vollebregt on 1 November 2010 at 17:30

After reconsidering it is probably a jsglr issue.

I have tried to confirm this by performing the following test.

I have performed the steps as described above in Spoofax, then taken the AST of the following EntityLang program:

empty.ent:

module Empty

empty.aterm:

Module(None(), "Empty", [])

Then I have created a parse table for the modified EntityLang syntax using the C toolchain:

$ pack-sdf -i syntax/EntityLang.sdf | sdf2table -o EntityLang.tbl -m EntityLang
$ sglri -p EntityLang.tbl -s Start < test/empty.ent | pp-aterm

Which resulted in the following AST (note the difference!):

Module(Some("module"), "Empty", [])

More closely related to the original issue, I now also inspected the RTG generated by the C toolchain. This appears to be identical to the RTG generated by the Java toolchain, although sdf2rtg gives an error about a missing con, and refuses to generate the RTG unless --ignore-missing-cons is given:

$ pack-sdf -i syntax/EntityLang.sdf | sdf2rtg -m EntityLang -o EntityLang.rtg --ignore-missing-cons
[ sdf2rtg | error ] No constructor name specified in production:
[ sdf2rtg | error ]   "module" -> <"module"?-CF>
[ sdf2rtg | error ] Resolution: please add a cons attribute to this production.
[ sdf2rtg | error ] Production in abstract syntax: 
        prod([lit("module")],cf(opt(lit("module"))),attrs([]))

regular tree grammar
  start Start
  productions
    ...
    Start -> Module(ID,ListStarOfDefinition0)
    ...

Which does not match with the actual results of a parse..

In conclusion, there are actually 2 issues:

• jsglr differs from sglr in that it always puts a None() term at the place of an optional literal in the grammar, instead of putting e.g. Some(“literal”) there when the literal is present in the input.

• Module/3 is parsed at runtime both by jsglr and sglr, but both sdf2rtg and it’s Java equivalent put Module/2 in the regular tree grammar.

Hey Tobi. It actually looks like we may reimplement the signature generation (sdf2rtg) somewhere in the near future in Spoofax. Note that we generally recommend to not use optional literals (and other optionals), but to write something like this:

“module” ID Definition* -> Start {cons(“Module”)}
ID Definition* -> Start {cons(“ModuleNoKW”)}

giving you a bit more control over the AST it generates. And it appears that sdf2rtg actually doesn’t handle this corner case with optional literals…

Split off the jsglr-sglr inconstency (first bullet point) to: https://yellowgrass.org/issue/Spoofax/295

I found the sdf2rtg issue is related to https://yellowgrass.org/issue/StrategoXT/691 and https://yellowgrass.org/issue/StrategoXT/546