The new SDF3-symbolsorts will flag unresolved references to sorts with an Unresolved reference to sort 'Foo' error. If that sort is used as an optional or repeated placeholder (<Foo?>, <Foo+> or <Foo*>) then the SDF3 file will compile fine. However, if that sort is used as a required placeholder (<Foo>), the compilation fails.

For example, this SDF3 file with an undefined reference to Foo:

module TestProject

context-free start-symbols
	Start

context-free syntax
	Start.Module = <<Foo>>

will throw these exceptions when compiled:

java.lang.RuntimeException: Analysis failed
Internal error evaluating strategy editor-save
	editor_save_0_0
	generate_all_0_0
	report_with_failure_0_1
	report_failure_0_2
	SRTS_EXT_fatal_err_0_2
org.spoofax.interpreter.core.InterpreterException: Exception during evaluation
Caused by: org.spoofax.interpreter.core.InterpreterErrorExit: Internal error: with clause failed unexpectedly in 'generate-all'
Caused by: org.strategoxt.lang.StrategoErrorExit: Internal error: with clause failed unexpectedly in 'generate-all'

Submitted by D. Pelsmaeker on 22 July 2014 at 15:28

I know what is happening here. Basically the unresolved reference is related with the signature generation, thus with the type system of SDF3. When generating from a simple template symbol (<Foo>), the type system lookup at the type of Foo, and since it is not defined, an exception is thrown.
On the other hand, when using a complex symbol (lists and optionals), the type of this complex symbol depends just on the name of the sort Foo, instead of its type (e.g. OptionType("Foo") for <Foo?>).
This is fixable adding the check whether a type lookup fails or not for a sort when defining the type of complex symbols of that sort.
I will add this check to the implementation.

I was wondering if this problem could be more generally solved if the AST for <Foo> could be treated the same as <Foo?> and <Foo*>. (Similar to how Foo* results in IterStar(Sort("Foo")), just Foo could for example result in Once(Sort("Import")) instead of Sort("Import").) So I took a look at the syntax and it struck me:

Symbol.Opt = <<Symbol>?>
Symbol.Iter = <<Symbol>+>
Symbol.IterStar = <<Symbol>*>
Symbol.IterSep = <{<Symbol> <sep:Separator>}+>
Symbol.IterStarSep = <{<Symbol> <sep:Separator>}*>

…the syntax is too general and allows me to write things that don’t make any sense to me:

X.Y1 = Foo?+*
X.Y2 = {Bar? ","}*
X.Y3 = {{FooBar ","}* ";"}*

However, SDF3, SDF2 and DEF happily accept some of this and cause a stack overflow in my Eclipse when trying to parse anything. So if I’m correct I would change the symbol syntax to disallow that, and at the same time allow Foo to be treated the same as Foo*.

Symbol.Once = <<Sort>>
Symbol.Opt = <<Sort>?>
Symbol.Iter = <<Sort>+>
Symbol.IterStar = <<Sort>*>

This is probably a huge change, I’m just throwing in an idea.

@Eduardo: Shouldn’t the more complex code generation patterns also ask for the type of the inner sort?

@Daniel: The stack overflow is probably caused by too many ambiguities (which cannot be avoided with these nested patterns).

@Guido: Yes and no. The type of <Foo> should be SortType("Foo") while the type of <Foo*> should be ListType("Foo") instead of ListType(SortType("Foo")). As I pointed above, the more complex symbols should do a lookup to check the type of the inner symbol, instead of just using the sort name (to avoid such problem).

But analysing the code, the type rules that are currently defined only handle the case of simple inner sorts. What should be the type of the symbols in his example:

X.Y2 = {Bar? ","}*
X.Y3 = {{FooBar ","}* ";"}*

I think there were some magic going on behind rtg2sig that would handle these cases, but how should we deal with them to generate the signatures?

(1) Exceptions are not an acceptable response to user errors. If a program cannot be compiled due to type errors then either the builder should be disabled, or it should respond with a message that explains why it cannot build, or it should work around the error and produce some target code, but still give a message that makes clear that something is wrong.

(2) Daniel’s proposal is not so farfetched. My guess is that most if not all SDF code would be parsable with this restriction. Perhaps a better way to achieve the restriction is by means of constraints that forbid or give warnings for such patterns (just like you are doing for WebDSL, Daniel).

The stackoverflows are indeed caused by cyclic ambiguities. SGLR should actually catch these and produce a cyclic parse forest. It may be the case that the non-termination is caused by the mapping from parse forest to abstract syntax tree.

In practice, we never want to write these patterns. So if we can warn the SDF writer when such patterns occur that would be useful. It is easy enough to make a mistake like that, and it is very annoying if Eclipse then hangs on you.

The restriction that Daniël proposes is useful, but not sufficient. A problematic production such as A = C** can also be written as

A = B*
B = C*

And it is actually more likely to occur in this form.