Accidentally I deleted a line of my grammar file and started the build. Then Spoofax worked with full power and swapped out more and more until about 3.5 GB, then eclipse freezed. The process was not stoppable. (Unfortunately I cannot restore that grammar anymore)

Error Log:
An internal error occurred during: “myxy ParserScheduler for example.def”.

Exception Stack trace

java.lang.OutOfMemoryError: Java heap space

at java.util.AbstractList.iterator(Unknown Source)

at org.spoofax.jsglr.Disambiguator.filterTree(Disambiguator.java:393)

at org.spoofax.jsglr.Disambiguator.filterTree(Disambiguator.java:362)

at org.spoofax.jsglr.Disambiguator.filterAmbiguities(Disambiguator.java:704)

at org.spoofax.jsglr.Disambiguator.filterTree(Disambiguator.java:356)

at org.spoofax.jsglr.Disambiguator.filterAmbiguities(Disambiguator.java:704)

at org.spoofax.jsglr.Disambiguator.filterTree(Disambiguator.java:356)

… then followed a lot of repetitions of the two lines:

at org.spoofax.jsglr.Disambiguator.filterAmbiguities(Disambiguator.java:704)

at org.spoofax.jsglr.Disambiguator.filterTree(Disambiguator.java:356)

(unstable version 0.5.2.91 of Spoofax on XP)

20:50 - I added some new infos (attachement and see my comment from Aug 12 below)

Submitted by Otto Hausbacher on 12 August 2010 at 13:24

As of revision 21162 (to be included in 0.5.3), the Disambiguator class no longer allocates iterator instances when recursively walking the parse tree, which should help with this problem. Without the grammar that caused the problem, it’s hard to tell whether or not that actually addresses this issue though. The error might also indicate an infinite loop in the Disambiguator.

Now I can proudly present You another grammar (don’t laugh) with a similar effect. I pushed all the potentially responsible parts into one file (Myxy.sdf) and left all the other parts away. I used Spoofax/IMP Version 0.5.2.93, created a new workspace with a new created Spoofax project Myxy. The generated file syntax/Myxy.sdf I edited according to the attached grammar file. I also changed the generated test file test/example.xyz according to the attached test file. Then I built the project and everything seemed to be unproblematic. I closed the test file and re-opened it explicitly with the IMP editor. Now eclipse worked with full power and the swapping memory became bigger and bigger. But this time, I was able to kill the eclipse process before eclipse freezed.

If I use line no 49 (48 commented out) the problem occurs.

If I instead use line no 48 (49 commented out) there is no problem.

47 %% !!!!!!!!!!!!!!!!!!!!!!!!!!!!! ->
48 %% LPID -> PId{cons(“PId”)} %% O.K.
49 [A-Za-z_$][A-Za-z0-9_$]* -> PId{cons(“PId”)} %% swap until freeze or kill process eclipse
50 %% !!!!!!!!!!!!!!!!!!!!!!!!!!!!! <-

The resulting .log file I copied and renamed it to ErrorLog.log and attached it.

I see. Using a character class iteration like [A-Za-z0-9]* in context-free syntax seems to be what triggered the error, which is probably not a good idea anyway. Unlike lexical syntax, ontext-free syntax can be interleaved by layout, so the pattern [A-Za-z0-9]* would also parse a string like foo bar baz as a single identifier. In the parse tree representation, it would be stored as a letter ‘f’, followed by optional layout, followed by ‘o’, followed by optional layout, etc. This leads to very big parse trees, and combined with the risk of ambiguitities, that must have triggered the problem. I added a warning for patterns like these in r21167. Together with the performance fix that should address this issue for the next version.