Found this because in my (old) Lua plugin I have:

start-symbols
Start

syntax
<FIRSTLINECOMMENT?-LEX>
<LAYOUT?-CF>

<LAYOUT?-CF> -> Start {cons(“File”)}

To allow a comment starting with # at the begin of the file.

But then this test fails:

test first line comment
#!/usr/bin/lua
parse succeeds

It succeeds if I put a <LAYOUT?-CF> before <FIRSTLINECOMMENT?-LEX>.

Submitted by Tobi Vollebregt on 17 June 2011 at 21:42

Sverre also encountered this problem with another tricky-whitespace language. Changing this behavior basically requires two things: FragmentParser.createTestFragmentString() should be called with a false argument to disable the padding, and Retokenizer.copyTokensFromFragment() should be changed so that it adjusts all offsets and line and column numbers of all the tokens that are in the tested fragment. The latter requires adding some new setters to the IToken interface, but may not be all that hard…

This is also a problem with minimal projects which do not specify LAYOUT at all. In the minimal MiniJava project, we want to provide the students only with a simple context-free rule, e.g.

"42" -> Start

and a corresponding simple test

test example [[42]] parse succeeds