I want to know if JSGLRI can be used in the same way like command line sglri tool. I would like to use JSGLRI library to load a *.tbl file and an input string and then to generate an ATerm.

Is this possible?
Is there any Javadoc for JSGLRI ?

Which jars must be included in a separate Java project in order to use JSGLRI?

Basically, I want to use Java to modify my AST and I don’t know where to start.

Submitted by Radu Mereuta on 6 May 2011 at 13:36

Hi Radu. We need to update the documentation on this subject. There’s a start at http://strategoxt.org/Stratego/JSGLR. But if you want to parse from the command-line, what you basically need to do is:

java -ss8m -mx512m -client -cp strategoxt.jar org.spoofax.jsglr.Main –implode -p TABLE -i INPUT

(The easiest is to just use the parser from strategoxt.jar.)

I’ll leave this issue open for now as an indication that we need to update the documentation.

Are there any options for strategoxt.jar to add line numbers as annotations in the output tree? I want to do something like:

java -ss8m -mx512m -client -cp strategoxt.jar org.spoofax.jsglr.Main –implode -p TABLE -i SOURCE –preserve-locations

In the end, I would like to have an ATerm which contains line numbers as annotations. Is this possible?

Right now, no, there is no --preserve-locations for JSGLR, since that’s a Stratego feature. You can however get position annotations within Stratego by doing something like the following:

<parse-file-pt(|<import-term(include/YourParseTable.tbl)>)> filename;
asfix-anno-locations;
implode-asfix

(Also see the parsing strategies in lib/editor-common.generated.str for some examples of how to use the parsing API.)

Could you please explain in a little bit more detail how to use this?
I’m trying to generate ATerms annotated with location information (line numbers).
So I tried to create a rule in my_project.str which is called via the transform-menu (specified by MyProject-Builders.esv). But based on the examples of rules in my_project.str and strategies in editor-common.generated.str, I could not figure out a way to do that.

First of all, I have to create a rule, not a strategy, right?
I tried to do something like that:

generate-annotated-aterm:
(selected, position, ast, path, project-path) -> (filename, result)
with
  filename := <guarantee-extension(|"aaterm")> path;
  result := <parse-file(strsglr-perror, strsglr-report-parse-error
			| <import-term(include/MyProject.tbl)>)> path;
  asfix-anno-locations;
  implode-asfix 

But asfix-anno-locations is not available. Which library do I need for that? (In the Stratego documentation, I only found stuff about the command-line tool.
Also, the parse-file call does not work that way, I get the message that the program wants to open a file with the name being the content of the file I’m actually trying to parse (from which I called the rule via the transformation menu).

For the sake of completeness, the line in my MyProject-Builders.esv looks like this:

builder  : "Generate annotated AST" = generate-annotated-aterm  (openeditor) (realtime) (meta) (source)

It’d be great to get some help here! I’m kind of lost right now…

I’m not sure if this is helpful to you, but you could use the origin-location strategy in lib/editor-common.generated.str to add location annotations to terms. Something like below:

generate-annotated-aterm:
(selected, position, ast, path, project-path) -> (filename, result)
with
filename := <guarantee-extension(|“aterm”)> path;
result := <topdown(anno-location)> ast

anno-location: trm -> trm{}

Yes, this is very helpful!!
Thanks a lot and keep up the great work! (and the great support :-) )

Hope this doesn’t go offtopic, but I have a follow-up question:

Is it possible to do something like what you described above from a Java environment?
What I want to do is the following:
1. Parse a file given a parse table (via an instance of SGLR)
2. Apply some rules/strategies to the parsed syntax tree
3. Do stuff with the result -> not important at the moment

Generally, these things work very well, but I can’t call the rule as described above via Java.

My program looks like this:

ParseTableManager ptm = new ParseTableManager();
PureFactory fac = new PureFactory();
SGLR sglr = new SGLR(new TreeBuilder(),ptm.loadFromFile(parseTable));
Object t = sglr.parse(new FileReader(inputFile));
Context context = Main.init();
IStrategoTerm t = Main.constantFactory.parseFromString(t.toString());
System.out.println(t.toString());
IStrategoTerm t2 = topdown_simplify_annotated_aterm_0_0.instance.invoke(context, t);
// IStrategoTerm t2 = topdown_anno_location_0_0.instance.invoke(context, t);
System.out.println(t2.toString());  

with

topdown-anno-location: x -> <topdown(anno-location)> x

The reason is obvious: I lose the position information by parsing via sglr.parse().

Is there a way to keep this info and make annotations similar to those generated by origin-location?

The background is that I want to create a tool (written in Java) analyzing source code with the use of Island Grammars. Therefore I want to use Spoofax/JSGLR functionality within a Java project.

Hi Johannes,

You could try and compile your stratego files into Java classes. I’m using something like this:
java -jar strategoxt.jar -i xmlify.str -o K2Str -la stratego-sglr –lib -I .. -p k2parser.lib –clean
(you can use the strategoxt.jar –help to see more information about options)

What I do in my stratego files, is to generate XML from my AST (ATerm in SDF). Because I use stratego directly, I have access to the location information.

After that I compile the stratego files into java, I just call the main strategy something like this: context.invokeStrategyCLI(java$Parse$String_0_0.instance, “a.exe”, kDefinition);

Maybe this will help you. Maybe my code will help you, so here a few links:
Stratego file:
http://code.google.com/p/k-framework/source/browse/branches/k3/javasources/K3Syntax/trans/xmlify.str

How I call it in Java:
http://code.google.com/p/k-framework/source/browse/branches/k3/javasources/K3Java/src/k2parser/KParser.java

If you have any more questions, don’t hesitate to ask.

Good luck,
Radu Mereuta

Hi Radu,

thank you so much for your answer. It helped a lot!

In fact, the solution was there all along, but until I read your answer (and parts of your code), I didn’t recognise it.
I already had all the rules available via Java, just had to create one that parses a string AND annotates the locations. So I can’t use JSGLR to do the parsing but call the stratego rule (which internally calls JSGLR of course). Your javaParseString rule gave me everything I needed.

The approach even got a little bit simpler as Spoofax already does the compiling for me.
The ant target “stratego.jar” produces a jar with all the compiled Stratego stuff in it. So I just reference this jar in my Java project and I can directly use all that I need.

My rule now looks like this:

parse-and-annotate-string = ?[a, b] ; 
    c := <parse-string(|<import-term(include/CobolIslandParser.tbl)>)> b;
    d := <topdown(anno-location)> c

This returns the parsed and annotated syntax tree.
In Java, I do something like this:

String inputString = readFileToString(inputFile);
Context context = Main.init();
IStrategoTerm a = context.invokeStrategyCLI(parse_and_annotate_string_0_0.instance, inputFile, inputString);  

Really simple. Only minor issue: I haven’t managed to skip the first argument (a in Stratego, inputFile in Java). I don’t use it at all but if I remove it, I get a “clause failed unexpectedly” error.

Stratego:

parse-and-annotate-string2 = ?[b] ; 
    c := <parse-string(|<import-term(include/CobolIslandParser.tbl)>)> b;
    d := <topdown(anno-location)> c

Java:

IStrategoTerm b = context.invokeStrategyCLI(parse_and_annotate_string2_0_0.instance, inputString);  

But that’s just a cosmetic thing, so not important at all.
Anyhow, at the moment everything is working fine! Thanks again, Radu!!

Best regards,
Johannes

I used to call that strategy from command line, so ‘a’ would take the name of the executable.

I tried to remove it, and it worked fine for me.

I’m not sure what went wrong for you, but make sure that you remove every generated file in every location that you are using them and then regenerate them.

Also you could use ‘debug(!"Hello: ")’ each time you are not sure what you have in that place.
For example: parse-and-annotate-string2 = debug(!"Init: ") ; ?[b] ; …

Alright, thanks again, Radu!
Since this is only a minor issue, I haven’t tried to fix it yet (more important stuff to do ;-) ).

However, a more serious problem came up: Strings are escaped at some point in Stratego and I didn’t find a way to avoid (revert) that. I found a workaround for my specific problem, but if anyone is interested, please go ahead and read the rest of this awfully long post: :-)

I am trying to do some kind of multi-stage parsing, i.e. I first parse a source file in order to get an abstract syntax tree and then I want to find patterns within this AST by use of another grammar. Unfortunately, the Java calls for that do not seem to work:

Context context = package1.Main.init();
AST = context.invokeStrategyCLI(parse_annotate_simplify_0_0.instance, fileName, originalCode);
System.out.println("AST: "+AST.toString());
Context context2 = package2.Main.init();
IStrategoTerm AST2 = context2.invokeStrategyCLI(parse_move_0_0.instance, fileName, AST.toString());
System.out.println("AST2: " +AST2);

The Stratego rule:

parse-move = ?[a, b] ;
    debug(!b);
    c := <parse-string(|<import-term(include/Parser2.tbl)>)> b

Problem now is, I get a parse error:
error(“Parse error”,[localized(“Syntax error near unexpected character ‘"’”,area-in-file(“string”,area(1,48,1,48,48,0)))])

Apparently, at some point, my quotation marks are escaped and this messes up the parsing.

The beginning of the program looks like this:
Program(ProcedureDivision(ProcDivHeader([“USING”,“BLABLABLA”]){(13,7,13,41),Cat(“other”)},ProcDivBody([“iwadawdeese”,“MOVAE”,“WASD”,“TAO”,“ZUG.”],[MoveStatement(“Awd”,[“AWD”]){(18,7,18,21),Cat(“data”)}]

The character at 48 is the second (!) " (after USING).
If I change the rule to just return the input,

parse-move = ?[a, b] ;
    c := b

the output from Java contains quotation marks at the beginning and end; all the inner quotation marks are escaped. Surprisingly, the line debug(!b) in Stratego prints the string without escaped quotation marks. Applying the strategy unescape to the input doesn’t work either, because at this point, the string is not escaped…

While writing this post, I found a workaround for the problem; I just remove all the quotation marks before making the second Stratego call. Not beautiful, but it works now.
But still, I think it’s an interesting question where this escaping happens and how it might be possible to avoid it.

Yeah.. about that… I’m not sure what is the proper way of unescaping, so I created a function of my own:

public static String unescape(String str) {
                str = str.replace("\\n", "\n");
                str = str.replace("\\t", "\t");
                str = str.replace("\\r", "\r");
                return str;
        }

Alright, thank you for that!