SDF3 analysis and all generation is broken. The on-save handler is not called and manually executing builders results in errors like:

org.spoofax.interpreter.core.UndefinedStrategyException: Definition 'generate-expanded' not found
at org.spoofax.interpreter.core.Interpreter.invoke(Interpreter.java:79)
at org.strategoxt.HybridInterpreter.invoke(HybridInterpreter.java:424)
at org.strategoxt.imp.debug.core.str.launching.DebuggableHybridInterpreter.invoke(DebuggableHybridInterpreter.java:150)
at org.strategoxt.imp.runtime.Environment$2.invoke(Environment.java:195)
at org.strategoxt.imp.runtime.services.StrategoObserver.invoke(StrategoObserver.java:704)
at org.strategoxt.imp.runtime.services.StrategoObserver.invoke(StrategoObserver.java:673)
at org.strategoxt.imp.runtime.services.menus.model.StrategoBuilder.invokeObserver(StrategoBuilder.java:334)
at org.strategoxt.imp.runtime.services.menus.model.StrategoBuilder.execute(StrategoBuilder.java:200)
at org.strategoxt.imp.runtime.services.menus.model.StrategoBuilder.access$1(StrategoBuilder.java:184)
at org.strategoxt.imp.runtime.services.menus.model.StrategoBuilder$1.run(StrategoBuilder.java:171)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)

and

org.eclipse.e4.core.di.InjectionException: java.lang.NullPointerException
at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:63)
at org.eclipse.e4.core.internal.di.InjectorImpl.invokeUsingClass(InjectorImpl.java:243)
at org.eclipse.e4.core.internal.di.InjectorImpl.invoke(InjectorImpl.java:224)
at org.eclipse.e4.core.contexts.ContextInjectionFactory.invoke(ContextInjectionFactory.java:132)
at org.eclipse.e4.core.commands.internal.HandlerServiceHandler.execute(HandlerServiceHandler.java:167)
at org.eclipse.core.commands.Command.executeWithChecks(Command.java:499)
at org.eclipse.core.commands.ParameterizedCommand.executeWithChecks(ParameterizedCommand.java:508)
at org.eclipse.e4.core.commands.internal.HandlerServiceImpl.executeHandler(HandlerServiceImpl.java:213)
at org.eclipse.ui.internal.handlers.LegacyHandlerService.executeCommand(LegacyHandlerService.java:420)
at org.eclipse.ui.menus.CommandContributionItem.handleWidgetSelection(CommandContributionItem.java:837)
at org.eclipse.ui.menus.CommandContributionItem.access$21(CommandContributionItem.java:823)
at org.eclipse.ui.menus.CommandContributionItem$5.handleEvent(CommandContributionItem.java:813)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1057)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4170)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3759)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1113)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:997)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:138)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:610)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:567)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:124)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:354)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:181)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:636)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:591)
at org.eclipse.equinox.launcher.Main.run(Main.java:1450)
at org.eclipse.equinox.launcher.Main.main(Main.java:1426)
Caused by: java.lang.NullPointerException
at org.strategoxt.imp.runtime.services.menus.ActionHandler.execute(ActionHandler.java:24)
at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:290)
at org.eclipse.ui.internal.handlers.E4HandlerProxy.execute(E4HandlerProxy.java:90)
at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:56)
... 37 more

Submitted by Gabriël Konat on 13 December 2013 at 12:00

Same for the Stratego editor apparently. The same thing happened a while ago when I updated spoofax-libs.jar in strategoxt..

Still works in http://hydra.nixos.org/build/7135616 (although task engine has a bug in this build), breaks in http://hydra.nixos.org/build/7152344. So it can be caused by changes in http://hydra.nixos.org/eval/1034301 and http://hydra.nixos.org/eval/1034425, but I don’t see any changes in these builds that could cause this behaviour..

These are the changes in runtime-libraries:
https://github.com/metaborg/runtime-libraries/compare/8c54d299033de18c54fc416e6c51ac1d98d31b07…4da9212be8b0765d735a7aa07d1dd8c00d341b4d

And the change in spoofax:
https://github.com/metaborg/spoofax/commit/fa3d16430245d05b0201e68fa53155bb911de596

I’m not sure if anything changed in strj, I only see nixpkgs changes, but did anything else change?

The (intermediary) cause is that that Spoofax (or the HybridInterpreter) is unable to find any strategies shipped with a language. It’s still unclear whether this is caused by loading problem (i.e. the strategies are not loaded), a HybridInterpreter problem (the interpreter forgets things that were loaded) or a Spoofax problem (it forgets to reuse the HybridInterpreter created when the language was loaded). In any case, the problem occurs for all deployment-building types of languages: deployed, dynamically loaded, interpreted or compiled.

@Gabriël: can you try to recreate the last build (http://hydra.nixos.org/eval/1035926) locally in your installation of NixOS and see if the problem also occurs in a local build? Try to first build without updating nixpkgs and then after updating nixpkgs. If the first build works but the second one is broken then we need to report an issue to Hydra guys.

Yes, I was already doing this last Friday, but the build in my NixOS VM stopped working. I’ll continue messing around today.

As an update. This breakage is not limited to builders. It seems to affect pretty much all kinds of strategy invocations. For example the analysis function for Stratego files returns [] (aka no messages) even if errors are supposed to occur.

I’ve begun a bit of a debugging process where i first disabled the DebuggableHybridInterpreter and caching of HybridInterpreters. The latter implies a full reload of every language for every strategy call. Here are my findings so far:

• the interpreter seems to load and lookup strategies properly
• the analysis function and the outline functions are found when analysis/outline is needed. However their invocation acts strange. For example, the analysis function correctly returns a 4-tuple. The 4-tuple contains the correct AST but it contains no messages (it is expected to contain an error). So something seems to be going horribly wrong in the guts of the analysis. Nothing shows up in the Error Log though.

Here’s the log (stdout + stderr) produced when opening a Stratego file for the first time in an instance. The file contains errors which are expected to show up in the editor, but they do not:

[org.strategoxt.imp] Editor services loaded: Stratego-Sugar-55 ms
[org.strategoxt.imp] Loaded parse table for Stratego-Sugar-195 ms
Created observer for lang Stratego-Sugar
[org.strategoxt.imp] File parsed: xxx.str-20 ms
[org.strategoxt.imp] Created new Stratego runtime instance-1119 ms
[org.strategoxt.imp] Loading Stratego modules [include/stratego_sugar.jar, include/stratego_sugar-java.jar]
[org.strategoxt.imp] Loading JARs from [file:/private/tmp/eclipse/plugins/org.strategoxt.imp.editors.stratego_1.2.0.201312151922/include/stratego_sugar.jar, file:/private/tmp/eclipse/plugins/org.strategoxt.imp.editors.stratego_1.2.0.201312151922/include/stratego_sugar-java.jar]
[org.strategoxt.imp] Parent class loader: org.strategoxt.imp.editors.stratego.StrategoSugarParseController; org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader
[ libstratego-parallel ] using 8 threads, jobsize 1.0
[org.strategoxt.imp] Successfully loaded [include/stratego_sugar.jar, include/stratego_sugar-java.jar]-135 ms
[org.strategoxt.imp] Loaded analysis components-1254 ms
Looked up outline found SDefT("outline_0_0", outline_0_0)
Invoking outline on org.strategoxt.imp.runtime.Environment$1@27c88cf3
Looked up outline found SDefT("outline_0_0", outline_0_0)
[org.strategoxt.imp] Evaluated strategy outline result: []-8 ms
Invoking editor-analyze on org.strategoxt.imp.runtime.Environment$1@27c88cf3
Looked up editor-analyze found SDefT("editor_analyze_0_0", editor_analyze_0_0)
[org.strategoxt.imp] Evaluated strategy editor-analyze result: (Module("xxx",[Rules([SDefNoArgs("main",CallNoArgs(SVar("failzzzzz")))])]),[[]],[[]],[[]])-79 ms
Looked up outline-expand-to-level found null
Created observer for lang Stratego-Sugar
[org.strategoxt.imp] Created new Stratego runtime instance-4 ms
[org.strategoxt.imp] Loading Stratego modules [include/stratego_sugar.jar, include/stratego_sugar-java.jar]
[org.strategoxt.imp] Loading JARs from [file:/private/tmp/eclipse/plugins/org.strategoxt.imp.editors.stratego_1.2.0.201312151922/include/stratego_sugar.jar, file:/private/tmp/eclipse/plugins/org.strategoxt.imp.editors.stratego_1.2.0.201312151922/include/stratego_sugar-java.jar]
[org.strategoxt.imp] Parent class loader: org.strategoxt.imp.editors.stratego.StrategoSugarParseController; org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader
[org.strategoxt.imp] Successfully loaded [include/stratego_sugar.jar, include/stratego_sugar-java.jar]-6 ms
[org.strategoxt.imp] Loaded analysis components-10 ms
Invoking analysis-message-update-partitions on org.strategoxt.imp.runtime.Environment$1@5b7701a0
Looked up analysis-message-update-partitions found SDefT("analysis_message_update_partitions_0_0", analysis_message_update_partitions_0_0)
[org.strategoxt.imp] Evaluated strategy analysis-message-update-partitions result: []-1 ms

I’m confirming that the runtime aspects of Stratego/XT are not the cause of this failure. I’ve tried with a few older builds and the problem still occurs. This confirms that the Stratego/XT runtime is not cause but it does not say anything about the sanity of the compiler.

I can confirm that the problem is caused by incorrectly built projects. Unknown yet which language causes this, but below the method and findings:

1. Two installations of spoofax: SPX_GOOD and SPX_BAD
2. Create language in SPX_GOOD and build it.
3. Change the build-target in build.main.xml to prevent building: <target name="all" depends="refresh, sdf2imp.eclipse.load"/>
4. Close the project
5. Import the project in SPX_BAD
6. Build it (causes just a load of the language)
7. Open a file and test a builder
8. Builder works

The same behaviour happens in both ctree and jar modes.

An update. It seems that the Stratego/XT compiler shipped with Spoofax is fine. The MYLANG.jar built on SPX_GOOD and SPX_BAD are interchangeable.

On Hydra the C-version of Stratego is used to build Spoofax. But Spoofax itself is shipped with the java-bootstrap Stratego/XT. Perhaps there is a difference between these two.

I’ve updated the spoofax-libs.jar in the C-version of Stratego/XT. A build of Spoofax using this is pending. The evaluation is http://hydra.nixos.org/eval/1036659.

I’ve just tried the new Spoofax build which is based on the C-version of Stratego/XT with the latest spoofax-libs and unfortunately there is no difference. The problem persists.

The other observation is that only SDF3 and the Stratego editor seem to suffer from this problem, if you comment out the pp import and the formatter builder in the entity project it works fine. SDF3 and Stratego both have custom Spoofax build, maybe that is causing it.

Ok this is weird.. I changed the Stratego editor to use a default jar build, moved the observers and providers to the main file and use the default menus esv file. This gives me the following action:

action: "Show abstract syntax" = debug-show-aterm (openeditor) (realtime) (meta) (source)

This action works, even though the Stratego editor has no strategy called debug-show-aterm at all! Is it calling strategies on the wrong language?

I found the issue. The line that causes this breakage is https://github.com/metaborg/runtime-libraries/commit/f0cea93fe9d6e250d5d97a7cdb9dd2c0eae7cf88#diff-ab4d388c7345ffcf2ef3d3d03878a628L3. The problem here is that the Analysis language does not have any builders, because I forgot to import the menus esv file. This somehow triggers a bug in Spoofax that breaks analysis and builders for languages that still use legacy builders, like SDF3 and Stratego.

I will ‘fix’ this now by importing the menus ESV file, but Oskar should have a look at a real fix for this. A language that doesn’t contribute builders shouldn’t break other languages..

I’ve just installed Spoofax/IMP 1.2.0.0-s40925, so that’s the result of http://hydra.nixos.org/eval/1037019. The builders are working. But the analysis for SDF3, SPT, Stratego and GM is broken, no errors are reported and no resolution works. At least in the case of GM the “Show analysis > Partition” builders shows that there are no tasks and no entries in the index. The analysis for NaBL is working.

To leave no doubt, these are the tasks and index entries for a non-empty GM program:

	tasks
	
	
	index "/Users/vladvergu/Documents/workspaces/greenmarl/foo/bugs/4.gm"

I am super confused now, even after reverting the changes I made yesterday it’s broken again..

If I run a new Eclipse instance with the analysis language project opened (and built), everything works fine. Otherwise it’s broken. Maybe something is wrong with the build farm build? I’ll make another local build in my NixOS VM..

[Comment removed]

This issue is still somehow the Analysis language. I’ve just launched a new Spoofax/IMP 1.2.0.0-s40925, manually deselecting the analysis language from the list of plugins and everything appears to be working fine. SDF3 has resolution and Stratego has analysis messages.

What is the analysis language?

Language for inspecting tasks and the index.

I’m pretty sure it’s calling strategies on the wrong language. See the difference between the broken output (left), and the good output (right): http://diffchecker.com/tvxz26bm.
In the left file, once it starts to parse and analyze Stratego files you see the following output:

File parsed: legacy.str
Evaluated strategy editor-analyze
Evaluated strategy analysis-message-update-partitions

which indicates that it tries to run analysis-message-update-partitions, which does not exist in the Stratego editor.

The issue is only triggered by the Analysis language that is shipped from the build farm. If I build it myself and run a second Eclipse instance, or if I disable it in the second Eclipse instance, everything works fine.

Ok, so the menus thing wasn’t the cause of this issue, but changing the Analysis language from a ctree to a jar build is! I changed it back to a ctree build for now and everything seems to be working.

The question now is why changing from a ctree to a jar build can cause other languages to break.