If SDF3 modules contain errors, the build of a Spoofax project may still succeed, but not incorporate the changes in SDF3 modules. This is very confusing. The build should not succeed in such cases.

Submitted by Eelco Visser on 16 June 2015 at 18:41

What is even more confusing is that the parse table is not updated if the error occurs in a module that is not imported into the main module that makes up the language of the project. It should be possible to have SDF3 modules with errors in a project if they are not used.

Ok, identified the problem:

Scenario: I want to show different variants of a syntax definition. To do that, I have multiple variants of a module with the same name in different files. For example, I have a module A in file A.sdf3 and another module A in file B.sdf3. When building the project, the on-save handler is invoked for all sdf3 files. Since file B.sdf3 is processed after A.sdf3, the A module in src-gen contains the code from B.sdf3, even while I expect to get the code from A.sdf3, since that is what I think I import.

I now understand the behavior, but it is likely to be a common cause of errors. For example, it is not unusual for me to make a back-up copy of a file before embarking on some major change. Or in this particular case, I want to create multiple versions of a module for presentation and don’t want to change the name of the module.

I’m not sure what the solution should be.

But the key problem here is that the generator overwrites files that it generated based on another file. That is a form of name capture. This should be detected and reported, so that the user is aware of the problem.

Most languages with a module system require the file and module name to be equal. Using that in SDF3 would allow you to have a backup of module A in file B.sdf3. That file would have an error and therefore not generate code. Is that the kind of solution you’re looking for?

Currently the SDF3 type system already generates an error for file B.sdf3. But then it happily goes off and overwrites the files generated for module A in A.sdf3 without telling me about it. So yes, it should not generate code for file B.sdf3 and it should tell me that it is not doing that, but if that does not affect the modules that actually make up the syntax definition for the language (i.e. B.sdf3 is not reachable from the main module), then it should still generate a parse table for that main module.

I think a proper solution could be something like an annotation on builders stating that they should not run on files with errors. As a quick fix I added this check of file/module name on all sdf3 builders before generating code.

I agree with Eduardo, I also check manually in my transform builders that they should not run if there is errors in the analysis.

I guess Spoofax Core would need a notion of error which the parser, analysis, custom stratego code and even other language services (like failing to pretty print a target language) can trigger. Which then stops propagation on ‘the pipeline’ where pipeline would currently the built-in order of parse-analyse-transform. (But I already have a 2nd analysis phase, and transform usually consists of steps which follow each other which should also be cancelled when there is an error in one of them.)

Some builders should work when there are errors in the file, for example ‘Show abstract syntax’, but ones that generate code usually should not. Having an annotation on builders and the on-save handler to prevent them from running when there are errors is a good start, but the problem is more complex.

SDF3 files are not separately compilable, one reason being that the compilation for priorities may require knowledge from imported files. So if any of the imported files contain errors, the current file cannot be compiled. Also, because SDF3 is a meta-language, if SDF3 files contain errors the entire language build should fail.

Note that in the example that I’m describing, the error is in an SDF3 file that is not reachable (through imports) from the language module. So there should no reason for failing the build. I think it is perfectly normal to have some files with errors in my project that do not affect the build since they are not used. In this scenario these files were temporary copies of modules with alternative ways to define a language.

I’d like to note that in general SDF3 files that aren’t reachable from the SDF3 file with the start symbol may still be intended to be compiled. For example, in the Green Marl spoofax project I’ve defined desugared syntax in SDF3 files which aren’t even in the syntax/ directory. Those are not for parsing, but for the generated signatures and pretty-printing strategies.

Right. That is a useful scenario indeed. That means that an SDF3 module can be reached through the import of a Stratego signature module generated from it. A (generated) Stratego module that is not imported into the main Stratego module of the project is not reachable either. But, in the current setting we cannot reason very well about what contributes to the build of a project. So it might indeed be reasonable to let the build fail if there are files in the project with analysis errors. That means that scratch files need to be stored somewhere else (or given a different file extension). However, it should be completely clear to the user why the build fails.