Stefan Hugtenburg
13:44
To what extent is it possible to use the docker back-end of WebLab also for question generation (instead of only for answer validation?)

I’m thinking for example, is it possible to do this (see screenshot) with a docker instead somehow?

Image Pasted at 2021-3-19 13-44.png

---

Elmer van Chastelet
14:05
This interface is not supported by the docker backend atm. You can only specify the question generator and answer validator in scala.

---

Stefan Hugtenburg
14:26
I see :slightly_smiling_face: Are there plans to change this/Do you think this is even a possible change in the current confines of WebLab?

14:27
(So what I’m imagining is: WL puts seed into docker. Docker generates markdown for question (or even the full json stuff we now import with zip files), and depending on the question type can even grade the answer given the seed (programming, multiple choice, maybe even some open questions? :thinking: ))

---

Elmer van Chastelet
14:42
Its current implementation is limited to a single form of question (it renders the markdown text generated by the provided scala code) with a textual input that will be passed to the provided scala answer evaluator (together with the question text or identifier generated with the question).

I doubt if its current implementation is reusable to generate other types of questions. Also, spinning up a docker container for each generated questions/answer evaluation is quite resource hungry compared to the jvm-based backend it currently uses.
It’s quite a big FR.

The idea is to generate variants from within WebLab instead of generating the json files with custom tooling?

---

Taico Aerts
14:45
(Parameterized questions on weblab would be nice)

---

Stefan Hugtenburg
14:45
That would be my thoughts exactly yes, mostly for formative practice so that a student can practice with multiple variations of a question.

---

Taico Aerts
14:46
Parameterization would allow us to be much more flexible in variants. We can make individually unique questions in many cases and also automate the grading this way.

---

Stefan Hugtenburg
14:46
I realise that starting a docker to generate every question can be quite resource hungry, so if it remains jvm-based instead so that you can use that backend that is also okay with me, but then I would ask for a more flexible interface (also allowing multiple choice, and even programming assignments).

---

Yoshi van den Akker
14:47
Essentially MapleTA-like but with a decent language behind it, right?

---

Stefan Hugtenburg
14:47
Exactly that!

---

Taico Aerts
14:50
I think the concept behind the programmatically generated question is quite good and flexible. The markdown support you added for me earlier this year made it already much more usable. However, the lack of support for the checklists (and variables in there) and question types limits this again.

---

Elmer van Chastelet
16:04
My thoughts at the moment:
if scala is “decent” enough, extending the current interface might work.

add question types at the question generation side. (with support for EssayQ or MCQ in its first impl for example)
add generation of a manual grading component (checklist items with weights).
But this won’t work in current implementation of Generated Questions, because these only generate the question texts for each single submissions, and there is no such thing as checklists at the submission level. If we add that, UI components and computations need to be updated to work with this. (grading interface, grading stats, grade/statistics calculation)

Also, it would be good if the display of generated questions is no different than ordinary EssayQ/MCQs/… in WebLab. This would require changing the rendering code to deal with/have a interface to work with both types (the native WebLab type and generated type).

With this approach, stage 1 could be adding support for different types of questions and updating the interface to work with both generated and native MCQs/EssayQs.
In stage 2, generation of manual grading components can be added.

A different approach would be to generate the variants as WebLab assignments as we currently know them, using this interface. But the current data model is not designed to generate those assignments on-the-go (i.e. as students open the assignment). I foresee many technical issues with this approach.

---

16:08
I’ll copy this discussion to our issue tracker. I’m not sure if we can work on this soon. Our current focus is on improving the editor services, extending support for software testing, adding learning analytics and adding revision history for other question types.

Submitted by Elmer van Chastelet on 19 March 2021 at 16:14