Document the task-authoring contract: initial state, control reachability, and non-observable evidence

[matheusgalvao1]

Context

CodeSignal/maestro-templates generates graded learner practices for this simulation. The generator writes three artifacts per practice: a learner-facing description, a probability-lab/config.json, and acceptance criteria that are graded from the activity.log event stream.

We are moving those generators away from hand-transcribed copies of this repo's docs and toward reading this repo directly at generation time (see maestro-templates#90). That makes README.md the contract, so gaps in it become wrong tasks.

README.md already covers the config schema, the event catalog, the milestone schedule, and example log lines — that part is genuinely good and does most of the work. Three things it does not cover are the ones that produce broken practices.

Gap 1 — Initial state before any learner interaction

resetSingleSimulation auto-selects the first outcome when the event selection would otherwise be empty:

https://github.com/CodeSignal/learn_probability-lab/blob/main/client/app.js#L229-L231

if (filtered.size === 0 && def.labels.length > 0) {
  filtered.add(def.labels[0]);
}

This determines whether acceptance criteria may assume event.selectedOutcomes and event.pTheoretical are populated on a fresh load, or must require the learner to select the event first. Reading only README.md — or only the eventSelected: new Set() initializer at app.js#L158 — leads to the wrong conclusion.

Ask: state the initial state a learner sees before touching anything: default event selection, default device, default bias, trials: 0.

Gap 2 — Which config flags make UI controls unreachable

visualElements.editExperimentButton: false hides #pl-open-settings, which is the only caller of openSettingsModal (app.js#L853). That modal is where bias settings, spinner sector count, the two-mode relationship selector, and "Define event E" (index.html#L207) all live.

So setting that one flag to false removes the learner's only path to every setting the README lists under "What is not configured through config.json". A generator that reads the README sees those listed as "adjusted in the app UI" and can reasonably write a practice that asks for them — while also disabling the button.

Ask: for each visualElements key, document which controls it gates and what becomes unreachable when it is false.

Gap 3 — What is not observable in activity.log

Acceptance criteria can only grade what reaches the log. Currently undocumented:

• buildConfigSnapshot omits visualElements entirely, so app_start can never confirm which UI elements were shown.
• app_start.data.config for a custom device carries only mode, device, and sections — not outcomes or probabilities. Those are only visible later, via status.
• logCellClick returns early unless state.mode === 'two', so click events do not exist in single mode at all — bar-chart and frequency-table interactions are unlogged.
• sections in the log is normalized to a fixed key set, so keys appear regardless of mode.

Ask: a short "not observable" list next to the event table, so authors do not write criteria against evidence that is never emitted.

Suggested shape

A single ## Task Authoring Contract section in README.md, right after "Activity Logging and Grading", covering the three points above. Roughly 30 lines. Happy to open the PR if you'd prefer — just let us know whether you want it in README.md or a separate docs/ file.