Repository navigation
Release v0.5.0 - #63
Merged
Merged
Conversation
- Correct library(mockData) -> library(MockData) in example README (case-sensitive; example would fail as written) - Standardize GitHub links to canonical Big-Life-Lab/MockData slug - Add cran-comments.md for first CRAN submission - Ignore R CMD check artefacts and rendered vignette output in .Rbuildignore and .gitignore
Sequences all post-release work across three phases plus a deferred register, cross-linked to issues #35-#43. Phase 1 (v0.4.x hardening) is execution-ready with TDD steps verified against the current tree; Phase 2/3 workstreams are ADR-gated. Records the edge-case behaviour probe, key code locations, and the recoverable formula-evaluator spike ref as the plan's evidence base.
Review finding on #35: suppressMessages() does not catch warning() conditions. Matches the vignette's handling of the same call. Plan code block amended to match.
…c layer Relates to #36.
Same review finding and remediation as the Task 1 sanity anchor. Plan code block amended to match.
Review finding on #37: the adapter rate-forwarding and allowlist fixes had no end-to-end coverage, and the direct-API test's name overstated its scope.
…al docs Final-review findings: mid-signature insertion silently shifted positional rtype arguments; moving rate after rtype restores compatibility. Adds exponential roxygen examples, notes the simstudy hybrid path in NEWS, trims a resolved contingency comment, and syncs the plan document.
…rors, docs Five-agent review plus Copilot comments on PR #45. Rejects non-finite n in both guards (closes #44). Returns empty output for n = 0 before the exponential no-mass guard. Normalizes distribution case at spec construction so validation, allowlist, and generators agree. Wraps v0.4 spec construction so metadata validation failures carry the diagnosis plus guidance on validate = FALSE. Adds boundary tests for rate <= 0, the no-mass guard, and n = Inf. Corrects stale docs: native backend details, reference-config distribution tables, migrating vignette fallback example (exponential -> lognormal), NEWS wording on simstudy routing and clip-vs-truncate semantics.
v0.4.x hardening: edge-case contract, entry validation, native exponential
Records the seed/RNG-stream contract for v0.5: L'Ecuyer-CMRG stage sub-streams via parallel::nextRNGStream(), unified save/restore isolation across native/simstudy/legacy paths, RNG-kind pinning within the generation scope, and an always-on fallback stream-switch message. Co-released with the formula evaluator (#39) so v0.5 carries exactly one seeded-output break. Fulfils #22; resolves #33 item 11. Implementation is a separate ADR-gated plan.
Three-task plan implementing the accepted v0.5 seed-contract ADR: L'Ecuyer-CMRG per-stage sub-streams, unified path isolation, always-on fallback message, and the budgeted seeded-output re-baseline.
Replace the seed/seed+1L split with named, provably-independent sub-streams (baseline, postprocess) derived from one public seed via parallel::nextRNGStream(). .with_mock_seed() now pins RNGkind within scope and restores it, so output no longer depends on the caller's ambient RNGkind(). Native, postprocess, and simstudy paths route through it. Re-baselines value-pinning tests for the RNG change.
The RNGkind-independence test switched the ambient generator to Marsaglia-Multicarry, whose built-in "poor statistical properties" caution fired twice (the switch and the on.exit restore). Wrap the switch and its generation in suppressWarnings; the test's semantics (output identical across ambient kinds) are unchanged.
Route the legacy generation loop through .with_mock_seed() so it no longer clobbers the caller's RNG stream, matching the native path. Emit an always-on message when the v0.4 path silently falls back to the legacy generator for an unsupported feature (resolves #33 item 11). Re-baselines affected legacy value-pin tests.
Whole-branch review found that set.seed(kind = "L'Ecuyer-CMRG") pinned only the primary generator, so rnorm() and equal-probability sample() still leaked the caller's ambient normal.kind/sample.kind - partially breaking the ADR D3 'independent of ambient RNGkind()' guarantee. Pin all three components (a no-op under R >= 3.6 defaults, so pinned values are unchanged) and add a regression test that varies normal.kind and sample.kind. Scope the NEWS isolation claim to seeded generation calls.
The on.exit restore of the ambient sample.kind = Rounding re-triggered R's caution during the second generation call; extend suppressWarnings to enclose it. Test semantics (output identical across ambient kinds) unchanged.
Five-agent PR review findings. Suppress the RNGkind restore warning in production (previously only silenced in tests), validate the stage name before the seed early-return, and guard the legacy loop against a silent empty result. Pin the frozen .MOCK_STAGES indices and a postprocess-stage value so a future stage insertion can't silently shift seeded output. Add coverage for the always-on fallback message and simstudy isolation. Scope the NEWS 'all randomness' claim to the create_mock_data() pipeline (standalone create_* helpers are unchanged), fix a stale 'set global seed' roxygen bullet, and correct stale seed/seed+1L prose and a render-false worked example in the migrating and recodeflow vignettes.
v0.5 seed & RNG-stream contract (#38)
Records the formula-derived-variable contract for v0.5: a mockFormula extension column on variable_details, evaluated as a post-baseline stage over generated data in a restricted sandbox (generated columns + a base-function allow-list), with dependency ordering and cycle detection recovered from the v0.4 spike. Post-baseline evaluation adds no seeded-output break beyond #38's, so #39 co-releases with #38 in v0.5 without spending a second break. Phase A (algebraic) only; Func:: dispatch deferred. Depends on #38.
Four-task plan implementing the accepted v0.5 formula-evaluator ADR: formula spec type with construction-time sandbox validation, a post-baseline evaluate_mock_formulas() stage on the reserved formula sub-stream, mockFormula adapter wiring with the DerivedVar carve-out, and a stale-claims docs sweep. Additive: no seeded-output change, enforced by the #48 stage-index and pinned-value tests.
mock_formula()/mock_spec_formula() constructors populate the reserved formula/depends_on fields; validation rejects unparseable formulas, unknown referents, disallowed function symbols, and dependency cycles at construction, using the allow-list and topological ordering recovered from the v0.4 spike.
Review finding: the constructor's eager depends_on computation threw on syntax errors, bypassing error accumulation and the validate = FALSE opt-out that every sibling constructor honours. Parse failures now surface only through validation (making the per-variable parse branch live), with tests for the validate = FALSE path and multi-error accumulation. Plan code block synced.
…39) generate_mock_data_native() now skips type = "formula" variables (they carry no distribution to sample from) instead of rejecting the whole spec; a formula-only spec still returns the usual n-row, zero-column frame. The new exported evaluate_mock_formulas(data, spec, seed) computes each formula variable afterwards, in dependency order, under .with_mock_seed(stage = "formula"): evaluation runs in a restricted environment (data columns via as.list(), enclosing on a fixed base-function allow-list whose parent is emptyenv()) so a formula can never reach the caller's frame or globalenv, matching the sandbox contract from the ADR/plan. Also fixes a Task 1 review finding: .validate_formula_referents() and .order_formula_variables() re-parsed every formula and aborted on the first parse failure, which both duplicated the "could not be parsed" error and masked referent/cycle findings for other, parseable formulas in the same spec. Both now skip unparseable formulas (the per-variable validator already reports that error once) and keep checking the rest.
…in simstudy (#39) Task 2 review: the accumulate-and-skip helpers let evaluate_mock_formulas() silently drop unparseable formula variables under validate = FALSE; the evaluator now strict-validates its spec at entry like postprocess_mock_data(), so bad formulas fail loudly before generation. The simstudy baseline path gains the same formula-variable skip as the native backend, restoring the documented two-step workflow. Pins the length-mismatch, single-row scalar, and formula-only-spec edges.
Adapter (R/mock_spec_recodeflow.R):
- .details_mock_formula(details) reads the first non-blank mockFormula
among a variable's detail rows ("" if absent/blank).
- .recodeflow_to_spec_variable() constructs via mock_spec_formula() when a
non-blank mockFormula is present, ahead of the kind-dispatched
constructors, reusing the row's already-extracted rtype/missing-code/
garbage settings so postprocess behaves identically to other variables.
- The exclude_derived block now keeps DerivedVar::/Func:: rows that carry a
non-blank mockFormula (D6 carve-out); bare DerivedVar::/Func:: rows stay
excluded exactly as before.
Orchestrator (R/create_mock_data.R):
- Inverted the unsupported-variable check: type == "formula" is now
supported end-to-end. A stray non-empty `formula` field on a
non-formula variable still triggers the legacy fallback (defensive).
- Pipeline is now baseline -> evaluate_mock_formulas() -> postprocess;
comment updated to name all three seed sub-streams.
Diagnostics (R/mock_spec_postprocess.R):
- .postprocess_empty_diagnostics() adds derived = TRUE, formula, and
depends_on to each formula variable's diagnostics entry (D5).
Tests: extended tests/testthat/test-formula-vars.R with the Task 3 brief's
end-to-end mockFormula test, the D6 backward-compat exclusion test, the
formula-missing-codes postprocess test, and the formula-free no-op pin.
Full suite: FAIL 0 | WARN 35 | SKIP 12 | PASS 746 (738 baseline + 8 new,
no new WARN/SKIP). #48 seed-contract pins: 13/13 pass untouched.
devtools::check(): 0 errors, 0 warnings, 3 pre-existing notes.
Review finding: the exclude_derived carve-out read unfiltered variable_details while construction read databaseStart-filtered rows, so a variable with a mockFormula in one cycle but not another crashed generation for the cycle without it instead of staying excluded. Both decisions now use the same .filter_recodeflow_details() view, with a multi-database regression test pinning both directions.
Add R CMD check CI and capture simstudy in renv.lock
Formula-derived variables, Phase A (#39)
Expand range-notation missing codes on the v0.4 path (#58)
An external review (Astra) raised four points; two were reproduced against the code and change behaviour: - D8's example derived status and time from different windows. For a death on day 50 and administrative censoring on day 20 it gave status 1 with follow-up 20. Status and time now come from one observation window, ties count as observed events (matching the strict censoring rule), and the plan's tests pin both. In a 400-row check, 114 of 200 generated deaths fell after administrative censoring and are not observed. - Chained censoring gave a wrong observed outcome: with A (day 30) censored by B (day 20) censored by C (day 10), A was reported as observed although observation ended on day 10. v0.5 rejects a censored_by target that is itself censored. The other points are framing, now stated as limits: the three data layers (clean truth, observed data, analysis variables) and how to keep the clean layer; the fixed stage order and the fixed-count event allocation as v0.5 restrictions rather than the definition of survival generation; and a corrected assessment of exposure-dependent extensions, where time-varying exposure is a substantial generator extension and feedback between health states needs a longitudinal mechanism.
Pins the create_wide_survival_data() behaviour the metadata-driven stage must reproduce: the death-censors-event rule in both directions, the exact floor(n * event_prop) event count, whole-day follow-up inside the window, no date before entry on the minimal example, and the #54 Gompertz clamp.
mock_spec_survival() builds a type = "survival" date with anchor, censored_by, follow-up window, event_prop and distribution parameters, and records depends_on = c(anchor, censored_by). validate_mock_spec() checks each variable's parameters and the spec-level referents (anchor is a date in the spec; censored_by is an uncensored survival variable with the same anchor, since chained censoring gives wrong observed outcomes) and reports mutual censored_by as a dependency cycle. The #39 formula ordering becomes a shared .order_dependent_variables(), which formulas and survival dates both use (ADR D7); formula ordering and its messages are unchanged.
Survival variables are drawn from their anchor dates with the legacy statistics ported exactly (fixed floor(n * event_prop) event count, uniform/exponential/Gompertz follow-up in whole days) and processed in dependency order: each is drawn, then dates before the anchor and dates after an earlier censored_by date become NA. Draws use a new survival sub-stream appended to .MOCK_STAGES as index 4, so no existing index or pinned value moves. Both backends now skip derived types (formula and survival) in baseline generation. Postprocess diagnostics mark survival variables as derived with their anchor, censored_by, depends_on and n_events. Tests pin D5 (garbage survives the rules; the rules use true dates) and the survival stream, whose reference values were computed independently of this code.
#40) The adapter builds a survival variable from any date row that sets anchor (reading censored_by and the survival parameters), and rejects a date with survival parameters but no anchor with a message naming the fix. The orchestrator runs baseline, survival, formulas, then postprocess, and no longer treats survival dates as a legacy-fallback trigger. When the legacy generator is selected on anchored metadata, create_mock_data() stops and names the survival variables instead of dropping them (ADR D10). The minimal example gains anchor and censored_by, so it now generates entirely through the v0.4 pipeline. Two legacy-path seed tests drop its survival rows, and the validate = TRUE skipped-summary test reaches the legacy path through detail-level databaseStart instead of an anchor-less survival date, which the adapter now rejects.
Adds is.na to the formula allow-list (survival ADR D8, amending the #39 ADR's D3). Tests derive status and follow-up time from one observation window (a death after administrative censoring is not observed; a death on the censoring date is), and pin that formula columns describe clean truth: a missing code applied to the date afterwards leaves them unchanged.
Warns once per session, pointing to anchor/censored_by metadata and create_mock_data(). The function is otherwise unchanged: it is the reference the characterization tests pin. A test helper marks the warning as shown so the legacy survival tests gain no unrelated warnings.
Rewrites the survival tutorial around anchor metadata and create_mock_data(), including the competing-risk rule, status and follow-up time derived from one observation window, the three data layers (clean truth, observed data, analysis variables), and notes on #54 and #55. Updates the garbage tutorial's survival section, the configuration reference (anchor, censored_by, date distributions, the extension-column placement rule, and worked examples that lacked anchor), advanced topics, design philosophy, the recodeflow-users guide and the migration guide's fallback list. NEWS gains the #40 section and marks the v0.2.0 known issue resolved. The create_mock_data() fallback-mode example ran the minimal example with variable_details = NULL, which the D10 guard now stops because the fixture sets anchor; the example drops the anchored rows first. R CMD check --as-cran: 0 errors, 0 warnings, 2 notes.
The v0.4 wrapper appended "or call create_mock_data() with validate = FALSE to use the legacy generator" to every metadata error, including the new survival ones. Taking that advice either hits the D10 stop (anchored metadata) or silently drops the survival dates (metadata still lacking anchor), which is the behaviour #40 removes. When the metadata describes survival dates (anchor set, or survival parameters present), the error now says validate = FALSE is not a workaround and asks for the metadata fix instead. Found by the whole-branch review.
MockData's vignettes dogfood the package and R CMD check rebuilds them in CI, so they are integration tests, but only if a render fails when the code stops doing what the prose says. Every factual claim in the survival docs is now computed with inline R and guarded by a hidden chunk of named stopifnot() conditions, including the #54 and #55 caveats, which will fail the build until the docs are updated when those issues are fixed. The survival documentation is split across the site's sections: the tutorial keeps the teaching path; a new how-to (survival-dates-v05) covers adding survival dates to metadata, migrating from create_wide_survival_data(), deriving observed status and follow-up time from one observation window, and keeping the uncontaminated data; a new explanation (survival-design-v05) brings the ADR's reasoning onto the site: the stages, event allocation, rules on clean dates, the three data layers, why chained censoring is rejected, and the v0.5 restrictions. A mutation check showed the assertions catch regressions: flipping the censoring comparison failed the tutorial and the how-to ("no event is recorded after a death"), and making it non-strict failed the explanation's tie claim. CONTRIBUTING.md now states the rule and the pattern; #62 tracks retrofitting the other vignettes. R CMD check --as-cran: 0 errors, 0 warnings.
Three findings from a review of b0a4e97, each verified before changing: - The explanation claimed that adding survival dates does not change other variables' values. That holds for generated values, but post-processing draws missing codes and garbage from one stream shared by all columns, so a survival date with missing codes placed before another column changed 42 of 100 of its returned values. The claim is narrowed to generated values, and the page now demonstrates and asserts both halves. - The tutorial's 'every removed event has an earlier death' checked only that a death date existed. The demo event now falls on a fixed day, so the claim asserts the death came before it; with the censoring rule flipped, the render fails on exactly this claim. - The layers example used !is.na(death), which cannot tell clean from contaminated dates because the missing code is not NA. It now uses as.numeric(death - entry): 100 to 200 days on clean dates, about -37,095 if recomputed from the returned missing code, and both are asserted.
…40) The explanation said post-processing was the only exception to seed stability when a survival date is added. Survival dates also share one stage stream, drawn in dependency order, so a survival date drawn earlier changes later ones: adding one before death changed 70 of 100 clean death dates with the same seed (a reviewer saw 74 in another setup), while adding one after it changed none. The paragraph now states both exceptions and the page asserts all five facts: baseline values unchanged, earlier and later survival dates, and post-processing with and without missing codes.
Metadata-driven survival dates in create_mock_data() (#40)
… 0.5.0 Brings #51, #52, #59 and #61 into release prep. NEWS: the dev entries for survival dates (#40), range missing codes (#58) and the minimal-example fixture corrections are filed under the 0.5.0 categories (breaking changes, new features, validation, bug fixes) plus new Deprecations and Known issues headings; every dev bullet is present, two in the forms release prep had already rewritten, which dev had not changed since the branches split. The heading carries the provisional release date. README mentions survival dates in the 0.5 summary and feature list. cran-comments corrects the local check record: the local check library lacks simstudy (3 notes, run with _R_CHECK_FORCE_SUGGESTS_=false), while CI installs all suggested packages and reports Status: OK.
Ticks the four Task 8 items with how each was met: #38, #39 and #40 ship together with one migrating paragraph; pinned seed tests in test-seed-contract.R; R CMD check --as-cran clean on the 0.5.0 tarball and pkgdown builds; version 0.5.0, with the extension columns documented in MockData rather than in a separate downstream notice (maintainer decision, 2026-09-22).
Prepare v0.5.0 release
Found in the 0.5.0 release review. When any variable forces the legacy path (an unsupported feature, validate = FALSE, and the other legacy triggers), the legacy generator ignored mockFormula: z = x * 2 came back as unrelated random values (0 of 200 rows matched), or vanished entirely with DerivedVar:: notation, with default validation on. create_mock_data() now stops and names the formula variables, mirroring the survival guard (ADR v05-survival-dates D10). The check runs before the legacy derived-variable exclusion, which would otherwise hide DerivedVar:: rows, and scopes mockFormula to the requested databaseStart the same way the v0.4 adapter does. Both guards share one description of when the legacy path is used.
) height's high-garbage range had an infinite upper bound, (2.1;inf], so sampling produced NaN in the packaged example's default output. The range is now (2.1;2.5] (one cell, changed through read.csv/write.csv), and a contract test pins that the example produces no NaN. Closes #60. README: the minimal-example output showed three columns from the legacy path; it is regenerated by running the code (five representative columns, with a note on the missing codes it shows). The two other README outputs were regenerated too and still match. Closes #49. NEWS: the fixture-corrections entry no longer says survival dates stay off the v0.4 pipeline, and mentions the height range; a Breaking changes entry records the new formula legacy guard.
Found in the 0.5.0 release review. .gitignore excluded man/*.Rd, so the repository held no Rd files: installing from GitHub gave "No man pages found" and no help for any function, including create_mock_data() and mock_spec_survival(). CI hid this because it runs roxygen2 before checking. The 61 generated Rd files are now tracked, as is usual for R packages; regenerate them with devtools::document() when roxygen comments change.
Fix 0.5.0 release blockers from the release review
Found in a second release review of 0.5.0. The legacy path chose which variables to generate from detail rows alone and ignored variable-level databaseStart, which the v0.4 adapter honours. So a formula or survival variable listed only for another database made the new guards stop generation for the requested one, and a plain variable listed only for another database was generated as random values, in every detail layout tested (no detail databaseStart, shared rows, other-only rows). The legacy path now filters enabled variables by variable-level databaseStart with the adapter's matcher and empty-means-all rule, before the guards run. Scoping only the guards' candidates would have removed the false stop but left the legacy path generating another database's formula variable as random values. This also corrects an earlier ruling that the survival guard's lack of databaseStart scoping could stand.
Scope legacy variable selection by databaseStart (0.5.0 release review)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges the v0.5.0 release from
devtomain.DESCRIPTIONcarries Version 0.5.0 and NEWS a dated# MockData 0.5.0 (2026-09-27)section (release prep in #53).Highlights since v0.4.0 (full details in NEWS.md):
mockFormulacolumn evaluated in a sandbox after baseline generation;DerivedVar::/Func::semantics unchanged.anchorandcensored_byonvariables.csv;create_mock_data()generates entry, event, death, loss-to-follow-up and administrative-censoring dates. The legacy statistics are ported exactly, andcreate_wide_survival_data()is deprecated.n = 0, stricter entry validation, native exponential distribution.[997,999]on the v0.4 path (v0.4 path fails on range-notation missing codes such as [997,999] #58, PR Expand range-notation missing codes on the v0.4 path (#58) #59; categorical variables had silently written the literal code into the data); the simstudy backend atn = 0(simstudy backend fails for n = 0 categorical variables (pre-existing) #50).R-CMD-checkworkflow on every push and PR tomain/dev; simstudy is captured inrenv.lock.CONTRIBUTING.mdstates the rule. Help pages (man/*.Rd) are now committed, so GitHub installs have help.z = x * 2matched 0 of 200 rows when another variable forced the fallback).create_mock_data()now stops and names them, mirroring the survival guard. The same PR commits the help files, fixes the minimal example'sNaNheights (Minimal example: height garbage range (2.1;inf] produces NaN values #60) and refreshes the README output (Refresh stale README create_mock_data() example output #49).databaseStart. A variable listed only for another database was generated as random values, and the new guards stopped on it. The legacy path now selects variables with the adapter'sdatabaseStartmatcher.State of the branch
[ FAIL 0 | ERROR 0 | WARN 35 | SKIP 3 | PASS 954 ](v0.4.0:[ FAIL 0 | WARN 35 | SKIP 10 | PASS 641 ])R CMD check --as-cran --no-manualonMockData_0.5.0.tar.gz: 0 errors, 0 warnings, 3 notes (new submission, simstudy not in the local check library, timestamp). CI:Status: OK.pkgdown::build_site()builds cleanly; merging this PR deploys the site frommain.dev's release-fix commits has help pages (62 topics); before, it had none. Follow-ups: [v0.6] ADR: sidecar overlay for MockData extension columns #57 (v0.6 sidecar ADR), Retrofit claim assertions into the existing vignettes (vignettes are integration tests) #62 (vignette assertions elsewhere), Add survival censoring and LTFU regression tests #23, Follow-ups from PR #32 review: loader validation, strict-mode semantics, doc polish #33, Route standalone create_*/survival generators through the seed primitive for package-wide RNG isolation #47.Issues this release resolves
Closing keywords only fire on merges into
main, so the phase PRs intodevdid not close these:Closes #22
Closes #35
Closes #36
Closes #37
Closes #38
Closes #39
Closes #40
Closes #44
Closes #50
Closes #58
Closes #60
Closes #49
(#39: Phase A ships here; the ADR moves Phase B,
Func::dispatch, to a separate future workstream.)After merging
Tag the merge commit
v0.5.0. If the release date is not 2026-09-27, update the NEWS heading first.🤖 Generated with Claude Code