From b840e8e9ef4fb0f03a883e1cd0086f5c4ab74693 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Sat, 29 Aug 2026 00:16:39 +0200 Subject: [PATCH 01/15] AI in Metascience: publish the provisional programme Replaces the "programme coming in early September" placeholder with the sessions and the talks in each, in running order. No clock times and no abstracts: presenter availability was never collected, so every slot is still being confirmed with the authors. One reply moves everything after it in its session, and a day plus morning or afternoon is what a reader in another time zone needs in order to decide. Times and abstracts follow on 10 September. The block between the PROGRAMME markers is generated by build/public_programme.py in the submission-review repo, which reads the same submission records the committee decided on. Rerun it to refresh the block. Co-Authored-By: Claude Opus 5 --- content/ai-metascience/index.md | 196 +++++++++++++++++++++++++++++++- 1 file changed, 194 insertions(+), 2 deletions(-) diff --git a/content/ai-metascience/index.md b/content/ai-metascience/index.md index e1902f087a2..a61bc252a56 100644 --- a/content/ai-metascience/index.md +++ b/content/ai-metascience/index.md @@ -66,6 +66,73 @@ image: } .am-reg-day p { margin: 0; } .am-reg-cta { margin: 1.6rem 0; } +.am-prog { margin: 1.4rem 0; } +.am-prog-day { margin: 0 0 2rem; } +.am-prog-day h3 { + color: #004055; + border-bottom: 2px solid #004055; + padding-bottom: 0.3rem; + margin: 0 0 0.9rem; + font-size: 1.2rem; +} +.am-prog-day h3 span { + display: block; + font-size: 0.9rem; + font-weight: 400; + color: #5d6b70; + margin-top: 0.15rem; +} +.am-prog-half { + font-size: 0.8rem; + font-weight: 700; + letter-spacing: 0.05em; + text-transform: uppercase; + color: #5d6b70; + margin: 1.1rem 0 0.5rem; +} +.am-prog-session { + border: 1px solid #e6e2d2; + border-left: 4px solid #004055; + border-radius: 8px; + background: #fff; + padding: 0.9rem 1.1rem; + margin-bottom: 0.7rem; +} +.am-prog-session h5 { + margin: 0 0 0.3rem; + font-size: 1.02rem; + color: #004055; +} +.am-prog-blurb { + margin: 0 0 0.7rem; + color: #5d6b70; + font-size: 0.92rem; +} +.am-prog-session ul { list-style: none; margin: 0; padding: 0; } +.am-prog-session li { + padding: 0.4rem 0; + border-top: 1px solid #f0ede1; +} +.am-prog-session li:first-child { border-top: 0; padding-top: 0; } +.am-prog-title { font-weight: 600; } +.am-prog-who { + display: block; + color: #5d6b70; + font-size: 0.88rem; + margin-top: 0.1rem; +} +.am-prog-tag { + display: inline-block; + margin-left: 0.45rem; + padding: 0.05rem 0.4rem; + border: 1px solid #e6e2d2; + border-radius: 4px; + font-size: 0.7rem; + text-transform: uppercase; + letter-spacing: 0.04em; + color: #5d6b70; + vertical-align: 0.1rem; +} How is AI changing the way research is done, checked and understood? The **FORRT AI in Metascience Online Conference** takes three days over that question, with 33 accepted submissions from people who are studying it or building with it. Registration is now open, and everyone is welcome. @@ -116,9 +183,134 @@ How is AI changing the way research is done, checked and understood? The **FORRT -## When can I get the detailed programme +## Programme + +Here are the sessions and the talks in each, in the order they run. It is provisional while a few speakers confirm their slots. Talk times and abstracts follow on **10 September**; until then, plan by day and by morning or afternoon. Sessions run between 09:30 and 18:00 CEST (UTC+2). + + + + +
+
+

Monday 28 September AI in the research pipeline

+

Morning

+
+
When AI writes the literature
+

The problem as it already shows up in the correction record, the pressure producing it, and two routes to a better literature.

+
    +
  • AI-Generated Content as an Emerging Driver of Journal RetractionslightningAnton Sokolov (Tyche Institute)
    • +
  • When every hypothesis is published: agentic AI and the collapse of selection at productionMatt Spick (University of Surrey)
    • +
  • The Conceptual Armament Effect: Armed AI Reaches Above a Human 99th-Percentile PaperKenji Yamada (Independent Researcher, Satosho, Okayama, Japan)
    • +
  • How should AI-assisted research be documented? Seven guiding questionsMaximilian Frank (Ludwig-Maximilians-Universität München)
    • +
+
+
+
Evidence synthesis: capability, comparison, assurance
+

From what frontier models can extract, through a head-to-head test of the tools built on them, to the safeguards that would make an AI-assisted review checkable.

+
    +
  • Can frontier LLMs code studies for meta-analysis?Shaheed Azaad (University of Muenster)
    • +
  • Fast, but incomplete and inaccurate? Performance of AI-powered tools for systematic literature reviewsRene Bekkers (VU Amsterdam)
    • +
  • ReviewAid: confidence-scored, locally runnable LLM screening and extractionlightningVihaan Sahu (Georgian National University SEU)
    • +
+
+

Afternoon

+
+
Evaluating science, and its evaluators
+

What human review actually does, what a computational novelty signal might add, and a tool that checks a paper against what its authors said they would do.

+
    +
  • Gender and disciplinary differences in 39,280 SNSF grant peer review reportsStefan Müller (University College Dublin)
    • +
  • Tracing Scientific Novelty in Transformer Representations: Toward Computational Models of Novelty SignalsAnna Kruzenshtern (LUT University)
    • +
  • RegCheck: an LLM-assisted tool for comparing registrations with papersJamie Cummins (University of Bern)
    • +
+
+
+
Conditions for legitimate adoption
+

What researchers do with these tools, what open infrastructure and open data would require, who gains or loses when the tools arrive, and the rules of use that follow.

+
    +
  • How health researchers actually use GenAI: findings from a global surveylightningDanny Maupin (University of Surrey)
    • +
  • Small open-weight models as the path to genuinely open AI-assisted metasciencePeder Mortvedt Isager
    • +
  • washopenresearch: an open dataset for studying data-sharing practices in WASH researchLars Schöbitz (ETH Zurich)
    • +
  • AI and researchers with Specific Learning Disabilities: access gains and new inequitiesChelsea Sawyer (University of Manchester)
    • +
  • The FOCUS Framework for Optimal and Careful Use of Generative AI in ScienceMohamed Helmy (Vaccine and Infectious Disease Organization (VIDO), University of Saskatchewan)
    • +
+
+
+
Closing provocation
+

A challenge to the premise that complex problems are the kind of thing better tools solve.

+
    +
  • Rethinking how AI is approached to address wicked problems and global challenges in a polycrisis worldBatool Almarzouq (University of Edinburgh)
    • +
+
+
+
+

Tuesday 29 September Replications and reproductions

+

Morning

+
+
Reproducing the record
+

Replication and reproduction as work actually done, with agents in the loop and with the paperwork that makes a result reusable, followed out to whether anyone downstream notices.

+
    +
  • Thirty agent-assisted replications, published as nanopublications: the honest debriefAnne Fouilloux (Lifewatch ERIC, Spain), Jean Iaquinta (Vitenhub AS, Norway)
    • +
  • A local, open-weights pipeline for semi-automated replication checklistsJohannes Breuer (Center for Advanced Internet Studies Bochum, University of Duisburg-Essen), Mario Haim (Ludwig Maximilians University Munich), Philipp Knöpfle (Ludwig Maximilians University Munich), Xinyue Zhao (Center for Advanced Internet Studies Bochum)
    • +
  • Do failed replications make their way into public knowledge?Youyou Wu (University College London)
    • +
+
+
+
Symposium: how can AI help us understand animal-to-human translation in drug development?
+

Four talks and a discussion on the oldest replication question in biomedicine — whether a finding survives the move from an animal model to a human trial. Chaired by Benjamin Ineichen.

+
    +
  • How Can AI Help Us Understand Animal-to-Human Translation in Drug Development?Miriam Kober (University of Bern, Department of Clinical Research), Tim Chopard (University of Bern, Department of Clinical Research), Vera Bernhard (University of Bern, Department of Clinical Research), Simona Doneva (University of Bern, Department of Clinical Research)
    • +
+
+

Afternoon

+
+
Does the finding survive another analyst?
+

The other way a result fails to hold. The study is not repeated badly; a defensible alternative analysis simply lands somewhere else. And the alternatives increasingly consult the same model.

+
    +
  • Shared model dependence as a source of correlated error in evidence synthesisAlyssa Columbus (Johns Hopkins Bloomberg School of Public Health; Institute for Statistics, LMU Munich)
    • +
  • Multiverse Analysis in the Age of Large Language ModelslightningArdin Kropff (Erasmus University Rotterdam), Tygo van Beek (Erasmus University Rotterdam), Steven Verheyen (Erasmus University Rotterdam)
    • +
  • PaperRobust: a multi-agent framework for assessing analytical robustnesslightningQian Ma (The Pennsylvania State University)
    • +
+
+
+
Symposium: predicting replicability
+

Three talks and a panel on forecasting which findings will hold, and what such forecasts are for. Convened by Andrew Tyner.

+
    +
  • The Predicting Replicability Challenge: three rounds of automated credibility assessmentAndrew Tyner (Center for Open Science)
    • +
  • What are automated replicability predictions for?Lukas Röseler (University of Münster)
    • +
  • Multi-agent AI measures of methodological complexity help forecast study reproducibilityDaniele Fanelli (Heriot-Watt University and London School of Economics), Yuan Yingfang (Northumbria University and Heriot-Watt University)
    • +
+
+
+
Machines that build and test the replication record
+

If agents are going to do this work at scale, the record has to be assembled and the agents have to be measured against ground truth.

+
    +
  • The Metascience Observatory: an AI-built replication database and Bird's Eye ReviewsDan Elton (The Metascience Observatory)
    • +
  • Keeping humans in the loop: using LLMs to enable replication database validation at scalelightningHamidreza Behbood (University of Münster), Rohan Tondlekar (FORRT)
    • +
  • ReplicatorBench: an end-to-end benchmark for AI agents in research replicationBang Nguyen (University of Notre Dame), Dominik Soós (Old Dominion University)
    • +
  • Can AI reviewers catch the errors that matter? A 100-error planted benchmarkPaul Litvak (Robyn Dawes Institute)
    • +
  • Can an agentic LLM run computational reproducibility checks at human quality?Yoel Inbar (University of British Columbia)
    • +
+
+
+
+

Wednesday 30 September Workshops

+

Morning

+
+
    +
  • How to use PreCheck: An AI-Assisted Tool and Evaluation Framework for Preregistration QualityBence Palfi (Goldsmiths, University of London)
    • +
+
+

Afternoon

+
+
    +
  • A personal literature wiki, written and maintained by an AI agentYu Heng Daryl Lee (Department of Experimental Psychology, University College London), Tao Ma (Global Business School for Health, University College London)
    • +
  • HubMeta: an AI-assisted workflow for systematic review and meta-analysisPiers Steel (University of Calgary), Hadi Fariborzi (Mount Royal University)
    • +
+
+
+
-The full programme will be available here in early September. *Register now*, then come back in September to plan which time slots you want to keep clear. + ## Joining the sessions From 329d01a3b3cd347efcdaa3405b342964042dd7d9 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Sat, 29 Aug 2026 00:19:53 +0200 Subject: [PATCH 02/15] AI in Metascience: do not repeat the symposium title under its own heading Co-Authored-By: Claude Opus 5 --- content/ai-metascience/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ai-metascience/index.md b/content/ai-metascience/index.md index a61bc252a56..10895ab40aa 100644 --- a/content/ai-metascience/index.md +++ b/content/ai-metascience/index.md @@ -258,7 +258,7 @@ Here are the sessions and the talks in each, in the order they run. It is provis
Symposium: how can AI help us understand animal-to-human translation in drug development?

Four talks and a discussion on the oldest replication question in biomedicine — whether a finding survives the move from an animal model to a human trial. Chaired by Benjamin Ineichen.

    -
  • How Can AI Help Us Understand Animal-to-Human Translation in Drug Development?Miriam Kober (University of Bern, Department of Clinical Research), Tim Chopard (University of Bern, Department of Clinical Research), Vera Bernhard (University of Bern, Department of Clinical Research), Simona Doneva (University of Bern, Department of Clinical Research)
    • +
  • Miriam Kober (University of Bern, Department of Clinical Research), Tim Chopard (University of Bern, Department of Clinical Research), Vera Bernhard (University of Bern, Department of Clinical Research), Simona Doneva (University of Bern, Department of Clinical Research)

Afternoon

From 64f90f0fc4c441a042fdf393e902e44656e24ae7 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Sat, 29 Aug 2026 00:37:26 +0200 Subject: [PATCH 03/15] AI in Metascience: abstracts in a pop-up, and a clearer programme layout Every talk row opens its abstract in a modal, so the page carries the full content rather than a list of titles. The abstracts come from the same submission records the committee decided on. Layout, after a design pass: - Session headings now outweigh talk titles, so the page can be read by session rather than talk by talk. - Vertical rhythm separates day, half-day and session instead of running them together. - A chevron on each row says it opens something; hovering no longer shifts the text sideways. - Rows are at least 44px tall, the half-day label meets contrast, and a long abstract scrolls inside the dialog instead of running off a phone screen. Co-Authored-By: Claude Opus 5 --- content/ai-metascience/index.md | 384 +++++++++++++++++++++----------- 1 file changed, 252 insertions(+), 132 deletions(-) diff --git a/content/ai-metascience/index.md b/content/ai-metascience/index.md index 10895ab40aa..885fb68428f 100644 --- a/content/ai-metascience/index.md +++ b/content/ai-metascience/index.md @@ -66,72 +66,160 @@ image: } .am-reg-day p { margin: 0; } .am-reg-cta { margin: 1.6rem 0; } -.am-prog { margin: 1.4rem 0; } -.am-prog-day { margin: 0 0 2rem; } -.am-prog-day h3 { - color: #004055; +.am-prog { margin: 1.6rem 0 0; } +.am-prog-day { margin: 0 0 3.4rem; } +.am-prog-date { + display: flex; + flex-wrap: wrap; + align-items: baseline; + gap: 0.2rem 0.8rem; + margin: 0 0 0.2rem; + padding-bottom: 0.35rem; border-bottom: 2px solid #004055; - padding-bottom: 0.3rem; - margin: 0 0 0.9rem; - font-size: 1.2rem; + color: #004055; + font-size: 1.25rem; } -.am-prog-day h3 span { - display: block; - font-size: 0.9rem; +.am-prog-date span { + font-size: 0.92rem; font-weight: 400; color: #5d6b70; - margin-top: 0.15rem; } .am-prog-half { - font-size: 0.8rem; + display: flex; + align-items: center; + gap: 0.7rem; + margin: 2.6rem 0 0.8rem; + font-size: 0.75rem; font-weight: 700; - letter-spacing: 0.05em; + letter-spacing: 0.09em; text-transform: uppercase; - color: #5d6b70; - margin: 1.1rem 0 0.5rem; + color: #6e644b; +} +.am-prog-half::after { + content: ""; + flex: 1; + height: 1px; + background: #e6e2d2; } .am-prog-session { - border: 1px solid #e6e2d2; - border-left: 4px solid #004055; - border-radius: 8px; - background: #fff; - padding: 0.9rem 1.1rem; - margin-bottom: 0.7rem; + margin: 0 0 2.4rem; + padding-left: 1rem; + border-left: 3px solid #b5ad98; } -.am-prog-session h5 { +.am-prog-session h4 { margin: 0 0 0.3rem; - font-size: 1.02rem; + font-size: 1.18rem; + font-weight: 700; color: #004055; } .am-prog-blurb { - margin: 0 0 0.7rem; + margin: 0 0 0.55rem; color: #5d6b70; - font-size: 0.92rem; + font-size: 0.9rem; + max-width: 46em; } -.am-prog-session ul { list-style: none; margin: 0; padding: 0; } -.am-prog-session li { - padding: 0.4rem 0; - border-top: 1px solid #f0ede1; +.am-prog-talks { list-style: none; margin: 0; padding: 0; } +.am-prog-talks li + li { border-top: 1px solid #eae5d5; } +.am-prog-item { + display: grid; + grid-template-columns: 1fr auto; + gap: 0.1rem 1rem; + width: calc(100% + 1.2rem); + min-height: 44px; + margin: 0 -0.6rem; + padding: 0.55rem 0.6rem; + border: 0; + border-radius: 5px; + background: none; + font: inherit; + color: inherit; + text-align: left; + cursor: pointer; +} +.am-prog-item:hover, .am-prog-item:focus-visible { + background: #f4f2e8; } -.am-prog-session li:first-child { border-top: 0; padding-top: 0; } -.am-prog-title { font-weight: 600; } +.am-prog-item:hover .am-prog-title { color: #004055; } +.am-prog-title { font-size: 0.98rem; font-weight: 600; line-height: 1.35; } .am-prog-who { - display: block; + grid-column: 1; color: #5d6b70; - font-size: 0.88rem; - margin-top: 0.1rem; + font-size: 0.86rem; } -.am-prog-tag { +.am-prog-len::after { + content: " \203a"; display: inline-block; - margin-left: 0.45rem; - padding: 0.05rem 0.4rem; - border: 1px solid #e6e2d2; - border-radius: 4px; - font-size: 0.7rem; + margin-left: 0.25rem; + color: #004055; + font-size: 0.95rem; + font-weight: 700; + transition: transform 0.15s ease; +} +.am-prog-item:hover .am-prog-len::after, +.am-prog-item:focus-visible .am-prog-len::after { transform: translateX(3px); } +.am-prog-item-plain .am-prog-len::after { content: none; } +.am-prog-len { + grid-row: 1; + grid-column: 2; + align-self: start; + padding-top: 0.15rem; + color: #9a927c; + font-size: 0.75rem; + font-variant-numeric: tabular-nums; + white-space: nowrap; +} +.am-prog-item-plain { cursor: default; } +.am-prog-item-plain:hover { background: none; } +.am-prog-dialog { + width: min(38rem, calc(100vw - 2rem)); + max-height: 85vh; + overflow-y: auto; + overscroll-behavior: contain; + padding: 1.6rem 1.7rem 1.8rem; + border: 0; + border-top: 5px solid #004055; + border-radius: 10px; + background: #fffdf7; + color: #12181b; + box-shadow: 0 18px 50px rgba(0, 32, 42, 0.28); +} +.am-prog-dialog::backdrop { background: rgba(0, 32, 42, 0.45); } +.am-prog-close { + float: right; + margin: -0.6rem -0.6rem 0 0.6rem; + border: 0; + background: none; + color: #8d8266; + font-size: 1.7rem; + line-height: 1; + cursor: pointer; +} +.am-prog-close:hover { color: #004055; } +.am-prog-dialog-len { + margin: 0; + font-size: 0.75rem; + font-weight: 700; + letter-spacing: 0.09em; text-transform: uppercase; - letter-spacing: 0.04em; + color: #8d8266; +} +.am-prog-dialog-title { + margin: 0.2rem 0 0.3rem; + font-size: 1.15rem; + color: #004055; +} +.am-prog-dialog-who { + margin: 0 0 1rem; + padding-bottom: 0.9rem; + border-bottom: 1px solid #e6e2d2; color: #5d6b70; - vertical-align: 0.1rem; + font-size: 0.9rem; +} +.am-prog-dialog-abstract p { margin: 0 0 0.8rem; } +.am-prog-dialog-abstract p:last-child { margin-bottom: 0; } +@media (max-width: 480px) { + .am-prog-session { padding-left: 0.6rem; } + .am-prog-dialog { padding: 1.3rem 1.1rem 1.4rem; } } @@ -185,130 +273,162 @@ How is AI changing the way research is done, checked and understood? The **FORRT ## Programme -Here are the sessions and the talks in each, in the order they run. It is provisional while a few speakers confirm their slots. Talk times and abstracts follow on **10 September**; until then, plan by day and by morning or afternoon. Sessions run between 09:30 and 18:00 CEST (UTC+2). +Here are the sessions and the talks in each, in the order they run. **Click any talk to read its abstract.** The running order is provisional while speakers confirm their slots, so there are no clock times yet; those go up on **10 September**. Until then, plan by day and by morning or afternoon. Sessions run between 09:30 and 18:00 CEST (UTC+2).
-

Monday 28 September AI in the research pipeline

-

Morning

+

Monday 28 SeptemberAI in the research pipeline

+

Morning

-
When AI writes the literature
+

When AI writes the literature

The problem as it already shows up in the correction record, the pressure producing it, and two routes to a better literature.

-
    -
  • AI-Generated Content as an Emerging Driver of Journal RetractionslightningAnton Sokolov (Tyche Institute)
    • -
  • When every hypothesis is published: agentic AI and the collapse of selection at productionMatt Spick (University of Surrey)
    • -
  • The Conceptual Armament Effect: Armed AI Reaches Above a Human 99th-Percentile PaperKenji Yamada (Independent Researcher, Satosho, Okayama, Japan)
    • -
  • How should AI-assisted research be documented? Seven guiding questionsMaximilian Frank (Ludwig-Maximilians-Universität München)
    • -
-
+
    +
    • +
    • +
    • +
    • +
-
Evidence synthesis: capability, comparison, assurance
+

Evidence synthesis: capability, comparison, assurance

From what frontier models can extract, through a head-to-head test of the tools built on them, to the safeguards that would make an AI-assisted review checkable.

-
    -
  • Can frontier LLMs code studies for meta-analysis?Shaheed Azaad (University of Muenster)
    • -
  • Fast, but incomplete and inaccurate? Performance of AI-powered tools for systematic literature reviewsRene Bekkers (VU Amsterdam)
    • -
  • ReviewAid: confidence-scored, locally runnable LLM screening and extractionlightningVihaan Sahu (Georgian National University SEU)
    • -
-
-

Afternoon

+
    +
    • +
    • +
    • +
+

Afternoon

-
Evaluating science, and its evaluators
+

Evaluating science, and its evaluators

What human review actually does, what a computational novelty signal might add, and a tool that checks a paper against what its authors said they would do.

-
    -
  • Gender and disciplinary differences in 39,280 SNSF grant peer review reportsStefan Müller (University College Dublin)
    • -
  • Tracing Scientific Novelty in Transformer Representations: Toward Computational Models of Novelty SignalsAnna Kruzenshtern (LUT University)
    • -
  • RegCheck: an LLM-assisted tool for comparing registrations with papersJamie Cummins (University of Bern)
    • -
-
+
    +
    • +
    • +
    • +
-
Conditions for legitimate adoption
+

Conditions for legitimate adoption

What researchers do with these tools, what open infrastructure and open data would require, who gains or loses when the tools arrive, and the rules of use that follow.

-
    -
  • How health researchers actually use GenAI: findings from a global surveylightningDanny Maupin (University of Surrey)
    • -
  • Small open-weight models as the path to genuinely open AI-assisted metasciencePeder Mortvedt Isager
    • -
  • washopenresearch: an open dataset for studying data-sharing practices in WASH researchLars Schöbitz (ETH Zurich)
    • -
  • AI and researchers with Specific Learning Disabilities: access gains and new inequitiesChelsea Sawyer (University of Manchester)
    • -
  • The FOCUS Framework for Optimal and Careful Use of Generative AI in ScienceMohamed Helmy (Vaccine and Infectious Disease Organization (VIDO), University of Saskatchewan)
    • -
-
+
    +
    • +
    • +
    • +
    • +
    • +
-
Closing provocation
+

Closing provocation

A challenge to the premise that complex problems are the kind of thing better tools solve.

-
    -
  • Rethinking how AI is approached to address wicked problems and global challenges in a polycrisis worldBatool Almarzouq (University of Edinburgh)
    • -
-
+
    +
    • +
-

Tuesday 29 September Replications and reproductions

-

Morning

+

Tuesday 29 SeptemberReplications and reproductions

+

Morning

-
Reproducing the record
+

Reproducing the record

Replication and reproduction as work actually done, with agents in the loop and with the paperwork that makes a result reusable, followed out to whether anyone downstream notices.

-
    -
  • Thirty agent-assisted replications, published as nanopublications: the honest debriefAnne Fouilloux (Lifewatch ERIC, Spain), Jean Iaquinta (Vitenhub AS, Norway)
    • -
  • A local, open-weights pipeline for semi-automated replication checklistsJohannes Breuer (Center for Advanced Internet Studies Bochum, University of Duisburg-Essen), Mario Haim (Ludwig Maximilians University Munich), Philipp Knöpfle (Ludwig Maximilians University Munich), Xinyue Zhao (Center for Advanced Internet Studies Bochum)
    • -
  • Do failed replications make their way into public knowledge?Youyou Wu (University College London)
    • -
-
+
    +
    • +
    • +
    • +
-
Symposium: how can AI help us understand animal-to-human translation in drug development?
+

Symposium: how can AI help us understand animal-to-human translation in drug development?

Four talks and a discussion on the oldest replication question in biomedicine — whether a finding survives the move from an animal model to a human trial. Chaired by Benjamin Ineichen.

-
    -
  • Miriam Kober (University of Bern, Department of Clinical Research), Tim Chopard (University of Bern, Department of Clinical Research), Vera Bernhard (University of Bern, Department of Clinical Research), Simona Doneva (University of Bern, Department of Clinical Research)
    • -
-
-

Afternoon

+
    +
    • +
+

Afternoon

-
Does the finding survive another analyst?
+

Does the finding survive another analyst?

The other way a result fails to hold. The study is not repeated badly; a defensible alternative analysis simply lands somewhere else. And the alternatives increasingly consult the same model.

-
    -
  • Shared model dependence as a source of correlated error in evidence synthesisAlyssa Columbus (Johns Hopkins Bloomberg School of Public Health; Institute for Statistics, LMU Munich)
    • -
  • Multiverse Analysis in the Age of Large Language ModelslightningArdin Kropff (Erasmus University Rotterdam), Tygo van Beek (Erasmus University Rotterdam), Steven Verheyen (Erasmus University Rotterdam)
    • -
  • PaperRobust: a multi-agent framework for assessing analytical robustnesslightningQian Ma (The Pennsylvania State University)
    • -
-
+
    +
    • +
    • +
    • +
-
Symposium: predicting replicability
+

Symposium: predicting replicability

Three talks and a panel on forecasting which findings will hold, and what such forecasts are for. Convened by Andrew Tyner.

-
    -
  • The Predicting Replicability Challenge: three rounds of automated credibility assessmentAndrew Tyner (Center for Open Science)
    • -
  • What are automated replicability predictions for?Lukas Röseler (University of Münster)
    • -
  • Multi-agent AI measures of methodological complexity help forecast study reproducibilityDaniele Fanelli (Heriot-Watt University and London School of Economics), Yuan Yingfang (Northumbria University and Heriot-Watt University)
    • -
-
+
    +
    • +
    • +
    • +
-
Machines that build and test the replication record
+

Machines that build and test the replication record

If agents are going to do this work at scale, the record has to be assembled and the agents have to be measured against ground truth.

-
    -
  • The Metascience Observatory: an AI-built replication database and Bird's Eye ReviewsDan Elton (The Metascience Observatory)
    • -
  • Keeping humans in the loop: using LLMs to enable replication database validation at scalelightningHamidreza Behbood (University of Münster), Rohan Tondlekar (FORRT)
    • -
  • ReplicatorBench: an end-to-end benchmark for AI agents in research replicationBang Nguyen (University of Notre Dame), Dominik Soós (Old Dominion University)
    • -
  • Can AI reviewers catch the errors that matter? A 100-error planted benchmarkPaul Litvak (Robyn Dawes Institute)
    • -
  • Can an agentic LLM run computational reproducibility checks at human quality?Yoel Inbar (University of British Columbia)
    • -
-
+
    +
    • +
    • +
    • +
    • +
    • +
-

Wednesday 30 September Workshops

-

Morning

+

Wednesday 30 SeptemberWorkshops

+

Morning

-
    -
  • How to use PreCheck: An AI-Assisted Tool and Evaluation Framework for Preregistration QualityBence Palfi (Goldsmiths, University of London)
    • -
-
-

Afternoon

+
    +
    • +
+

Afternoon

-
    -
  • A personal literature wiki, written and maintained by an AI agentYu Heng Daryl Lee (Department of Experimental Psychology, University College London), Tao Ma (Global Business School for Health, University College London)
    • -
  • HubMeta: an AI-assisted workflow for systematic review and meta-analysisPiers Steel (University of Calgary), Hadi Fariborzi (Mount Royal University)
    • -
-
+
    +
    • +
    • +
+ + +

+

+

+
+ + + From 894bbdd34eb2d2e235c9570226f31c8ac1ec08cc Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Sat, 29 Aug 2026 09:53:37 +0200 Subject: [PATCH 04/15] AI in Metascience: count all 34 talks and workshops, not just the submitted ones Co-Authored-By: Claude Opus 5 --- content/ai-metascience/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/ai-metascience/index.md b/content/ai-metascience/index.md index 885fb68428f..33453aed3cc 100644 --- a/content/ai-metascience/index.md +++ b/content/ai-metascience/index.md @@ -4,7 +4,7 @@ type: plain_page url: "/ai-metascience/" hide_title: true hide_date: true -summary: "Registration is open for the FORRT AI in Metascience Online Conference, 28–30 September 2026. Online, free to attend, 33 sessions over 3 days." +summary: "Registration is open for the FORRT AI in Metascience Online Conference, 28–30 September 2026. Online, free to attend, 34 talks and workshops over 3 days." sharing_description: "Register for the FORRT AI in Metascience Online Conference, 28–30 September 2026. Online and free to attend." sharing_image_alt: "FORRT AI in Metascience Conference — online, 28–30 September 2026" image: @@ -223,7 +223,7 @@ image: } -How is AI changing the way research is done, checked and understood? The **FORRT AI in Metascience Online Conference** takes three days over that question, with 33 accepted submissions from people who are studying it or building with it. Registration is now open, and everyone is welcome. +How is AI changing the way research is done, checked and understood? The **FORRT AI in Metascience Online Conference** takes three days over that question, with 34 talks and workshops from people who are studying it or building with it. Registration is now open, and everyone is welcome.
@@ -234,7 +234,7 @@ How is AI changing the way research is done, checked and understood? The **FORRT
Cost
Free to attend.
Programme
-
33 accepted submissions in a single track, so you can follow the whole conference if you want - or dip in and out as you like..
+
34 talks and workshops in a single track, so you can follow the whole conference if you want - or dip in and out as you like.
From d7dcb7bc16d42dbb48ea1c0567ff15451829a4f1 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Sat, 29 Aug 2026 12:30:43 +0200 Subject: [PATCH 05/15] AI in Metascience: move Elton later in his session, and list Sokolov at TalTech Co-Authored-By: Claude Opus 5 --- content/ai-metascience/index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/ai-metascience/index.md b/content/ai-metascience/index.md index 33453aed3cc..8331c77a8b7 100644 --- a/content/ai-metascience/index.md +++ b/content/ai-metascience/index.md @@ -286,7 +286,7 @@ Here are the sessions and the talks in each, in the order they run. **Click any

When AI writes the literature

The problem as it already shows up in the correction record, the pressure producing it, and two routes to a better literature.

    -
    • +
    • @@ -363,10 +363,10 @@ Here are the sessions and the talks in each, in the order they run. **Click any

    Machines that build and test the replication record

    If agents are going to do this work at scale, the record has to be assembled and the agents have to be measured against ground truth.

      -
      • -
      • +
      • +
    @@ -392,7 +392,7 @@ Here are the sessions and the talks in each, in the order they run. **Click any

    - + + + + + + + + + +