babylm

Code & data for the EMNLP 2024 paper: Is Child-Directed Speech Effective Training Data for Language Models?

Code implementation for our paper "BERTtime Stories: Investigating the Role of Synthetic Story Data in Language Pre-training" as part of the 2024 BabyLM Challenge

Qiushi Engine autonomous research on BabyLM 2026 Strict-Small: two model generations, learning principles, reports, code, experiments and research notes.

UT Austin's submission to BabyLM Challenge

Code and data for the paper "Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models"

Vision-seeded word embeddings for a BabyLM-scale masked language model

Do state space models dream of recursive sheep? No, it was actually sinks all the way down.

Learning language by compression — an MDL language learner with no transformer and no backprop. Predicts, chunks, induces grammatical categories, and learns new words in one shot.

🍼 I taught a baby-sized LLM to chat. It went... poorly. Here's the code, try it for yourself

Multilingual Age-Ordered CHILDES

ACL Computational Developmental Linguistics 2026: Barbenel et al (2026)

This repository contains the training and evaluation code accompanying the thesis "Learning with Less: Contrastive Weight Tying on the BabyLM Challenge" (Ino van de Wouw, VU Amsterdam, 2025). The project studies headless language models, models pretrained with Contrastive Weight Tying (CWT) (Godey et al., 2024) instead of a standard cross-entropy

Looped transformers vs. multimodal degradation under the BabyLM 2026 100M-word budget

Baby AI is a minimalistic artificial intelligence system designed to evolve from scratch by observing character sequences and forming abstract concepts.

Code and data for Ding, Houghton & Jasbi, "Reading Acquisition Order out of BabyLM Representations: Child-Scale Training Produces Developmentally Aligned Semantic Spaces".