Rune-R1

A 351M-parameter GPT trained from scratch on 5B tokens, then reasoning-tuned with SFT + GRPO — end to end on a single GPU.

---

A small GPT-style language model, built and trained from scratch, then reasoning-tuned with supervised fine-tuning (SFT) and GRPO (group-relative policy optimization) on math problems — end to end on a single AMD MI300X GPU.

Architecture

rune/model.py — a ~351M parameter decoder-only transformer:

tokens → embed → [22x Block(GQA attention w/ RoPE + QK-norm, SwiGLU FFN, RMSNorm)] → RMSNorm → logits

• RoPE (rotary position embeddings)
• RMSNorm (with QK-norm)
• SwiGLU feed-forward
• Grouped-query attention (GQA), with a KV cache for fast incremental decoding
• GPT-2 (tiktoken) vocabulary

params	351M
layers	22
emb dim	1024
heads / kv groups	16 / 4
context length	1024

Pipeline

Three stages, each continuing the previous stage's weights:

pretrain (scripts/train.py)  →  SFT (scripts/sft.py)  →  GRPO (scripts/grpo.py)
   base LLM                     learns the CoT format      learns to be *correct*

1. Pretraining — next-token prediction on streamed FineWeb-Edu. Produces checkpoints/ckpt_latest.pt, a general-purpose base LLM (no instruction-following yet). → 🤗 samueljayasingh/rune-0.3b-base
2. SFT — fine-tunes the base checkpoint on distilled chain-of-thought math solutions (<think>...</think> + boxed final answer), teaching the response format. → 🤗 samueljayasingh/rune-0.3b-sft
3. GRPO — RLVR (reinforcement learning from verifiable rewards): samples multiple responses per problem, rewards the ones with the correct final answer (verified symbolically via rune/verifier.py, MATH-style), and updates the policy with a PPO-style clipped objective plus a KL penalty back to the SFT checkpoint. → 🤗 samueljayasingh/Rune-R1 (final model)

Workspace Structure

rune/                         installable library (pip install -e .)
  model.py                    transformer + KV cache
  data.py                     tokenization / streaming dataset
  generate.py                 batched, KV-cached sampling
  verifier.py                 MATH answer extraction/grading, dataset loaders
scripts/                      runnable pipeline stages
  train.py                    base pretraining
  sft.py                      supervised fine-tuning on distilled CoT data
  grpo.py                     GRPO (RLVR) reasoning training
  ops.py                      supervise/watch-grpo/budget CLI (crash recovery, plateau detection, cost tracking)
evals/
  eval_base.py                perplexity (held-out FineWeb-Edu) + sample generations for the base checkpoint
  eval_all.py                 perplexity / format-compliance / pass@k, compared across all 3 checkpoints
  plot_results.py             loss curves + reasoning-metrics charts from the logs + eval_all.py results
  plot_efficiency.py          inference throughput + precision/memory-footprint charts (no GPU needed)
  plot_training_summary.py    one consolidated pretrain+SFT+GRPO overview chart
  plots/                      generated charts
tests/
  test_ops.py
data/                         downloaded/cached datasets (gitignored)
checkpoints/                  model checkpoints (gitignored)
logs/                         training logs

Setup

python3 -m venv .venv && source .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/rocm7.2  # or the CUDA wheel for your GPU
pip install -e .
pip install -r requirements.txt

Usage

All commands run from the repo root.

Self-tests (no GPU required):

python3 rune/model.py       # forward pass + KV-cache correctness check
python3 rune/data.py
python3 rune/generate.py
python3 rune/verifier.py
python3 tests/test_ops.py

Pretraining, supervised (auto-restart on crash, auto-stop on plateau or deadline):

python3 scripts/ops.py supervise --max-hours 26 \
  -- --data-mode fineweb --fineweb-config sample-10BT --max-tokens 5000000000 \
     --batch-size 64 --grad-accum 8 --ckpt-dir checkpoints

Base model eval:

python3 evals/eval_base.py --ckpt checkpoints/ckpt_latest.pt

SFT:

python3 scripts/sft.py --pretrained-ckpt checkpoints/ckpt_latest.pt

GRPO:

python3 scripts/grpo.py --sft-ckpt checkpoints_sft/sft_latest.pt --steps 500
python3 scripts/ops.py watch-grpo   # separate terminal: alerts on reward collapse / policy drift

Evaluate and compare all three checkpoints:

python3 evals/eval_all.py --n 20 --pass-k-n 10 --k 8   # perplexity, boxed-format rate, pass@k
python3 evals/plot_results.py                          # loss curves + reasoning-metrics charts
python3 evals/plot_efficiency.py                        # inference throughput + precision charts
python3 evals/plot_training_summary.py                  # one consolidated overview chart + table

Hardware

My choice of hardware consisted of a single hourly-rented droplet that I used across all three pipeline stages.

GPU	1x AMD MI300X
VRAM	192 GB
vCPU	20
RAM	240 GB
Boot disk	720 GB NVMe SSD
Scratch disk	5 TB NVMe SSD

Results

All three pipeline stages completed on a single MI300X, for well under $100 total.

Stage	Steps	Key metric
Pretraining	9,624	loss 10.88 → 2.999 (5.05B tokens)
SFT	5,650	val loss 2.55 → 1.400 (2 epochs, distilled CoT math data)
GRPO	2,001	mean reward 0.010 → 0.023 (1st half vs 2nd half); 152/2001 steps got nonzero reward

Inference efficiency & precision

Throughput is essentially identical across all three checkpoints (~73-75 tok/s, KV-cached greedy decoding on one MI300X) — reasoning-tuning changes behavior, not inference cost, since the architecture never changes. Weights footprint: 1.4GB (fp32, training) down to 351MB (int8) for deployment.

Qualitative: what SFT and GRPO actually add

Sample responses to the same problem across all three stages:

Base (no instruction-tuning): loops the input back, no reasoning —

(0,3)$
Answer: (0,3)$
Answer: (0,3)$
Answer: (0,3)$
...

SFT (learned the CoT format): produces real, structured reasoning —

<think>Okay, so I need to convert the rectangular coordinates (0,3) to polar
coordinates. Let me think about how to do this. The problem says that r is 0,
and the point (0,3) is in polar coordinates...

GRPO (reward-tuned on top of SFT): same structured format, and — unlike SFT on this example — actually converges to a final answer instead of spinning indefinitely:

Problem: What is $1^{(2^{235423523})}$? (ground truth: 1)

GRPO — reasoning is still imperfect, but it commits to an answer: ...The square root of 1 is 1, so the answer should be 1.</think> \boxed{1}. Correct.

This is the clearest evidence that GRPO added something beyond SFT: reward only comes from producing a correct boxed answer, so looping forever gets zero reward every time — exactly the behavior RLVR training is meant to suppress.

Sample prompts across all three stages

Same prompt, same generation settings (100 tokens, greedy), run through the base, SFT, and GRPO checkpoints:

Math prompt — "You are a helpful math assistant.\nAnswer the question and wr[ite the answer]..." (12 + 15):

Stage	Response
Base	12 + 15 = 12 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + 15 + ...
SFT	<think>Okay, so I need to find 12 + 15. Hmm, let me think. I remember that when you have two numbers, the sum of their squares is equal to their sum divided by the product of their squares... — CoT format learned
GRPO	<think>Okay, so I need to find the value of 12 plus 15. Hmm, let me start by recalling what the question is asking. The question is asking for the answer to the number 12 plus 15... — same format, still reasoning at the 100-token cutoff

General prompt — "The capital of France is":

Stage	Response
Base	Paris.\nThe city is the capital of France. It is the largest city in the world. It is the largest city in the world. ...
SFT	Paris. The city is also known as the capital of France. The city is also known as the capital of France. ...
GRPO	Paris. The city is also the capital of the French Republic.\nThe city is also the capital of the French Republic. ...

Story prompt — "Once upon a time,":

Stage	Response
Base	the first thing that happened was that the first person to see the moon was a woman. The first woman to see the moon was a woman named Mary...
SFT	the first thing I remember is that the first person to write a book is the first person to write it. So, the first person to write a book is the first person to write it...
GRPO	the first person to reach the top of the pyramid was a woman. The first woman to reach the top of the pyramid was a man...

All three stages get the fact right (Paris) — general-knowledge and storytelling prompts weren't part of the SFT/GRPO training data, which was math CoT only, so that behavior comes straight from pretraining across all three checkpoints.

Full sample transcripts for all three models are in evals/results.json.

References

Everything here builds on one foundation, then branches into architecture, data, and the RLVR reasoning recipe:

Attention Is All You Need (Vaswani et al., 2017)
         │
         ├── RoFormer (Su et al., 2021)              → RoPE
         ├── RMSNorm (Zhang & Sennrich, 2019)         → RMSNorm (+ QK-norm)
         ├── GLU Variants Improve Transformer (Shazeer, 2020) → SwiGLU FFN
         ├── GQA (Ainslie et al., 2023)                → grouped-query attention
         └── Qwen3 Technical Report (Qwen Team, 2025)  → this specific combination
                                                          (RoPE+RMSNorm+QK-norm+SwiGLU+GQA)

Architecture (rune/model.py):

• Vaswani et al., 2017. Attention Is All You Need.
• Su et al., 2021. RoFormer: Enhanced Transformer with Rotary Position Embedding.
• Zhang & Sennrich, 2019. Root Mean Square Layer Normalization.
• Shazeer, 2020. GLU Variants Improve Transformer.
• Ainslie et al., 2023. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints.
• Qwen Team, 2025. Qwen3 Technical Report — the specific RoPE+RMSNorm+QK-norm+SwiGLU+GQA combination rune/model.py is adapted from.

Pretraining data & sizing (rune/data.py, scripts/train.py):

• Radford et al., 2019. Language Models are Unsupervised Multitask Learners — GPT-2, source of the tiktoken vocabulary used throughout.
• Penedo et al., 2024. The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale — FineWeb-Edu, the pretraining corpus.
• Hoffmann et al., 2022. Training Compute-Optimal Large Language Models ("Chinchilla") — informed the params-vs-tokens sizing given the compute budget.

Reasoning stage (rune/verifier.py, scripts/sft.py, scripts/grpo.py):

• Hendrycks et al., 2021. Measuring Mathematical Problem Solving With the MATH Dataset — the MATH / MATH-500 problems and answer format the verifier grades against.
• Cobbe et al., 2021. Training Verifiers to Solve Math Word Problems — GSM8K, and the verifier-reward pattern behind RLVR.
• Schulman et al., 2017. Proximal Policy Optimization Algorithms — the clipped objective GRPO's loss is built on.
• Shao et al., 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models — introduces GRPO.
• DeepSeek-AI, 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — the RLVR + KL-to-reference recipe scripts/grpo.py follows.

Code References — not papers, but the actual base this project's code was reffered from

• Sebastian Raschka. Build a Large Language Model (From Scratch).
• Sebastian Raschka. Build a Reasoning Model (From Scratch).