Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .agents/skills/add-construct/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,9 @@ step before starting the next one.
and its name to `GOLDEN_NAMES`. Run `uv run pytest --update-golden` and
read the new file in `tests/golden/cases/`: it should look like Array
API code a person would write. If values are numeric, add a Hypothesis
test in `tests/differential/test_differential.py`. Add unit tests for any
new lowering or optimizer branch so coverage stays at or above 95%.
test in `tests/differential/test_scalar_equivalence.py`. Add unit
tests for any new lowering or optimizer branch so coverage stays at
or above 95%.

6. **Fuzz it.** If the construct is expression-level, add it to the grammar
lists in `src/array_vectorize/fuzz.py` (`_UNARY_MATH`, `_BINOPS`, ...).
Expand Down
4 changes: 2 additions & 2 deletions .claude/hooks/format_python.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@
"""

import json
import pathlib
import subprocess
import sys
from pathlib import Path

path = json.load(sys.stdin).get("tool_input", {}).get("file_path", "")
if path.endswith((".py", ".pyi")) and Path(path).is_file():
if path.endswith((".py", ".pyi")) and pathlib.Path(path).is_file():
ruff = ["uv", "run", "--frozen", "ruff"]
subprocess.run([*ruff, "check", "--select", "I", "--fix", "--force-exclude", "-q", path])
subprocess.run([*ruff, "format", "--force-exclude", "-q", path])
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,30 @@ jobs:
run: uv sync --locked --group docs
- name: Build docs
run: make docs

# PR commit subjects follow Conventional Commits, the same rule the
# local commit-msg hook enforces (compilerla/conventional-pre-commit,
# pinned to the same tag); the full history is needed to walk the
# PR's commit range
commits:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: astral-sh/setup-uv@v10.2.0
- name: Check commit subjects
run: |
base="origin/${{ github.base_ref }}"
status=0
for sha in $(git rev-list --reverse --no-merges "$base"..HEAD); do
git log -1 --format=%s "$sha" > /tmp/commit-subject.txt
if ! uvx --from git+https://github.com/compilerla/conventional-pre-commit@v4.4.0 \
conventional-pre-commit feat fix docs style refactor perf test build ci chore revert \
/tmp/commit-subject.txt; then
echo "::error::$(git log -1 --oneline "$sha") is not a Conventional Commit"
status=1
fi
done
exit $status
18 changes: 18 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
# default_stages keeps the file-filtered hooks above from running at the
# commit-msg stage, where pre-commit would otherwise feed them
# .git/COMMIT_EDITMSG; default_install_hook_types makes
# `pre-commit install` install both hook types.
default_install_hook_types: [pre-commit, commit-msg]
default_stages: [pre-commit]
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
Expand All @@ -12,6 +18,12 @@ repos:
# never disagree on the ruff version
- repo: local
hooks:
- id: cleanporter-fix
name: cleanporter fix
entry: uv run --frozen cleanporter --fix --whole-project
language: system
types_or: [python, pyi]
require_serial: true
- id: ruff-check
name: ruff check
entry: uv run --frozen ruff check --fix --force-exclude
Expand All @@ -30,3 +42,9 @@ repos:
language: system
files: ^(pyproject\.toml|uv\.lock)$
pass_filenames: false
- repo: https://github.com/compilerla/conventional-pre-commit
rev: v4.4.0
hooks:
- id: conventional-pre-commit
stages: [commit-msg]
args: [feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert]
16 changes: 14 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,11 @@ Invariants:
`uv run pytest --update-golden` only when the generated code is meant to
change, then review `git diff tests/golden/cases` line by line.

The code follows the Google Python Style Guide, single deviation:
100-column lines. Imports follow Google §2.2 (import modules, not
names), enforced by cleanporter in `make lint` and `make fmt`; ruff
enforces the rest via `make lint` / pre-commit.

## Tests

The suite is organized by level and topic. Put each test in the file for
Expand All @@ -57,6 +62,7 @@ the behavior, not the bug that found it).
| end-to-end semantics on NumPy | `tests/behavior/test_<topic>.py` |
| a public option (`fallback`, `protect_domains`, `verify`, `namespace`) | `tests/features/` |
| `array-api-strict`, jax, torch | `tests/backends/` |
| benchmarks and the perf gate | `tests/perf/` |
| a new corpus function: its generated source | `tests/corpus.py` + `GOLDEN_NAMES`, then `--update-golden` |
| a new corpus function: values against scalar Python | `tests/differential/` (Hypothesis) |

Expand Down Expand Up @@ -95,5 +101,11 @@ then link it for Claude:
divergences, and API. `README.md` only links to it. A user-visible
change updates the matching page in the same commit; moving modules
updates `docs/architecture.md`.
- Commit subjects are `area: summary` in lowercase, e.g. `codegen: ...`,
`lower: ...`, `tests: ...`, `docs: ...`, `ci: ...`, `build: ...`.
- Commit subjects follow [Conventional Commits 1.0.0]
(https://conventionalcommits.org): `type(scope): summary`, lowercase.
Types: `build`, `chore`, `ci`, `docs`, `feat`, `fix`, `perf`,
`refactor`, `revert`, `style`, `test`. The scope is optional but
recommended, from the package areas: `codegen`, `lower`, `optimize`,
`frontend`, `ir`, `runtime`, `api`, `pipeline`, `fuzz`, `agents`.
Mark breaking changes with `!` after the type/scope, or with a
`BREAKING CHANGE` footer.
16 changes: 14 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,25 @@ install:
$(UV) sync
$(RUN) pre-commit install

# fixers run to completion whatever findings remain; `make lint` stays
# the gate. cleanporter gets `test $$? -le 1`: it exits 1 whenever
# anything remains it will not rewrite (no exit-zero flag), but exit 2
# is an operational error and must fail the recipe. ruff check --fix
# gets --exit-zero: ruff cannot fix E501 itself, and cleanporter's
# module prefixes push lines past 100 columns; ruff format runs first
# and last so the fixes land on wrapped code
fmt:
$(RUN) ruff check --fix $(PY_PATHS)
$(RUN) cleanporter --fix $(PY_PATHS); test $$? -le 1
$(RUN) ruff format $(PY_PATHS)
$(RUN) ruff check --fix --exit-zero $(PY_PATHS)
$(RUN) ruff format $(PY_PATHS)

lint:
$(RUN) ruff check $(PY_PATHS)
$(RUN) ruff format --check $(PY_PATHS)
# the import gate: exits 0 only when clean -- treat_unresolved_as_error
# makes unclassifiable imports fail too
$(RUN) cleanporter $(PY_PATHS)

type:
$(RUN) mypy
Expand Down Expand Up @@ -72,7 +84,7 @@ fuzz:
done

bench:
$(RUN) pytest tests/test_bench.py -m slow --benchmark-enable \
$(RUN) pytest tests/perf/test_bench.py -m slow --benchmark-enable \
--benchmark-only --benchmark-columns=min,median,ops \
--benchmark-group-by=func

Expand Down
15 changes: 8 additions & 7 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,12 @@ identity-stable.
Tests are organized by level and behavior, never by discovery date:

- `tests/unit/` — white-box tests, one file per source module
(frontend, ir, lower, optimize, codegen, runtime).
(frontend, ir, lower, optimize, codegen, runtime), plus the fuzz
smoke test (`test_fuzz.py`).
- `tests/behavior/` — black-box end-to-end semantics on NumPy:
arithmetic dtype exactness, control flow, loops, calls/math/casts,
closures and helpers, documented divergences.
closures and helpers, inspectability (`test_inspectability.py`:
source/`getsource` attributes), documented divergences.
- `tests/features/` — public-option behaviors: fallback, protected
domains, differential verification.
- `tests/backends/` — array-api-strict compatibility.
Expand All @@ -138,11 +140,10 @@ Tests are organized by level and behavior, never by discovery date:
(`0, ±1, subnormals, ±inf, NaN`).
- `tests/golden/` — generated-source snapshots (`cases/`) compared
via `ast.dump` equality, immune to formatting drift.
- Top level — `tests/test_inspectability.py` (source/`getsource`
attributes), `tests/test_perf.py` (performance gate: within 5x of
the hand-written array expression and >= 10x faster than
`np.vectorize`), `tests/test_bench.py` (detailed timings, run
`make bench`), and `tests/test_fuzz_smoke.py`.
- `tests/perf/` — the performance gate (`test_speed_gate.py`: within
5x of the hand-written array expression and >= 10x faster than
`np.vectorize`) and detailed timings (`test_bench.py`, run
`make bench`).

The regression corpus from the 26-round independent adversarial
review is preserved inside `behavior/` and `features/` (provenance in
Expand Down
70 changes: 65 additions & 5 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ dependency at its floor (`--resolution lowest-direct`) on Python 3.12.
## Commands

```bash
make check # ruff + mypy (strict) + pytest + coverage (gate: 95%)
make check # cleanporter + ruff + mypy (strict) + pytest + coverage (gate: 95%)
make check PY=3.13 # any target on another Python (env: .venv-3.13)
make lint type test # the same gates without coverage
make lowest # the test suite at the minimum dependency versions
make fmt # format + autofix
make fmt # cleanporter --fix, then ruff format / check --fix / format
make smoke # run scripts/smoke.py, examples/demo.py, the notebooks
make fuzz # grammar fuzzer on the CI seeds
make bench # pytest-benchmark suite (timings + dispatch gates)
Expand All @@ -43,6 +43,10 @@ The default `make check` also runs the slow-marked performance gate;
plain `pytest` skips nothing but keeps benchmarks disabled
(`--benchmark-disable` is in `addopts`).

`make lint` includes the cleanporter import gate, and `make fmt` runs the
same check with `--fix`: cleanporter rewrites first, so ruff formats the
already-wrapped lines (its module prefixes push lines past 100 columns).

## Testing layers

| Layer | Where |
Expand All @@ -53,10 +57,10 @@ plain `pytest` skips nothing but keeps benchmarks disabled
| backends (array-api-strict) | `tests/backends/` |
| differential (Hypothesis, edge values) | `tests/differential/` |
| golden source snapshots | `tests/golden/` (`cases/`, ast.dump equality) |
| inspectability (getsource, linecache) | `tests/test_inspectability.py` |
| inspectability (getsource, linecache, memoization, docstrings) | `tests/behavior/test_inspectability.py` etc. |
| grammar fuzzer | `src/array_vectorize/fuzz.py` + CI seeds |
| fuzz smoke | `tests/test_fuzz_smoke.py` |
| performance gate + benchmarks | `tests/test_perf.py`, `tests/test_bench.py` |
| fuzz smoke | `tests/unit/test_fuzz.py` |
| performance gate + benchmarks | `tests/perf/` |

The historical review-round regression tests are dissolved into
`behavior/` and `features/` (provenance in git history).
Expand All @@ -65,6 +69,36 @@ Warnings are errors in the test suite, except numpy's data-dependent
numeric warnings (`overflow encountered`, `invalid value`,
`divide by zero`) which are expected on documented divergence paths.

## Style

The code follows the [Google Python Style Guide]
(https://google.github.io/styleguide/pyguide.html). Ruff enforces it in
`make lint`: pydocstyle with the google convention, pep8-naming, import
conventions (`numpy` as `np`), absolute imports only (TID252,
`ban-relative-imports = "all"`), and additionally BLE001, PLW0603, G004,
PGH003 and PGH004 (evaluated: zero violations).

Imports follow Google §2.2 (import modules, not names), enforced by
[cleanporter](https://github.com/grAItools/cleanporter) in `make lint` and
`make fmt` (scope `all`; the golden cases and the notebooks are excluded
as generated and marimo-owned, `tests/corpus.py` is skipped as golden
test data, and `treat_unresolved_as_error` is on, so an import
cleanporter cannot classify fails the gate). The deliberate exceptions
are inline `# cleanporter: ignore[...]` suppressions, each naming its
finding code and carrying its reason in the suppression comment or the
comment block above it. One import
per line per §3.13 (ruff isort `force-single-line`).

Evaluated and deliberately skipped: TD (conflicts with Google's current
TODO format) and PLC0415 (import-inside-function is a deliberate pattern
in this compiler's test subjects).

The only deviation from the guide is the line length: 100 columns (the
guide says 80). Tests relax D1 (missing docstring) — test names are
self-documenting; `tests/corpus.py` ignores all D rules (golden-pinned
test data), and `tests/behavior/test_docstrings.py` ignores D4 (it holds
the NumPy-style docstring subject).

## CI

- **check** — the full `make check` gate on Python 3.12–3.14
Expand Down Expand Up @@ -106,6 +140,32 @@ git push origin main --tags
or the lock is stale; tagging stays a manual step so a re-run of the
target can never move a tag.

## Commit messages

Commit subjects follow [Conventional Commits 1.0.0]
(https://conventionalcommits.org): `type(scope): summary`, lowercase.
Types: `build`, `chore`, `ci`, `docs`, `feat`, `fix`, `perf`, `refactor`,
`revert`, `style`, `test`. The scope is optional but recommended, from
the package areas: `codegen`, `lower`, `optimize`, `frontend`, `ir`,
`runtime`, `api`, `pipeline`, `fuzz`, `agents`. Mark breaking changes
with `!` after the type/scope, or with a `BREAKING CHANGE` footer. A
`commit-msg` hook (`conventional-pre-commit`) enforces the format.

The history before the switch used `area: summary` subjects; map them
to the new format like this:

| Old subject | Conventional equivalent |
|---|---|
| `codegen: x` | `feat(codegen): x` or `fix(codegen): x` |
| `agents: x` | `chore(agents): x` |
| `tests: x` | `test: x` (type, no scope) |
| `docs: x` | `docs: x` (type, no scope) |
| `ci: x` | `ci: x` (type, no scope) |
| `build: x` | `build: x` (type, no scope) |

Existing clones must re-run `make install` once to pick up the new
commit-msg hook.

## Coding agents

Instructions and skills for coding agents are harness-agnostic; each
Expand Down
12 changes: 8 additions & 4 deletions examples/demo.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,35 @@

import numpy as np

from array_vectorize import vectorize
import array_vectorize


@vectorize
@array_vectorize.vectorize
def sigmoid(x):
"""Numerically stable logistic sigmoid."""
if x >= 0:
return 1.0 / (1.0 + math.exp(-x))
z = math.exp(x)
return z / (1.0 + z)


@vectorize
@array_vectorize.vectorize
def gelu(x):
"""GELU activation (tanh approximation)."""
return 0.5 * x * (1.0 + math.tanh(0.7978845608028654 * (x + 0.044715 * x * x * x)))


@vectorize
@array_vectorize.vectorize
def l2_dist(x, y):
"""Distance accumulated in a constant-trip loop, then sqrt."""
d = 0.0
for i in range(3):
d = d + (x - y) * (x - y)
return math.sqrt(d)


def main() -> None:
"""Run the demo on NumPy and array-api-strict inputs."""
x = np.asarray([-3.0, -1.0, 0.0, 1.0, 3.0])
y = np.asarray([2.0, 0.5, -0.5, 0.5, 2.0])

Expand Down
Loading
Loading