diff --git a/AGENTS.md b/AGENTS.md index 45a6c78b3..e70be4d8a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,6 +9,16 @@ examples, build or CI workflows, benchmark methodology, or documented limitations. Keep entries concise and outcome-focused; do not add release notes for internal cleanup that has no visible effect. +Write user documentation as a concise guide to the current product. Lead with +the task a user wants to complete, show the necessary command or example, and +state only the behavior, choices, and limitations needed to use it correctly. +Do not narrate implementation history, prior bugs, rejected designs, internal +mechanics, defensive checks, or why a newly added behavior differs from an old +one unless that context changes what the user must do. Integrate changes into +the existing workflow instead of appending a change report, and remove any +sentence whose only purpose is to justify the implementation or record the +development process. + Treat developer documentation as durable guides, not as per-change implementation logs. Do not update developer pages merely because code changed, and do not add incidental low-level details that are unnecessary for following diff --git a/CHANGELOG.md b/CHANGELOG.md index decd6f3f0..01ca9c684 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,69 @@ release tags add a leading `v` to the package version. ## Unreleased +- Generated contracts now represent a one-level primitive C pointer as + runtime-rank `T[...]` NumPy storage instead of choosing a scalar temporary. + It accepts ranks 0 through 15 with any strides, so a Fortran-ordered array + or a strided slice reaches the native call unchanged, and it can be narrowed + to contiguous, rank-zero, fixed-rank, or scalar-address storage in an edited + contract. `Arg(i).size` supplies the total element count to a native + parameter, alongside the existing `Arg(i).shape[d]` and `Arg(i).strides[d]` + layout projections; an axis projection against storage that has no such axis + now raises `TypeError` instead of reading past the actual's shape. + +- Getting Started now offers complete Fortran and C paths for toolchain + verification, building the same first function, and the edit-review-build-test + loop. Fortran modules now begin in their task-focused User Guide page instead + of a separate mandatory beginner step. + +- Added a dedicated C section to the User Guide for scalar functions, pointer + contracts, arrays and strings, outputs and errors, and native symbols and + dependencies. The C Support page now serves as a concise capability and + boundary map with the same wrapper-area, boundary, and source-entry structure + as Fortran Support. User Guide navigation presents separate Fortran and C + paths followed by their shared build workflows. + +- An array argument now requires the NumPy storage of the C element type its + source declares, rather than the canonical storage of the same width. A + target's `int64_t` may be `long` or `long long`, and NumPy independently + gives `NPY_INT64` to whichever of the two is 64 bits, so those two choices + could disagree: a `long long *` buffer asked for `numpy.longlong` on one + target and `numpy.int64` on another. One C source now keeps one accepted + dtype everywhere. + +- A scalar argument whose native parameter is a 64-bit C integer now accepts + either NumPy spelling of that width and converts it, so `np.int64` and + `np.longlong` are both valid for a `long long` or `long` parameter whichever + one the target calls `int64_t`. Array arguments are unchanged: an element + buffer cannot be converted, so it still requires the exact native dtype. + +- A cell magic that reads a dash-prefixed flag value as another option now + names the equals form and, for the flag groups, the quoted-group form. + +- Added optional `%%fortran`, `%%c`, and `%%pyi` IPython/Jupyter cell magics. + Native-source cells compile directly or, with `--pyi`, persist their exact + source and insert editable per-module or direct-declaration contract cells. + Executing the generated `%%pyi` cell recovers the source language from its + digest, builds against that cached source, and publishes declared Fortran + modules or standalone declarations directly in the notebook namespace. + Exact cells reuse a persistent SHA-256 build cache unless `--force` is + selected, and PRIK does not expose an internal package entry. Wrapped + functions follow the published notebook path (`maths.square` or standalone + `square`) instead of exposing the private cache extension name; ordinary + file builds retain their user-selected package root, such as + `geometry.maths.square`. Existing notebook build artifacts are rebuilt once + so cached extensions cannot retain the old private function identity. + Multi-module `--pyi` cells are presented sequentially in terminal IPython, + whose next-input prompt can hold only one editable contract, while Jupyter + frontends continue to receive every generated module cell immediately. All + cells in one generated contract bundle retain the source cell's effective + compiler and build flags; changing that configuration requires regenerating + the bundle and is rejected before compiler execution. Independently authored + `%%pyi` cells can instead name one or more existing implementation files with + `--native-fortran-sources` or `--native-c-sources`; each cell builds and + publishes only its own contract module, and native file-content changes + invalidate its persistent cache. + - A one-character `@native_call` literal is now buildable: `String[1]("N")` declares the character a native parameter receives instead of leaving it a visible Python argument. It crosses the boundary as an interoperable `char`, diff --git a/README.md b/README.md index 44dddaa31..7d1d49663 100644 --- a/README.md +++ b/README.md @@ -23,9 +23,10 @@ public APIs may still change before `1.0`. PRIK supports both languages. Fortran currently has the broader, more mature wrapper surface. C currently supports a focused wrapper subset: primitive values, one-level pointers, NumPy arrays, and strings. In both languages, editable -`.pyi` contracts let you shape the Python API. See [C +`.pyi` contracts let you shape the Python API. See the [C User +Guide](https://pynumlab.github.io/prik/user/guide/c/) for C workflows and [C Support](https://pynumlab.github.io/prik/user/language-support/c-support/) for -C examples and current limits. +current coverage. [Read the documentation](https://pynumlab.github.io/prik/) for installation, the user guide, examples, and reference material. @@ -40,6 +41,7 @@ the user guide, examples, and reference material. - [C support](#c-support) - [Current C limitations](#current-c-limitations) - [Installation & Quick Start](#installation--quick-start) +- [IPython and Jupyter](#ipython-and-jupyter) - [How it works](#how-it-works) - [Python API](#python-api) - [Development](#development) @@ -183,7 +185,8 @@ compiler matrix; each project guide also records its own tested platforms. ## Key Features - **Native APIs that feel like Python.** Fortran modules become Python - namespaces, while derived types become classes with fields and methods. + namespaces, derived types become classes, and C functions expose designed + scalar, array, string, and output interfaces. - **First-class NumPy array interop.** Pass ordinary NumPy arrays to native procedures, including multidimensional and in-place data, with generated dtype, shape, layout, and mutability handling at the language boundary. @@ -194,6 +197,9 @@ compiler matrix; each project guide also records its own tested platforms. - **Editable contracts for reshaping APIs.** Edit the generated `.pyi` contract to rename, hide, reorganize, or overload the public interface, backed by readable generated docstrings. +- **Interactive notebook builds.** Compile Fortran and C cells with + `%%fortran` and `%%c`, or edit a generated semantic contract in a `%%pyi` + cell. - **Unsupported contracts fail before the build.** PRIK identifies the exact boundary and reason before attempting code generation or compilation. @@ -241,7 +247,7 @@ code generation with a diagnostic naming the boundary and the reason. The [language feature matrix](https://pynumlab.github.io/prik/user/language-support/feature-matrix/) records the full support status of every feature with its evidence. The -[C support guide](https://pynumlab.github.io/prik/user/language-support/c-support/) +[C Support page](https://pynumlab.github.io/prik/user/language-support/c-support/) states the current C wrapper boundary. ## C support @@ -334,8 +340,8 @@ structs, unions, function pointers, or callbacks. Unsupported declarations stop before wrapper generation or compilation; parsing a declaration alone does not promise that it can be built. -[Read the C support guide for executable source, `.pyi`, CLI, and Python API -examples.](https://pynumlab.github.io/prik/user/language-support/c-support/) +[Continue with the C User Guide for source, `.pyi`, CLI, and Python API +workflows.](https://pynumlab.github.io/prik/user/guide/c/) ## Installation & Quick Start @@ -435,6 +441,24 @@ The custom wrapper flags appear in the relevant command lines: -shared ... -O2 ... geometry_debug ... ``` +## IPython and Jupyter + +Install the optional notebook integration and load it once per session: + +```bash +python3 -m pip install "prik[jupyter]" +``` + +```ipython +%load_ext prik.jupyter +``` + +Use `%%fortran` or `%%c` to compile native source in a cell. Add `--pyi` to +review and edit the generated contract before compilation, or use `%%pyi` with +existing native source files. See [IPython and Jupyter +Notebooks](https://pynumlab.github.io/prik/user/guide/notebooks/) for the +complete workflow. + ## How it works ```text @@ -470,7 +494,8 @@ print(result.shared_library) Use `build_c_extension("api.c", output_dir="build")` for a C source build, or `build_pyi_extension(..., native_language="c", native_c_sources=[...])` -for an authored C contract. The C support guide shows complete examples. +for an authored C contract. The [C User +Guide](https://pynumlab.github.io/prik/user/guide/c/) shows complete examples. ## Development @@ -513,11 +538,13 @@ notice when redistributed. - **[Documentation](https://pynumlab.github.io/prik/)** — Learn how to install and use PRIK - **[Project Vision](https://github.com/PyNumLab/prik/wiki)** — Long-term direction for PRIK's semantic interoperability model -- **[Getting Started](https://pynumlab.github.io/prik/user/getting-started/)** — Installation, verification, standalone procedures, modules, and rebuild workflow -- **[User Guide](https://pynumlab.github.io/prik/user/guide/)** — Data types, functions, modules, arrays, derived types, callbacks, ownership, and runtime behavior +- **[Getting Started](https://pynumlab.github.io/prik/user/getting-started/)** — Installation, verification, matched Fortran and C first functions, and rebuild workflow +- **[User Guide](https://pynumlab.github.io/prik/user/guide/)** — Separate Fortran and C paths followed by shared build workflows +- **[C User Guide](https://pynumlab.github.io/prik/user/guide/c/)** — C functions, pointer contracts, arrays, strings, outputs, errors, symbols, headers, and dependencies +- **[IPython and Jupyter](https://pynumlab.github.io/prik/user/guide/notebooks/)** — Compile native cells and edit semantic contracts interactively - **[`.pyi` Format](https://pynumlab.github.io/prik/user/reference/pyi-format/)** — Contract projects, declarations, decorators, types, storage, metadata, and C and Fortran forms - **[Editing `.pyi` Contracts](https://pynumlab.github.io/prik/user/reference/pyi-contracts/)** — Supported recipes for reshaping the generated Python API -- **[C Support](https://pynumlab.github.io/prik/user/language-support/c-support/)** — C ABI scope, contracts, CLI, Python API, and executable examples +- **[C Support](https://pynumlab.github.io/prik/user/language-support/c-support/)** — Supported C wrapper areas, boundaries, source inputs, and public entry points - **[CLI Reference](https://pynumlab.github.io/prik/user/reference/cli-commands/)** — Every command, option, and checked workflow - **[Language Support](https://pynumlab.github.io/prik/user/language-support/)** — Supported, partially supported, and unsupported native-language features - **[FAQ](https://pynumlab.github.io/prik/user/faq/)** — Concise answers to common questions diff --git a/docs/developer/codebase-map.md b/docs/developer/codebase-map.md index 57a6f76f5..e5f5fd4d2 100644 --- a/docs/developer/codebase-map.md +++ b/docs/developer/codebase-map.md @@ -22,6 +22,7 @@ to its documentation and evidence. | --- | --- | | `prik/__init__.py` | Public build entry points and version. | | `prik/cli.py` | CLI argument validation, stage selection, and output routing. | +| `prik/jupyter/` | Optional IPython cell-magic parsing, editable-contract cell insertion, persistent source/build caching, extension loading, and notebook namespace publication. | | `prik/pipeline/build.py` | Source-first and contract-first extension-build orchestration. | | `prik/pipeline/pyi.py` | Semantic `.pyi` loading and external-type reconciliation. | | `prik/pipeline/wrapper.py` | Completed plan to rendered-wrapper orchestration and artifact records. | diff --git a/docs/developer/feature-to-code-map.md b/docs/developer/feature-to-code-map.md index 2abd9169f..3acab407a 100644 --- a/docs/developer/feature-to-code-map.md +++ b/docs/developer/feature-to-code-map.md @@ -38,8 +38,8 @@ change crosses a stage boundary. | Derived objects, allocatables, pointers, and lifetimes | [Derived types](../user/guide/wrapping-derived-types.md), [allocatables](../user/guide/allocatables.md), [pointers](../user/guide/pointers.md), [memory management](../user/guide/memory-management.md) | `prik/policy/ownership.py` → `prik/policy/construction.py` → `prik/policy/native_array_handles.py` → `prik/planning/planner.py` → `prik/runtime/handles.py` | `tests/fortran/derived_types/`, `tests/fortran/allocatables/`, `tests/fortran/pointers/` | | Callbacks | [Callbacks](../user/guide/callbacks.md) | `prik/policy/models.py` → `prik/policy/completion.py` → `prik/planning/planner.py` → `prik/codegen/c/binding.py` and `prik/codegen/fortran/bridge.py` | `tests/fortran/callbacks/` | | Projected errors | [Error handling](../user/guide/error-handling.md) | `prik/policy/models.py` → `prik/policy/completion.py` → `prik/planning/planner.py` → `prik/codegen/c/binding.py` and `prik/codegen/fortran/bridge.py` | `tests/fortran/error_handling/` | -| C values, pointers, arrays, strings, outputs, and status | [C Support](../user/language-support/c-support.md) | `prik/semantics/c2ir.py` or `prik/semantics/pyi2ir.py` → `prik/policy/completion.py` → `prik/planning/planner.py` → `prik/codegen/c/binding.py` → `prik/pipeline/build.py` | `tests/c/primitive_scalars/`, `tests/c/primitive_pointers/`, `tests/c/primitive_strings/`, `tests/c/infrastructure/building/` | -| C binding-header symbol collisions | [Collision forwarders](../user/language-support/c-support.md#symbols-your-bindings-own-headers-declare) | `prik/pipeline/build.py` → `prik/planning/planner.py` → `prik/codegen/c/binding.py` → C-only compilation and link | `tests/c/symbol_collisions/codegen/`, `tests/c/symbol_collisions/end_to_end/` | +| C values, pointers, arrays, strings, outputs, and status | [C User Guide](../user/guide/c/index.md) | `prik/semantics/c2ir.py` or `prik/semantics/pyi2ir.py` → `prik/policy/completion.py` → `prik/planning/planner.py` → `prik/codegen/c/binding.py` → `prik/pipeline/build.py` | `tests/c/primitive_scalars/`, `tests/c/primitive_pointers/`, `tests/c/primitive_strings/`, `tests/c/infrastructure/building/` | +| C binding-header symbol collisions | [Collision forwarders](../user/guide/c/symbols-headers-and-dependencies.md#symbols-declared-by-binding-headers) | `prik/pipeline/build.py` → `prik/planning/planner.py` → `prik/codegen/c/binding.py` → C-only compilation and link | `tests/c/symbol_collisions/codegen/`, `tests/c/symbol_collisions/end_to_end/` | | Native compilation, extension runtime, and public build API | [Compiler](packages/compiler.md), [Quality Assurance](workflows/quality-assurance.md) | `prik/__init__.py` → `prik/pipeline/build.py` → `prik/compiler/objects.py` → `prik/compiler/compilers.py` → `prik/compiler/native_support.py` → `prik/runtime/native_support/` | `tests/fortran/infrastructure/building/end_to_end/test_runtime_compatibility.py`, `tests/fortran/infrastructure/parsing/test_public_entrypoints.py` | Each change route begins with the first owner for a capability; it is not a diff --git a/docs/developer/packages/codegen/c-binding.md b/docs/developer/packages/codegen/c-binding.md index efd3c863e..6365e081c 100644 --- a/docs/developer/packages/codegen/c-binding.md +++ b/docs/developer/packages/codegen/c-binding.md @@ -60,6 +60,14 @@ ModulePlan.binding + ModulePlan.entrypoint `require_supported()` checks that the already selected primitive spellings are available. It is capability preflight, not a second policy pass. +`BindingModulePlan` also supplies the public package root used for Python +function metadata. The binding combines that completed root with each planned +namespace path; it does not infer notebook execution or reuse the private +extension-loading name as presentation. An empty public root means the caller +is publishing directly into an interactive namespace. Generated filenames, +the `PyInit_*` symbol, and native helper symbols continue to use the plan's +internal owner path. + Numeric scalar boundaries retain their exact NumPy contract without using the generic dtype-conversion path on a successful call. The native support helper checks the planned NumPy scalar class, reads its typed payload directly, and diff --git a/docs/developer/packages/planning.md b/docs/developer/packages/planning.md index 439a9fb7f..3126bdc83 100644 --- a/docs/developer/packages/planning.md +++ b/docs/developer/packages/planning.md @@ -93,10 +93,12 @@ whether the binding or bridge implements the callable; the opposite side uses the same record as its declaration/call contract. Static CPython helpers and bridge-internal procedures are deliberately absent. -`BindingModulePlan` separately records which derived-type owners need -binding-local capsule and holder surfaces. Those static CPython helpers are not -entrypoints, but their membership is still planned rather than rediscovered by -C lowering. `BridgeModulePlan` likewise records the broad typed-holder +`BindingModulePlan` records the public Python package root separately from the +internal module owner used for generated symbols and extension loading. It also +records which derived-type owners need binding-local capsule and holder +surfaces. Those static CPython helpers are not entrypoints, but their membership +is still planned rather than rediscovered by C lowering. `BridgeModulePlan` +likewise records the broad typed-holder definitions required by adapter calls and the narrower holder field-support inventories. Planning derives both backend-local inventories and the external support-procedure registry together. Validation requires every planned local diff --git a/docs/index.md b/docs/index.md index dc3805be5..ffbddd036 100644 --- a/docs/index.md +++ b/docs/index.md @@ -25,7 +25,8 @@ PRIK supports both languages. Fortran currently has the broader, more mature wrapper surface. C currently supports a focused wrapper subset: primitive values, one-level pointers, NumPy arrays, and strings. In both languages, editable `.pyi` contracts let you shape the Python API. See [C -Support](user/language-support/c-support.md) for C examples and current limits. +User Guide](user/guide/c/index.md) for C workflows and [C +Support](user/language-support/c-support.md) for current coverage. --- @@ -103,7 +104,7 @@ print(native_math.add(np.float64(3.0), np.float64(2.5))) # 5.5 ``` This source build also writes an editable contract. For C pointers, arrays, -and authored contracts, see [C Support](user/language-support/c-support.md). +and authored contracts, see the [C User Guide](user/guide/c/index.md). ## Shape the Python API @@ -324,7 +325,7 @@ values below `1.0×` favor f2py. **Wrapping a supported C API?** -[Read C Support →](user/language-support/c-support.md){ .prik-primary-cta } +[Read the C User Guide →](user/guide/c/index.md){ .prik-primary-cta } **Working on PRIK itself?** diff --git a/docs/user/about.md b/docs/user/about.md index a5a023a5c..f3a2ad6a2 100644 --- a/docs/user/about.md +++ b/docs/user/about.md @@ -16,7 +16,7 @@ native libraries through natural Python APIs. PRIK supports **Fortran-to-Python and C-to-Python interoperability**, with a focus on native behavior that traditional wrapper generators often cannot represent reliably. In both languages, editable `.pyi` contracts let you shape -the Python API. The [C support guide](language-support/c-support.md) describes +the Python API. The [C Support](language-support/c-support.md) page describes its current coverage. PRIK is for Python users who need native numerical, scientific, or systems code without having to design and maintain the entire language boundary themselves. diff --git a/docs/user/examples/c/libm-wrapper.md b/docs/user/examples/c/libm-wrapper.md index 8b60c227d..9c3aeb797 100644 --- a/docs/user/examples/c/libm-wrapper.md +++ b/docs/user/examples/c/libm-wrapper.md @@ -31,9 +31,9 @@ coverage audits, numerical tests, documentation, and CI execution. ordinary NumPy types in the public Python signature. - Test every exported function and audit the inventory against the built module. -Read [C support](../../language-support/c-support.md) and the -[CLI reference](../../reference/cli-commands.md) first if the C workflow is -new to you. For a maintained C example built around NumPy arrays and an edited +Read the [C User Guide](../../guide/c/index.md) and [CLI +reference](../../reference/cli-commands.md) first if the C workflow is new to +you. For a maintained C example built around NumPy arrays and an edited semantic contract, see [TA-Lib](ta-lib-wrapper.md). --- diff --git a/docs/user/examples/index.md b/docs/user/examples/index.md index f5cb0ad06..2b9701a39 100644 --- a/docs/user/examples/index.md +++ b/docs/user/examples/index.md @@ -2,7 +2,7 @@ title: Examples Gallery audience: users prerequisites: getting started -related: ../guide/building-shared-library.md, ../language-support/c-support.md, ../reference/cli-commands.md, ../reference/python-api.md +related: ../guide/building-shared-library.md, ../guide/c/index.md, ../language-support/c-support.md, ../reference/cli-commands.md, ../reference/python-api.md status: maintained publication: reviewed --- @@ -73,10 +73,10 @@ library's implementation sources as part of the wrapper build. | Wrap 60 target-generated ISO C99 math routines from a system library | [libm wrapper](c/libm-wrapper.md) | | Wrap and reference-check all 322 TA-Lib double and float-input indicators | [TA-Lib wrapper](c/ta-lib-wrapper.md) | -For smaller introductory workflows, start with [First Wrapped -Function](../getting-started/first-wrapped-function.md), [First Wrapped -Module](../getting-started/first-wrapped-module.md), or the [C support -guide](../language-support/c-support.md). +For a smaller introductory workflow, start with the dual-language [First +Wrapped Function](../getting-started/first-wrapped-function.md). Continue with +[Wrapping Modules](../guide/wrapping-modules.md) for Fortran or the [C User +Guide](../guide/c/index.md) for C. To learn how an edited `.pyi` contract can reshape a low-level library, follow the [Pythonic BLAS API tutorial](../tutorials/pythonic-blas.md). diff --git a/docs/user/faq/index.md b/docs/user/faq/index.md index 3469f0259..921d4dffd 100644 --- a/docs/user/faq/index.md +++ b/docs/user/faq/index.md @@ -30,7 +30,7 @@ for a complete source-to-result example. Pass the module source to PRIK. It generates the extension and exposes supported public procedures and module state through Python. Start with -[Generate Python Bindings for a Fortran Module](../getting-started/first-wrapped-module.md). +[Generate Python Bindings for a Fortran Module](../guide/wrapping-modules.md). @@ -77,7 +77,7 @@ silently copied. See [Pass NumPy Arrays to Fortran](../guide/arrays.md). Build a supported C source with PRIK and import the generated extension. A primitive function can be wrapped directly from source; pointers, arrays, strings, hidden outputs, and status handling use an editable semantic `.pyi` -contract. Follow [Build a Scalar C Function](../language-support/c-support.md#build-a-scalar-c-function) +contract. Follow [Build a Scalar C Function](../guide/c/functions-and-scalars.md#build-a-scalar-c-function) for the complete first example. @@ -89,18 +89,19 @@ C pointer syntax does not say whether a pointer represents one value, an output, an array, or caller-owned storage. PRIK generates a conservative starter contract instead of guessing. Edit it to state the intended storage, shape, and result projection, then build it against the C implementation. See -[Author a Contract for Pointers and Arrays](../language-support/c-support.md#author-a-contract-for-pointers-and-arrays). +[Author a Contract for Pointers and Arrays](../guide/c/pointers-arrays-and-strings.md#author-a-contract-for-pointers-and-arrays).
Which C arrays and strings are supported? -C wrappers support primitive non-Boolean NumPy arrays of ranks 1–15 with -C-contiguous storage. Strings are supported as rank-zero inputs and -caller-owned storage. Arrays of strings, Boolean arrays, native C array -declarators, ranks outside 1–15, and non-C-contiguous arrays are unsupported. -See [C Support: What Is Supported](../language-support/c-support.md#what-is-supported) +C pointer contracts support primitive non-Boolean NumPy storage of ranks 0–15. +Runtime-rank `T[...]` storage accepts any strides; an explicit shape such as +`T[:]` requires C-contiguous storage. Strings are supported as rank-zero +inputs and caller-owned storage. Arrays of strings, Boolean arrays, native C +array declarators, and ranks above 15 are unsupported. +See [C Support: Supported Wrapper Areas](../language-support/c-support.md#supported-wrapper-areas) for the complete boundary.
@@ -148,11 +149,11 @@ Continue with the page that matches the part you need: header](../reference/cli-commands.md#c-include-exposure) explains the `symbols.txt` format, included-header visibility, and selection failures. - [Choose the pointer - contract](../language-support/c-support.md#choose-the-pointer-contract) + contract](../guide/c/pointers-arrays-and-strings.md#choose-the-pointer-contract) explains how to describe one value, an output, an array, or caller-owned storage in `vendor.pyi`. - [Supply native - dependencies](../language-support/c-support.md#native-dependencies) explains + dependencies](../guide/c/symbols-headers-and-dependencies.md#native-dependencies) explains when to use implementation sources, objects, library names, library directories, include paths, and compiler definitions. - [Wrap the system math library](../examples/c/libm-wrapper.md) is the complete @@ -169,8 +170,8 @@ Continue with the page that matches the part you need: PRIK rejects unsupported C forms before wrapper planning or native compilation; parser acceptance alone is not a build promise. The diagnostic -identifies the blocked declaration or contract. Check [Current C -Limits](../language-support/c-support.md#current-limits), the [feature +identifies the blocked declaration or contract. Check [Important C +Boundaries](../language-support/c-support.md#important-boundaries), the [feature matrix](../language-support/feature-matrix.md#unsupported-or-blocked-forms), and [diagnostic codes](../reference/diagnostic-codes.md#c-wrapper-diagnostics). diff --git a/docs/user/getting-started/beginner-workflow.md b/docs/user/getting-started/beginner-workflow.md index 837666d55..176400f28 100644 --- a/docs/user/getting-started/beginner-workflow.md +++ b/docs/user/getting-started/beginner-workflow.md @@ -2,28 +2,28 @@ title: Common Beginner Workflow description: Recommended development loop — edit, review contract, build, test, and rebuild audience: users -prerequisites: first wrapped module -related: ../guide/index.md +prerequisites: first wrapped function +related: ../guide/index.md, ../guide/c/index.md status: maintained publication: reviewed --- # Common Beginner Workflow -Now that you have built a function and a module, use this loop for your own -project: edit the source, review its Python interface, build, and test. +Use this loop for a Fortran or C project: edit the source, review its Python +interface, build, and test. --- ## Recommended Project Layout -This layout continues with `scale.f90` from -[First Wrapped Function](first-wrapped-function.md): +This layout continues with `scale.f90` or `scale.c` from [First Wrapped +Function](first-wrapped-function.md): ``` my-project/ ├── src/ -│ └── scale.f90 +│ └── scale.f90 # or scale.c ├── build/ # ← Generated, do not commit ├── tests/ │ └── test_scale.py @@ -36,21 +36,41 @@ Keep `src/` and `tests/` under version control. Never commit the `build/` folder ## 1. Edit and Review -Edit the Fortran source, then preview the generated Python interface: +Edit the native source, then preview the generated Python interface with the +command for your language. + +Fortran: ```bash python3 -m prik generate --pyi src/scale.f90 ``` +C: + +```bash +python3 -m prik generate --pyi --language c src/scale.c +``` + Check the function names, arguments, result types, and required NumPy dtypes. -This review is especially useful after changing a public Fortran declaration. +This review is especially useful after changing a public declaration. --- ## 2. Build the Extension +Fortran: + ```bash -python3 -m prik src/scale.f90 --out-dir build/scale +python3 -m prik src/scale.f90 --out scale --out-dir build/scale +``` + +C: + +```bash +python3 -m prik --language c src/scale.c \ + --compiler cc \ + --out scale \ + --out-dir build/scale ``` Rerun the same command after source changes. Add `--verbose` only when you need @@ -85,7 +105,14 @@ python3 -m pytest tests/test_scale.py -q ## 4. Optionally Edit the Contract -Save a contract package when you want to change the Python interface: +Save the generated contract when you want to change the Python interface. +Create the optional directory from the layout above first: + +```bash +mkdir -p contracts +``` + +Fortran writes a contract package: ```bash python3 -m prik generate --pyi src/scale.f90 --out contracts/scale @@ -96,17 +123,33 @@ Edit `contracts/scale/scale.pyi`, then build through its package entry: ```bash python3 -m prik contracts/scale/__init__.pyi \ --native-fortran-sources src/scale.f90 \ + --out scale \ + --out-dir build/scale-edited +``` + +C writes one contract file: + +```bash +python3 -m prik generate --pyi --language c src/scale.c \ + --out contracts/scale.pyi +``` + +Edit `contracts/scale.pyi`, then build it with the C implementation: + +```bash +python3 -m prik --language c contracts/scale.pyi \ + --native-c-sources src/scale.c \ + --compiler cc \ + --out scale \ --out-dir build/scale-edited ``` Use this form instead of the source build in step 2 when the edited contract -should control the wrapper. The `.pyi` controls the Python surface; the Fortran -source still supplies the native implementation. Keep its native symbol names, -types, rank, and argument order accurate. +should control the wrapper. The `.pyi` controls the Python surface; the native +source still supplies the implementation. Keep its native symbol names, types, +rank, and argument order accurate. -The User Guide introduces small edits next to the feature they affect, such as -renaming a function, changing array layout, adding an overload, or exposing a -module procedure as a method. +The User Guide introduces small edits next to the feature they affect. Use [`.pyi` Format](../reference/pyi-format.md) to understand the generated project, declarations, and keywords. Use [Editing `.pyi` @@ -126,3 +169,6 @@ when you need to rule out stale build files. ## Next - Continue with the [User Guide](../guide/index.md). +- For Fortran modules, see [Wrapping Modules](../guide/wrapping-modules.md). +- For C pointers, arrays, strings, and outputs, see the [C User + Guide](../guide/c/index.md). diff --git a/docs/user/getting-started/first-wrapped-function.md b/docs/user/getting-started/first-wrapped-function.md index 48f9bfcde..2c3cff6c9 100644 --- a/docs/user/getting-started/first-wrapped-function.md +++ b/docs/user/getting-started/first-wrapped-function.md @@ -1,20 +1,22 @@ --- title: First Wrapped Function -description: Build and call your first Fortran function as a Python extension +description: Build and call the same scalar function from Fortran or C audience: users prerequisites: installation, verification -related: first-wrapped-module.md, ../guide/wrapping-functions.md +related: beginner-workflow.md, ../guide/wrapping-functions.md, ../guide/c/functions-and-scalars.md status: maintained publication: reviewed --- # First Wrapped Function -This example shows how to build a simple scalar Fortran function and call it from Python using the exact NumPy dtypes required by its contract. +Choose the [Fortran](#fortran-path) or [C](#c-path) path below. Both define the +same operation, build an extension named `scale`, and expose `scale.scale` to +Python. After building, continue with [Call the Function](#call-the-function). --- -## Source Code +## Fortran Path Create `scale.f90`: @@ -26,10 +28,6 @@ real(8) function scale(value, factor) result(output) end function scale ``` ---- - -## Inspect the Generated Contract - Preview the Python interface before building: ```bash @@ -49,71 +47,65 @@ def scale( ) -> Float64: ... ``` -`Float64` means the function requires `numpy.float64` scalar arguments and -returns a `numpy.float64` result. -`@standalone` identifies a procedure outside a Fortran module. -`@native_call(...)` maps the two Python arguments to the native call and passes -each scalar by address. - -This file is both the wrapper contract and an editable description of the -Python interface. You can leave it unchanged for this example; later pages -show useful edits in context. [`.pyi` Format](../reference/pyi-format.md) -defines every part of the contract, while [Editing `.pyi` -Contracts](../reference/pyi-contracts/index.md) shows how to change it safely. - ---- - -## Build the Extension - -From the directory containing `scale.f90`, run: +Build the extension: ```bash -python3 -m prik scale.f90 --out-dir build/first-function +python3 -m prik scale.f90 \ + --out scale \ + --out-dir build/first-function ``` -This creates an importable `scale` extension module in the `build/first-function` directory. +`@standalone` records that the procedure is outside a Fortran module, and +`@native_call(...)` records Fortran's by-address scalar arguments. ---- +Continue with [Call the Function](#call-the-function), or read the C path to +compare the native source and generated contract. -## Inspect the Generated Docstring +## C Path -PRIK creates NumPy-style docstrings from the same contract. Import the built -extension and inspect the function: +Create `scale.c`: -```python -import sys +```c +double scale(double value, double factor) { + return value * factor; +} +``` -sys.path.insert(0, "build/first-function") -import scale +Preview the Python interface: -print(scale.scale.__doc__) +```bash +python3 -m prik generate --pyi --language c scale.c ``` -```text -scale(value, factor) -> float64 +The generated semantic `.pyi` contains: -Parameters ----------- -value : float64 -factor : float64 +```python +from prik.contracts import Float64 -Returns -------- -result : float64 +def scale(value: Float64, factor: Float64) -> Float64: ... ``` -`help(scale.scale)` shows the same signature, parameter types, result, and -documented exceptions. Generated modules, classes, methods, and properties -also provide docstrings. The displayed `float64` records the exact NumPy scalar -type accepted and returned by this function. +Build the extension: ---- +```bash +python3 -m prik --language c scale.c \ + --compiler cc \ + --out scale \ + --out-dir build/first-function +``` ## Call the Function +Whichever source path you chose, import and call the extension in the same way: + ```python +import sys + import numpy as np +sys.path.insert(0, "build/first-function") +import scale + result = scale.scale(np.float64(3.0), np.float64(2.5)) print(result) # 7.5 assert result == 7.5 @@ -123,7 +115,8 @@ assert result == 7.5 ## Common Pitfall: Wrong Scalar Type -You **must** pass the exact NumPy scalar types: +`Float64` requires `numpy.float64` scalar arguments and returns a +`numpy.float64` result. Pass the exact NumPy scalar types: ```python # This will raise TypeError @@ -133,15 +126,19 @@ scale.scale(3.0, 2.5) scale.scale(np.float64(3.0), np.float64(2.5)) ``` -Always convert at the call site for scalar arguments. - ---- +Always convert at the call site for scalar arguments. If the build fails, +rerun the command for your selected path with `--verbose`. -If the build fails, rerun it with `--verbose`. +The semantic `.pyi` is an editable description of the Python interface. The +[Common Beginner Workflow](beginner-workflow.md) shows how to save and edit it. --- ## Next -- Continue with [Your First Wrapped Module](first-wrapped-module.md). -- For more function behavior, see [Wrapping Functions](../guide/wrapping-functions.md). +- Continue with the [Common Beginner Workflow](beginner-workflow.md). +- For Fortran procedures and modules, see [Wrapping + Functions](../guide/wrapping-functions.md) and [Wrapping + Modules](../guide/wrapping-modules.md). +- For C functions and pointer contracts, see the [C User + Guide](../guide/c/index.md). diff --git a/docs/user/getting-started/first-wrapped-module.md b/docs/user/getting-started/first-wrapped-module.md deleted file mode 100644 index c83ed1cc5..000000000 --- a/docs/user/getting-started/first-wrapped-module.md +++ /dev/null @@ -1,162 +0,0 @@ ---- -title: First Wrapped Module -description: Wrap a Fortran module with public procedures and state variables -audience: users -prerequisites: first wrapped function -related: beginner-workflow.md, ../guide/wrapping-modules.md -status: maintained -publication: reviewed ---- - -# First Wrapped Module - -A Fortran `module` becomes a **child namespace** inside the generated Python extension. Public procedures and supported public variables are exposed under that namespace. - ---- - -## Source Code - -Create a file named `module_state.f90`: - -```fortran -module module_state - implicit none - private - - public :: nmax, counter, scale, saved_counter - public :: summarize, scaled_counter, next_local - - integer(4), parameter :: nmax = 12 - integer(4) :: counter = 3 - real(8) :: scale = 1.5d0 - integer(4), save :: saved_counter = 6 - integer(4) :: hidden_counter = 17 - -contains - - integer(4) function summarize() result(value) - value = counter + nmax - end function summarize - - real(8) function scaled_counter() result(value) - value = real(counter, 8) * scale - end function scaled_counter - - integer(4) function next_local() result(value) - integer(4), save :: local_counter = 0 - local_counter = local_counter + 1 - value = local_counter - end function next_local - -end module module_state -``` - ---- - -## Build the Extension - -Run the following command: - -```bash -python3 -m prik module_state.f90 --out-dir build/first-module -``` - -The extension will be named `module_state`, and the Fortran module will be available as `module_state.module_state`. - ---- - -## Inspect the Generated Docstring - -Import the built module and print its generated docstring: - -```python -import sys - -sys.path.insert(0, "build/first-module") -import module_state.module_state as mod - -print(mod.__doc__) -``` - -```text -module_state - -Module Attributes ------------------ -nmax : int32 - Read-only constant. -counter : int32 -scale : float64 -saved_counter : int32 - -Functions ---------- -summarize() -> int32 -scaled_counter() -> float64 -next_local() -> int32 -``` - -`help(mod)` shows the same index. Individual functions have their own detailed -docstrings. - ---- - -## Usage Example - -```python -import numpy as np - -print(mod.nmax) # 12 -print(mod.counter) # 3 -print(mod.scale) # 1.5 - -print(mod.summarize()) # 15 -print(mod.scaled_counter()) # 4.5 -``` - ---- - -## Mutating Module State - -```python -mod.counter = np.int32(9) -print(mod.summarize()) # 21 - -mod.scale = np.float64(2.0) -print(mod.scaled_counter()) # 18.0 -``` - -Procedure-local `save` variables also persist across calls: - -```python -print(mod.next_local()) # 1 -print(mod.next_local()) # 2 -``` - ---- - -## Inspect the Contract - -Preview the generated interface without building: - -```bash -python3 -m prik generate --pyi module_state.f90 -``` - ---- - -## Key Rules - -- The extension name is derived from the source filename. -- Each Fortran `module` becomes a child Python namespace. -- Only **public** entities are exposed. -- Private variables (like `hidden_counter`) are hidden. -- Assign module variables with the matching NumPy scalar dtype. - ---- - -## Next - -- Continue with the [Beginner Workflow](beginner-workflow.md) to turn these - steps into a repeatable development loop. -- For module details, see [Wrapping Modules](../guide/wrapping-modules.md). diff --git a/docs/user/getting-started/index.md b/docs/user/getting-started/index.md index ad02f4f74..c6bdf3ca7 100644 --- a/docs/user/getting-started/index.md +++ b/docs/user/getting-started/index.md @@ -1,6 +1,6 @@ --- title: Getting Started -description: Install PRIK, set up compilers, and build your first Fortran-to-Python extension +description: Install PRIK and build your first Python extension from Fortran or C code audience: users prerequisites: none related: installation.md, verification.md @@ -11,11 +11,8 @@ publication: reviewed # Getting Started This guide takes you from a fresh environment to your first working Python -extension built from Fortran code. - -Start with GNU (`gfortran` and `gcc`), tested on Linux and macOS. LLVM Flang -is tested on both platforms; Intel IFX is tested on Linux. See -[Installation](installation.md#compiler-toolchains) for every compiler option. +extension. Choose Fortran or C for the native source; both paths generate a +semantic `.pyi` contract, build an extension, and call the same Python API. --- @@ -24,18 +21,17 @@ is tested on both platforms; Intel IFX is tested on Linux. See Follow these pages in order: 1. **[Installation](installation.md)** — Install PRIK and the required native compilers. -2. **[Verification](verification.md)** — Check the package, headers, and compiler. -3. **[Your First Function](first-wrapped-function.md)** — Wrap a simple scalar Fortran function. -4. **[Your First Module](first-wrapped-module.md)** — Work with Fortran modules and saved state. -5. **[Development Workflow](beginner-workflow.md)** — Learn the edit → review → build → test loop. +2. **[Verification](verification.md)** — Check the package, headers, and your selected toolchain. +3. **[Your First Function](first-wrapped-function.md)** — Build the same scalar function from Fortran or C. +4. **[Development Workflow](beginner-workflow.md)** — Repeat the edit → review → build → test loop. --- ## What You Will Build -In [Your First Function](first-wrapped-function.md), you will create -`scale.f90`, build it as a Python extension named `scale`, and call its -`scale` function: +In [Your First Function](first-wrapped-function.md), you will create either +`scale.f90` or `scale.c`. Both paths build a Python extension named `scale` +with the same call: ```python import numpy as np @@ -46,9 +42,10 @@ result = scale.scale(np.float64(3.0), np.float64(2.5)) print(result) # 7.5 ``` -The first example exposes a standalone Fortran function directly on the extension. -Later guides show how Fortran modules become Python namespaces and derived -types become Python classes. +After this first build, continue with the [Fortran User +Guide](../guide/wrapping-functions.md) or the [C User Guide](../guide/c/index.md). +Fortran modules and their Python namespaces are covered in [Wrapping +Modules](../guide/wrapping-modules.md). --- diff --git a/docs/user/getting-started/installation.md b/docs/user/getting-started/installation.md index 905b299cb..1eb21952b 100644 --- a/docs/user/getting-started/installation.md +++ b/docs/user/getting-started/installation.md @@ -1,6 +1,6 @@ --- title: Installation -description: Install PRIK from PyPI and choose a Fortran compiler +description: Install PRIK from PyPI and choose a C or Fortran toolchain audience: users, contributors prerequisites: Python 3.10 or newer related: verification.md @@ -10,9 +10,9 @@ publication: reviewed # Installation -Install PRIK from the Python Package Index with `pip`. Building Python -extensions also requires a Fortran compiler, its matching C compiler, and -standard build tools. +Install PRIK from the Python Package Index with `pip`. A Fortran build requires +a Fortran compiler and its matching C compiler. A C-only build requires a C +compiler. Both paths need the standard Python extension build tools. --- @@ -31,7 +31,11 @@ python3 --version ## Native Prerequisites -The beginner path uses: +Every build needs NumPy, Python development headers, a linker, and a C +compiler. The Fortran path also needs a Fortran compiler from the same +toolchain family. + +The recommended Fortran path uses: - `gfortran` (GNU Fortran compiler) - `gcc` (normally provided by `build-essential`) @@ -39,14 +43,26 @@ The beginner path uses: - NumPy (includes required development files) - `build-essential` (linker and build tools) -On **Ubuntu / Debian**: +On **Ubuntu / Debian**, install the C-only prerequisites with: ```bash sudo apt-get update -sudo apt-get install build-essential gfortran python3-dev +sudo apt-get install build-essential python3-dev ``` -On **macOS** with Homebrew: +Add GNU Fortran when wrapping Fortran code: + +```bash +sudo apt-get install gfortran +``` + +On **macOS**, the Xcode command-line tools provide Clang for C-only builds: + +```bash +xcode-select --install +``` + +Install GNU Fortran with Homebrew when wrapping Fortran code: ```bash brew install gcc@13 @@ -56,8 +72,10 @@ Homebrew provides versioned commands such as `gfortran-13` and `gcc-13`. ## Compiler Toolchains -`gfortran` is the default. Use `--compiler` to choose another option. Install -the matching C compiler shown below as well. +For C-only builds, `cc` is the default; pass `--compiler gcc`, `clang`, or +another compatible executable when needed. Fortran builds use `gfortran` by +default. Use `--compiler` to choose another Fortran option and install its +matching C compiler from the table. | Fortran compiler | Required C compiler | Test status | | --- | --- | --- | @@ -93,6 +111,13 @@ prik --help python3 -m prik --help ``` +IPython and Jupyter support is optional. Install it when you want to use the +[`%%fortran`, `%%c`, and `%%pyi` cell magics](../guide/notebooks.md): + +```bash +python3 -m pip install "prik[jupyter]" +``` + --- ## Contributor Installation @@ -123,4 +148,5 @@ python3 -m pip install -e ".[qa]" ## Next -- Go to [Verification](verification.md) to check the installation and compiler. +- Go to [Verification](verification.md) to check the installation and your + selected toolchain. diff --git a/docs/user/getting-started/verification.md b/docs/user/getting-started/verification.md index 1618e39ca..5a5c2abbb 100644 --- a/docs/user/getting-started/verification.md +++ b/docs/user/getting-started/verification.md @@ -42,9 +42,15 @@ Both commands should print existing directories. --- -## 3. Verify the Compiler Pair +## 3. Verify Your Toolchain -For the recommended GNU path: +For a C-only build, check the C compiler you plan to use. The default is `cc`: + +```bash +cc --version +``` + +For the recommended GNU Fortran path, check both compilers in the pair: ```bash gfortran --version @@ -63,8 +69,9 @@ flang --version clang --version ``` -Both commands for your chosen compiler should report a version. If either is -missing, install it or add its `bin` directory to `PATH`. +Fortran builds need both commands in the selected pair. A C-only build needs +only its C compiler. If a required command is missing, install it or add its +`bin` directory to `PATH`. --- @@ -74,7 +81,7 @@ missing, install it or add its `bin` directory to `PATH`. |--------------------------------|---------------------------------------------| | Cannot import PRIK / NumPy | Check active virtual environment | | A header directory is missing | Reinstall Python development files or NumPy | -| Compiler not found | Fix `PATH` or install both executables from a supported pair | +| Compiler not found | Fix `PATH` or install the compiler required by your selected path | --- diff --git a/docs/user/guide/building-shared-library.md b/docs/user/guide/building-shared-library.md index d22c69c5c..78d22f7db 100644 --- a/docs/user/guide/building-shared-library.md +++ b/docs/user/guide/building-shared-library.md @@ -61,11 +61,10 @@ for versions and other recognized options. ## Build a primitive C API directly -PRIK supports C source as well. Start with [C -Support](../language-support/c-support.md) for complete source and -semantic-contract examples, Python API, supported C and NumPy types, pointer -contracts, preprocessing, generated Makefiles, and current limits. C input -always requires `--language c`. +PRIK supports C source as well. Start with the [C User Guide](c/index.md) for +source and semantic-contract examples, then use [C +Support](../language-support/c-support.md) for the exact supported surface. C +input always requires `--language c`. ## Import @@ -83,6 +82,12 @@ import scale_api The shared-library filename includes a platform- and Python-specific suffix, but the import uses only the module name. +The imported module name is also part of each wrapped function's public +identity. If `--out geometry` contains a child namespace `points`, its function +is identified as `geometry.points.distance`. A standalone function is +identified as `geometry.distance`. PRIK keeps this user-selected package path; +only private build and cache names are excluded from public function metadata. + ## Multiple Source Files Pass every wrapped source file in one command. Choosing the module name diff --git a/docs/user/guide/c/functions-and-scalars.md b/docs/user/guide/c/functions-and-scalars.md new file mode 100644 index 000000000..9a3ee17e9 --- /dev/null +++ b/docs/user/guide/c/functions-and-scalars.md @@ -0,0 +1,196 @@ +--- +title: C Functions and Scalars +description: Build C scalar functions and shape their Python call surface +audience: users +prerequisites: C user guide overview +related: index.md, pointers-arrays-and-strings.md, ../../language-support/c-support.md, ../../reference/pyi-contracts/calls-and-results.md +status: maintained +publication: reviewed +--- + +# C Functions and Scalars + +## Build a scalar C function + +Create `native_math.c`, build it, and call the generated extension: + +
+
+ + + +
+ +
+ +```c +double add(double left, double right) { + return left + right; +} +``` + +```bash +python3 -m prik --language c native_math.c \ + --compiler cc \ + --out native_math \ + --out-dir build +``` + +
+ +
+ +The source build writes this editable `native_math.pyi` contract beside the +extension: + +```python +from prik.contracts import Float64 + +def add(left: Float64, right: Float64) -> Float64: ... +``` + +Generate the contract without compiling when you only want to inspect it: + +```bash +python3 -m prik generate --pyi --language c native_math.c --out native_math.pyi +``` + +
+ +
+ +```python +import sys + +import numpy as np + +sys.path.insert(0, "build") +import native_math + +print(native_math.add(np.float64(3.0), np.float64(2.5))) +``` + +```text +5.5 +``` + +
+
+ +Pass the NumPy scalar matching the generated contract type—for example, +`np.float64` for a C `double`. C contract extraction writes one file rather +than the Fortran package layout; the [`.pyi` format +reference](../../reference/pyi-format.md#source-to-contract-layout) shows both +forms. + +## Rename and reorder arguments + +An authored contract can present an existing C ABI under a better Python name +and argument order. It names the real C symbol, then states each native +argument explicitly. + +When the Python declaration and C symbol have the same name, omit `@bind`: +that name is the default native target. Use `@bind("native_name")` only for a +different C symbol. + +
+
+ + + +
+ +
+ +Create `projected.c`: + +```c +int combine_native(int right, int *left, int bias) { + return 100 * right + 10 * *left + bias; +} + +void read_status(int value, int *output) { + *output = value + 1; +} +``` + +
+ +
+ +Create `projected.pyi`: + +```python +from prik.contracts import Addr, Arg, Int32, Return, bind, native_call + +@bind("combine_native") +@native_call([Arg(1), Addr(Arg(0)), Int32(5)]) +def combine(left: Int32, right: Int32) -> Int32: ... + +@bind("read_status") +@native_call([Arg(0), Return("output", 0)]) +def status(value: Int32) -> Int32: ... +``` + +`combine` is the Python name, `combine_native` is the linked C symbol, +`Addr(Arg(0))` passes the address of `left`, and `Int32(5)` supplies the literal +third native argument. `Return(...)` turns the output pointer into the Python +result. + +```bash +python3 -m prik --language c projected.pyi \ + --native-c-sources projected.c \ + --compiler cc \ + --out projected \ + --out-dir build +``` + +
+ +
+ +```python +import sys + +import numpy as np + +sys.path.insert(0, "build") +import projected + +print(projected.combine(np.int32(2), np.int32(3))) +print(projected.status(np.int32(7))) +``` + +```text +325 +8 +``` + +
+
+ +## Exact native scalar identities + +Generated C contracts are target-specific. Distinct C types such as `long` +and `long long` may use the same public NumPy contract type while retaining +their exact native identity inside `@native_call(...)`: + +```python +from prik.contracts import Arg, CLongLong, Float64, Int64, Return, native_call + +@native_call([Arg(0)], result=CLongLong(Return(0))) +def llround(value: Float64) -> Int64: ... +``` + +The public signature continues to use ordinary NumPy contract types. Scalars +and scalar addresses accept that public dtype and convert at the native +boundary. Ranked arguments require the exact NumPy element storage so their +pointer path remains zero-copy. See [Preserve an Exact C Scalar at the Native +Call](../../reference/pyi-contracts/calls-and-results.md#preserve-an-exact-c-scalar-at-the-native-call) +for arguments, addresses, results, arrays, and exact-storage rules. + +## Next + +Continue with [Pointers, Arrays, and +Strings](pointers-arrays-and-strings.md) when a C parameter uses pointer +syntax. diff --git a/docs/user/guide/c/index.md b/docs/user/guide/c/index.md new file mode 100644 index 000000000..e4d45b906 --- /dev/null +++ b/docs/user/guide/c/index.md @@ -0,0 +1,34 @@ +--- +title: C User Guide +description: Build C functions as NumPy-aware Python extensions with PRIK +audience: users +prerequisites: installation, basic Python and NumPy +related: functions-and-scalars.md, ../../language-support/c-support.md, ../../examples/c/libm-wrapper.md +status: maintained +publication: reviewed +--- + +# C User Guide + +PRIK builds C functions as importable Python extensions. Start from C source +when its declarations already describe the Python API. Use an editable +semantic `.pyi` contract when a pointer represents an array, output, string, +or another Python-facing value that C syntax cannot identify on its own. + +Follow the guide in this order: + +1. [Functions and Scalars](functions-and-scalars.md) — build a C function, + inspect its contract, and shape its Python name and arguments. +2. [Pointers, Arrays, and Strings](pointers-arrays-and-strings.md) — state the + Python meaning of C pointer parameters. +3. [Outputs and Errors](outputs-and-errors.md) — return output parameters and + project native status values into Python exceptions. +4. [Symbols, Headers, and Dependencies](symbols-headers-and-dependencies.md) — + wrap overload sets, broad headers, and linked libraries. + +For the exact supported surface and current boundaries, see [C +Support](../../language-support/c-support.md). For complete library examples, +see the [libm](../../examples/c/libm-wrapper.md) and +[TA-Lib](../../examples/c/ta-lib-wrapper.md) guides. + +Start with **[Functions and Scalars](functions-and-scalars.md)**. diff --git a/docs/user/guide/c/outputs-and-errors.md b/docs/user/guide/c/outputs-and-errors.md new file mode 100644 index 000000000..1cff314a0 --- /dev/null +++ b/docs/user/guide/c/outputs-and-errors.md @@ -0,0 +1,192 @@ +--- +title: C Outputs and Errors +description: Return C output parameters and project native status values into Python errors +audience: users +prerequisites: C pointers, arrays, and strings +related: pointers-arrays-and-strings.md, symbols-headers-and-dependencies.md, ../../reference/pyi-contracts/calls-and-results.md, ../error-handling.md +status: maintained +publication: reviewed +--- + +# C Outputs and Errors + +An authored contract decides which C pointer parameters are visible Python +arguments, returned values, or native-only status storage. + +## Return several C outputs + +Use a named `Return(...)` slot for every native output pointer that should +become part of the Python return value. + +
+
+ + + +
+ +
+ +Create `stats.c`: + +```c +#include + +void stats_compute(size_t count, const double *values, double *mean, double *total) { + double sum = 0.0; + for (size_t index = 0; index < count; ++index) { + sum += values[index]; + } + *total = sum; + *mean = count ? sum / (double)count : 0.0; +} +``` + +
+ +
+ +Create `stats.pyi`: + +```python +from prik.contracts import Arg, Float64, Return, Returns, bind, native_call + +@bind("stats_compute") +@native_call([Arg(0).shape[0], Arg(0), Return("mean", 0), Return("total", 1)]) +def summarize(values: Float64[:]) -> tuple[Returns["mean", Float64], Returns["total", Float64]]: ... +``` + +```bash +python3 -m prik --language c stats.pyi \ + --native-c-sources stats.c \ + --compiler cc \ + --out stats \ + --out-dir build +``` + +
+ +
+ +```python +import sys + +import numpy as np + +sys.path.insert(0, "build") +import stats + +mean, total = stats.summarize(np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float64)) +print(mean, total) +``` + +```text +2.5 10.0 +``` + +
+
+ +See [Calls and Results](../../reference/pyi-contracts/calls-and-results.md) for +the complete shared contract vocabulary. + +## Hide native outputs and raise Python exceptions + +Use `Hidden(name, T)` for C output storage that Python should not return. A +common case is a status value and diagnostic message consumed by `@raises`. + +
+
+ + + +
+ +
+ +Create `checked.c`: + +```c +#include + +void checked_sqrt(double value, double *root, int *status, char *message) { + if (value < 0.0) { + *status = -1; + *root = 0.0; + strcpy(message, "value must not be negative"); + return; + } + *status = 0; + message[0] = '\0'; + *root = value == 4.0 ? 2.0 : value; +} +``` + +
+ +
+ +Create `checked.pyi`: + +```python +from prik.contracts import Arg, Float64, Hidden, Int32, Return, Returns, String, native_call, raises + +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Return("root", 0), Hidden("status", Int32), Hidden("message", String[64])]) +def checked_sqrt(value: Float64) -> Returns["root", Float64]: ... +``` + +```bash +python3 -m prik --language c checked.pyi \ + --native-c-sources checked.c \ + --compiler cc \ + --out checked \ + --out-dir build +``` + +
+ +
+ +```python +import sys + +import numpy as np + +sys.path.insert(0, "build") +import checked + +print(checked.checked_sqrt(np.float64(4.0))) +try: + checked.checked_sqrt(np.float64(-1.0)) +except RuntimeError as error: + print(error) +``` + +```text +2.0 +value must not be negative +``` + +
+
+ +The function returns only `root`; `status` and `message` produce a +`RuntimeError` on failure. A hidden message needs a fixed capacity because +PRIK allocates its native storage. + +A caller-owned visible message buffer uses rank-zero NumPy storage: + +```python +@raises(status="status", message="message", success=0) +@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) +def checked(value: Float64, message: String[64][()]) -> None: ... +``` + +The caller can inspect the `S64` array after the call or exception. + +## Next + +Continue with [Symbols, Headers, and +Dependencies](symbols-headers-and-dependencies.md) for larger native APIs and +libraries. diff --git a/docs/user/guide/c/pointers-arrays-and-strings.md b/docs/user/guide/c/pointers-arrays-and-strings.md new file mode 100644 index 000000000..3a5c9a639 --- /dev/null +++ b/docs/user/guide/c/pointers-arrays-and-strings.md @@ -0,0 +1,307 @@ +--- +title: C Pointers, Arrays, and Strings +description: Give C pointer parameters precise Python and NumPy contracts +audience: users +prerequisites: C functions and scalars +related: functions-and-scalars.md, outputs-and-errors.md, ../../language-support/c-support.md, ../../reference/pyi-contracts/calls-and-results.md +status: maintained +publication: reviewed +--- + +# C Pointers, Arrays, and Strings + +C syntax cannot tell whether `double *` represents one scalar, an output, or +the first element of an array. A source-generated contract therefore starts +with runtime-rank caller-owned storage such as `Float64[...]`. It accepts a +zero-dimensional or higher-rank NumPy array without guessing one fixed rank. +Edit the semantic `.pyi` when the Python API should require a scalar value, +one exact rank, or a projected result. + +## Author a contract for pointers and arrays + +When a pointer is a NumPy buffer, state its shape and the native call order in +the contract: + +
+
+ + + +
+ +
+ +Create `scale.c`: + +```c +#include + +void scale(size_t count, double *values) { + for (size_t index = 0; index < count; ++index) { + values[index] *= 2.0; + } +} +``` + +
+ +
+ +Create `scale.pyi`: + +```python +from prik.contracts import Arg, Float64, native_call + +@native_call([Arg(0).shape[0], Arg(0)]) +def scale(values: Float64[:]) -> None: ... +``` + +`Arg(0).shape[0]` provides `count`; `Arg(0)` passes the NumPy buffer to +`double *values`. + +```bash +python3 -m prik --language c scale.pyi \ + --native-c-sources scale.c \ + --compiler cc \ + --out scale \ + --out-dir build +``` + +
+ +
+ +```python +import sys + +import numpy as np + +sys.path.insert(0, "build") +import scale + +values = np.array([1.0, 2.0, 3.0], dtype=np.float64) +scale.scale(values) +print(values) +``` + +```text +[2. 4. 6.] +``` + +
+
+ +Runtime-rank pointer storage accepts ranks 0 through 15 with primitive +non-Boolean elements, and it constrains neither ordering nor strides: a +Fortran-ordered array and a strided slice are both accepted. PRIK validates +dtype, rank, any declared shape, layout, and writeability before calling C. +An explicit shape such as `Float64[:]` still requires C-contiguous storage. + +## Choose the pointer contract + +`Arg(i)` uses the annotation's normal C representation: a numeric scalar +crosses by value, while rank-zero and array storage cross by address. Use +`Addr(Arg(i))` only when a scalar must become a C pointer. + +Every row below is a valid edit of the generated `Float64[...]`. Choose the row +whose Python column matches the API you want, and the caller passes exactly +that: + +| Python contract | Accepted Python value | `@native_call` entry | Native effect | +| --- | --- | --- | --- | +| `value: Float64` | `np.float64(3.0)` | `Arg(0)` | Passes `double` by value, not a pointer. | +| `value: Float64` | `np.float64(3.0)` | `Addr(Arg(0))` | Passes the address of call-local storage; native mutation is discarded unless returned. | +| `value: Float64[()]` | `np.array(3.0)` | `Arg(0)` | Passes the address of caller-owned rank-zero storage; native mutation is visible. | +| `values: Float64[:]` | `np.array([1.0, 2.0])` | `Arg(0)` | Requires rank one, contiguous. | +| `values: Float64[4]`, `Float64[n]` | a contiguous rank-one array of that extent | `Arg(0)` | Requires rank one and validates the declared extent. | +| `values: Float64[:, :]` | `np.ones((2, 3))` | `Arg(0)` | Requires rank two, C-contiguous. | +| `values: Float64[...]` | any of the above **except** `np.float64(3.0)` | `Arg(0)` | Passes the data address of caller-owned storage of any rank 0 through 15 and any strides. | +| `Annotated[Float64[...], Contiguous]` | the same, restricted to C-contiguous storage | `Arg(0)` | The same runtime rank, narrowed to C-contiguous storage. | + +The `@native_call` entry is optional when it would be `[Arg(0), Arg(1), ...]` +in declaration order; write one only to reorder, project, or hide arguments. + +Two distinctions decide most edits: + +- **`Float64` versus `Float64[()]`.** Both correspond to `double *`. `Float64` + takes a Python scalar and needs `Addr(Arg(0))` to become a pointer, and the + native write lands in call-local storage the caller never sees. `Float64[()]` + takes `np.array(3.0)`, is already an address, and the native write is visible + in the caller's array. +- **`Float64[...]` versus an exact shape.** `Float64[...]` accepts rank 0 + through 15 and any strides, so it fits a pointer whose meaning the source did + not settle. An exact shape states the rank the function actually requires and + lets PRIK validate extents and layout for you. + +Do not wrap `Float64[()]` or an array in `Addr(...)`; their native +representation is already an address. `Addr(Float64)` as an *annotation* is a +different form — a Python integer holding a raw native address — and it is +**not** supported on the direct C route; a contract that uses it is rejected +with `C_DIRECT_RAW_ADDRESS` before the build runs. Use `Float64` plus +`Addr(Arg(0))` when you want an address taken for you, or `Float64[()]` when +the caller should own the storage. + +Return a modified call-local scalar explicitly: + +```python +from prik.contracts import Addr, Arg, Float64, Returns, native_call + +@native_call([Addr(Arg(0))]) +def scale_scalar(value: Float64) -> Returns["value", Float64]: ... +``` + +`Float64[...]` does not pass a rank or extent to C. When a native parameter +needs the total element count, project the array's `size`: + +```python +from prik.contracts import Arg, Float64, native_call + +@native_call([Arg(0).size, Arg(0)]) +def scale(values: Float64[...]) -> None: ... +``` + +## Pass a layout the caller chose + +`Float64[...]` accepts whatever strides the caller's array already has, and +PRIK passes only the data address. Project the layout the native code needs: + +| Projection | Value the binding materializes | +| --- | --- | +| `Arg(i).size` | Total number of elements. | +| `Arg(i).shape[d]` | Extent of axis `d`. | +| `Arg(i).strides[d]` | Stride of axis `d` **in bytes**, exactly as `ndarray.strides` reports it. | + +```python +from prik.contracts import Arg, Float64, Int64, native_call + +@native_call([Arg(0).shape[0], Int64(Arg(0).strides[0]), Arg(0)]) +def scale(values: Float64[...]) -> None: ... +``` + +```c +void scale(size_t count, long long stride_bytes, double *values); +``` + +An axis projection requires the actual to have that axis. `Arg(0).shape[0]` +against rank-zero storage raises `TypeError` rather than reading past the +array's shape, so narrow the annotation to `Float64[:]` when the function +always needs rank one. + +A native routine that walks `values[index]` needs contiguous elements. Either +pass the stride the routine should honor, or require contiguity in the +contract; `Arg(i).size` alone does not make a strided view safe to walk +contiguously. + +```python +from prik.contracts import Annotated, Arg, Contiguous, Float64, native_call + +@native_call([Arg(0).size, Arg(0)]) +def scale(values: Annotated[Float64[...], Contiguous]) -> None: ... +``` + +`Contiguous` keeps runtime ranks 0 through 15 and rejects an actual that is +not C-contiguous. + +The contract cannot infer how many elements native code accesses. Keep a +native count visible, derive it with `.size` or `.shape[d]`, or declare an +exact shape as appropriate. When the source declares `const T *`, do not +author writable storage or write-back through it. + +## Pass C strings + +Choose the string contract from what the C function does with the pointer: + +| C parameter | Contract | Python value | +| --- | --- | --- | +| Read-only `const char *` | `String` | Python `str` | +| Writable `char *` | `String[n][()]` or `String[...][()]` | Rank-zero NumPy `S` array | + +`String` borrows the UTF-8 buffer of the Python `str`; the C function must not +write through it. Writable strings use caller-owned NumPy bytes storage. A +declared capacity such as `String[32][()]` validates the array itemsize, while +`String[...][()]` accepts the caller's itemsize. + +
+
+ + + +
+ +
+ +Create `text.c`: + +```c +#include +#include + +int name_length(const char *text) { + return (int)strlen(text); +} + +void shout(const char *text, char *out) { + size_t index = 0; + for (; text[index]; ++index) { + char value = text[index]; + out[index] = (value >= 'a' && value <= 'z') ? (char)(value - 32) : value; + } + out[index] = '\0'; +} +``` + +
+ +
+ +Create `text.pyi`: + +```python +from prik.contracts import Int32, String + +def name_length(text: String) -> Int32: ... + +def shout(text: String, out: String[32][()]) -> None: ... +``` + +```bash +python3 -m prik --language c text.pyi \ + --native-c-sources text.c \ + --compiler cc \ + --out text \ + --out-dir build +``` + +
+ +
+ +```python +import sys + +import numpy as np + +sys.path.insert(0, "build") +import text + +print(text.name_length("hello")) +buffer = np.array(b"", dtype="S32") +text.shout("hello", buffer) +print(buffer[()]) +``` + +```text +5 +b'HELLO' +``` + +
+
+ +When C uses an explicit byte length, pass it with `Len(Arg(i))` in +`@native_call(...)`. The contract does not impose a terminator convention. + +## Next + +Continue with [Outputs and Errors](outputs-and-errors.md) when pointer +parameters should become Python results or exceptions. diff --git a/docs/user/guide/c/symbols-headers-and-dependencies.md b/docs/user/guide/c/symbols-headers-and-dependencies.md new file mode 100644 index 000000000..03668ac3e --- /dev/null +++ b/docs/user/guide/c/symbols-headers-and-dependencies.md @@ -0,0 +1,218 @@ +--- +title: C Symbols, Headers, and Dependencies +description: Present C overloads and build extensions from headers and native libraries +audience: users +prerequisites: C functions and semantic contracts +related: index.md, ../../reference/cli-commands.md, ../../reference/python-api.md, ../../examples/c/libm-wrapper.md, ../../language-support/c-support.md +status: maintained +publication: reviewed +--- + +# C Symbols, Headers, and Dependencies + +## Present several C symbols as one Python name + +An authored contract can dispatch supported dtype and rank variants behind one +Python name. Mark the concrete candidates `@private`, then name them with +`@overload(...)`. + +
+
+ + + +
+ +
+ +Create `overloads.c`: + +```c +int scale_integer(int value) { return value * 2; } + +double scale_real(double value) { return value * 2.0; } +``` + +
+ +
+ +Create `overloads.pyi`: + +```python +from prik.contracts import Float64, Int32, overload, private + +@private +def scale_integer(value: Int32) -> Int32: ... + +@private +def scale_real(value: Float64) -> Float64: ... + +@overload("scale_integer") +def scale(value: Int32) -> Int32: ... + +@overload("scale_real") +def scale(value: Float64) -> Float64: ... +``` + +```bash +python3 -m prik --language c overloads.pyi \ + --native-c-sources overloads.c \ + --compiler cc \ + --out overloads \ + --out-dir build +``` + +
+ +
+ +```python +import sys + +import numpy as np + +sys.path.insert(0, "build") +import overloads + +print(overloads.scale(np.int32(21))) +print(overloads.scale(np.float64(1.5))) +print([name for name in dir(overloads) if not name.startswith("_")]) +``` + +```text +42 +3.0 +['scale'] +``` + +
+
+ +Candidates must remain distinguishable by their supported dtype and rank. + +## Qualifiers and compiler attributes + +Use C qualifiers as constraints when authoring a contract: `const T *` must +not be presented as writable NumPy storage. `const` and `restrict` do not add +a separate Python type or calling convention. + +Common non-ABI attributes such as `deprecated` and `warn_unused_result` do not +change a wrapper. An attribute that may change the ABI, symbol identity, or +layout stops the build instead of being ignored. + +Compiler-preprocessed system headers may contain unavailable extended floating +types in private declarations. PRIK can use those declarations as parsing +context without adding wrapper support for the extended type. Prototype +parameters may omit names; actual K&R definitions remain unsupported. + +## Symbols declared by binding headers + +A header included by the generated binding may already declare the same name +for a different API. Select that symbol for an isolated collision forwarder: + +```bash +python3 -m prik --language c vendor.pyi \ + --native-library vendor \ + --collision-adapter evaluate \ + --out vendor_api --out-dir build +``` + +The build writes a separate forwarding translation unit that includes no +Python header. Its signature uses the completed exact native C types: + +```c +long long evaluate(double x); + +long long prik_collision_adapter_evaluate(double x) { + return (evaluate)(x); +} +``` + +The adapter targets a real function symbol; PRIK does not expose macros. The +forwarder has hidden visibility and works with or without `--lto`. Use +`--collision-adapter-all` to adapt every eligible C source function. + +This solves a declaration collision inside the generated binding. It does not +choose between two linked libraries that export the same external symbol. +Normal linker and loader resolution must already select the intended library. + +A source-free `.pyi` contract must preserve every exact native scalar identity +needed by the declaration. A target-generated contract does this +automatically. See [CLI Commands](../../reference/cli-commands.md#wrapper-builds) +for selection, validation, and LTO options. + +## Build and inspect APIs + +Use `build_c_extension()` for source builds or `build_pyi_extension()` for +authored contracts from Python. See the [Python +API](../../reference/python-api.md) for those calls and [CLI +Commands](../../reference/cli-commands.md) for build, generation, Makefile, +and inspection options. + +### Native dependencies + +Pass public C source files as positional inputs. Add implementation-only C +files with `--native-c-sources`, compiler flags with +`--native-c-compile-flags`, existing objects with `--native-objects`, and +libraries with `--native-library` and `--native-library-dir`. + +For headers and conditional source, pass the native project's preprocessing +configuration with `-I`, `-D`, `--std`, and, when available, +`--compile-commands build/compile_commands.json`. + +To wrap a reviewed subset of a broad or system header, list the public +functions in a file and generate their contract: + +```bash +python3 -m prik generate --pyi --language c api_probe.h \ + --include-exposure roots-only \ + --export-symbols reviewed_functions.txt \ + --out contracts/api.pyi +``` + +The export file selects the semantic API, not linker exports. Selected +functions still need native link inputs and a signature supported by the C +wrapper. See [C include +exposure](../../reference/cli-commands.md#c-include-exposure) for the file +format and validation rules. + +The Python API accepts already-resolved names: + +```python +build = build_c_extension( + "api_probe.c", + export_symbols=("evaluate", "normalize"), + native_libraries=("vendor",), +) +``` + +### Inspect a broader C API + +The parser and contract generator accept more syntax than the supported +wrapper surface. Use them to inspect declarations: + +```bash +python3 -m prik parse --language c include/library.h --json +python3 -m prik semantics --language c include/library.h +python3 -m prik generate --pyi --language c include/library.h --out contracts/library.pyi +``` + +Pass the native preprocessing configuration when the header needs it: + +```bash +python3 -m prik parse --language c include/library.h \ + -I include \ + -D LIBRARY_ENABLE_FAST=1 \ + --std c11 \ + --compile-commands build/compile_commands.json +``` + +Only declarations in the wrapped translation unit become a source build's +public API; headers provide declarations and preprocessing context. + +## Next + +See [C Support](../../language-support/c-support.md) for the complete supported +surface, or continue with the [libm](../../examples/c/libm-wrapper.md) and +[TA-Lib](../../examples/c/ta-lib-wrapper.md) examples. diff --git a/docs/user/guide/index.md b/docs/user/guide/index.md index f3c6e9206..228a22a30 100644 --- a/docs/user/guide/index.md +++ b/docs/user/guide/index.md @@ -1,76 +1,55 @@ --- title: User Guide -description: Detailed guides for wrapping Fortran code with PRIK +description: Guides for binding Fortran and C code with PRIK audience: users prerequisites: getting started -related: data-types.md +related: data-types.md, c/index.md, building-shared-library.md, ../language-support/feature-matrix.md status: maintained publication: reviewed --- # User Guide -This section continues the [Getting Started](../getting-started/index.md) -workflow. Read it in sidebar order to move from basic values and procedures to -objects, storage, and advanced runtime behavior. - ---- - -## Start Here - -- [Data Types](data-types.md) — Fortran types, semantic `.pyi` names, exact NumPy dtypes, strings, and arrays -- [Arrays](arrays.md) — Rank, shape, strides, contiguity, and layout rules -- [Strings](strings.md) — Immutable text, mutable byte storage, and string arrays -- [Wrapping Functions](wrapping-functions.md) -- [Wrapping Subroutines](wrapping-subroutines.md) -- [Wrapping Modules](wrapping-modules.md) -- [Optional Arguments](optional-arguments.md) -- [Generic Interfaces](generic-interfaces.md) -- [Wrapping Derived Types](wrapping-derived-types.md) - ---- - -## Storage and Objects - -- [Allocatables](allocatables.md) -- [Pointers](pointers.md) -- [Memory Management](memory-management.md) - ---- - -## Runtime Behavior - -- [Callbacks](callbacks.md) -- [Enumerations](enumerations.md) -- [Raw Addresses](raw-addresses.md) — Advanced primitive, array, and fixed-string address boundaries -- [Error Handling](error-handling.md) - ---- - -## Building - -- [Building the Shared Library](building-shared-library.md) - ---- - -**Important Note** - -The recommended workflow starts from Fortran source. The generated semantic -`.pyi` file describes the Python interface and native call. Editing that file -lets you customize the wrapper without changing the native implementation. -This guide introduces useful edits on the pages where they matter. The [`.pyi` -Format](../reference/pyi-format.md) defines the contract language, and [Editing -`.pyi` Contracts](../reference/pyi-contracts/index.md) collects the supported -editing recipes. - ---- - -**Checking whether a feature is supported** - -Each page below documents its own limitations. For the complete picture in one -table — including unsupported and partially supported forms — see the -[language feature matrix](../language-support/feature-matrix.md). - ---- - -Start with **[Data Types](data-types.md)**. +Use the section for your native language, then continue with the shared build +workflow that matches how you run PRIK. + +## Fortran + +- [Data Types](data-types.md) — Fortran types and exact NumPy dtypes +- [Arrays](arrays.md) — rank, shape, layout, strides, and mutation +- [Strings](strings.md) — scalar text, mutable storage, and string arrays +- [Functions](wrapping-functions.md) and + [Subroutines](wrapping-subroutines.md) +- [Modules](wrapping-modules.md), [Optional + Arguments](optional-arguments.md), and [Generic + Interfaces](generic-interfaces.md) +- [Derived Types](wrapping-derived-types.md) +- [Allocatables](allocatables.md), [Pointers](pointers.md), and [Memory + Management](memory-management.md) +- [Callbacks](callbacks.md), [Enumerations](enumerations.md), [Raw + Addresses](raw-addresses.md), and [Error Handling](error-handling.md) + +## C + +- [Overview](c/index.md) — choose the source-driven or authored-contract path +- [Functions and Scalars](c/functions-and-scalars.md) — build a function and + shape its Python call surface +- [Pointers, Arrays, and Strings](c/pointers-arrays-and-strings.md) — assign + precise Python meanings to pointer parameters +- [Outputs and Errors](c/outputs-and-errors.md) — return output storage and + project native failures +- [Symbols, Headers, and + Dependencies](c/symbols-headers-and-dependencies.md) — overloads, headers, + libraries, and API inspection + +## Build Workflows + +- [Building the Shared Library](building-shared-library.md) — compilers, + source sets, output placement, and Makefiles +- [IPython and Jupyter Notebooks](notebooks.md) — compile Fortran and C cells + and edit semantic contracts interactively + +The [`.pyi` Format](../reference/pyi-format.md) defines the shared semantic +contract language. Use the [language feature +matrix](../language-support/feature-matrix.md#at-a-glance) to check current +Fortran and C coverage. diff --git a/docs/user/guide/notebooks.md b/docs/user/guide/notebooks.md new file mode 100644 index 000000000..e0e82aba1 --- /dev/null +++ b/docs/user/guide/notebooks.md @@ -0,0 +1,247 @@ +--- +title: IPython and Jupyter Notebooks +description: Compile Fortran and C cells with PRIK cell magics +audience: users +prerequisites: installation, first wrapped function +related: wrapping-modules.md, ../reference/python-api.md, ../language-support/c-support.md +status: maintained +publication: reviewed +--- + +# IPython and Jupyter Notebooks + +PRIK provides optional `%%fortran`, `%%c`, and `%%pyi` cell magics for compiling +Fortran and C code inside IPython or a Jupyter notebook. Install the notebook +dependency and load the extension once per session: + +```bash +python3 -m pip install "prik[jupyter]" +``` + +```ipython +%load_ext prik.jupyter +``` + +## Run native code + +### Fortran + +The cell body is ordinary Fortran source: + +```ipython +%%fortran +module maths +contains + real(8) function square(x) + real(8), intent(in) :: x + square = x*x + end function +end module +``` + +PRIK publishes the declared Fortran module namespace: + +```python +import numpy as np + +maths.square(np.float64(4.0)) +``` + +Each declared module becomes a notebook name. Standalone procedures are +published directly. + +### C + +C functions are published directly in the notebook namespace: + +```ipython +%%c +double square(double x) { + return x * x; +} +``` + +```python +square(np.float64(4.0)) +``` + +## Edit the generated `.pyi` + +Add `--pyi` when the generated Python API is not the one you want. PRIK +compiles nothing yet; it persists the source and hands back an editable +contract cell. + +```ipython +%%c --pyi +#include + +void scale(size_t count, double *values) { + for (size_t i = 0; i < count; ++i) values[i] *= 2.0; +} +``` + +PRIK inserts the contract it derived from that source: + +```ipython +%%pyi + +# prik: source-sha256= + +from prik.contracts import Float64, UInt64 + +def scale( + count: UInt64, + values: Float64[...] +) -> None: ... +``` + +This is a faithful reading of the C, but it is not a good Python API: the +caller has to pass a length that NumPy already knows, and `Float64[...]` +accepts any rank because `double *` does not say which one it is. Edit the +cell, keep the `# prik:` line, and run it: + +```ipython +%%pyi + +# prik: source-sha256= + +from prik.contracts import Arg, Float64, native_call + +@native_call([Arg(0).size, Arg(0)]) +def scale(values: Float64[:]) -> None: ... +``` + +`Float64[:]` requires a rank-one array, and `Arg(0).size` supplies `count` +from it, so `count` disappears from the Python signature: + +```python +values = np.array([1.0, 2.0, 3.0]) +scale(values) +values # array([2., 4., 6.]) -- updated in place +``` + +The native code and the compiler options are unchanged; only the Python API +you call is different. + +### Contract cells and their names + +Fortran source generates one editable cell per declared module, each carrying +a `file=` field: + +```ipython +%%pyi + +# prik: file=maths.pyi source-sha256= +``` + +**The `file=` leaf name is the Fortran module name**, not a name you choose — +`maths.pyi` is the contract for `module maths`, and it publishes `maths` in +the notebook. C source and standalone Fortran declarations have no module to +name, so their cells carry no `file=` field and publish their functions +directly. + +Jupyter inserts every generated cell at once; terminal IPython presents them +in sequence as you execute each one. Compiler and build options are copied +from the source cell, so rerun the source cell to change them. + +The same workflow works with `%%fortran --pyi`. + +## Wrap an existing source file + +Use `%%pyi` directly when the native source already exists as a file. There is +no source cell to generate from, so you write the contract yourself and name +the sources on the magic line: + +```ipython +%%pyi --native-fortran-sources maths.f90 + +# prik: file=maths.pyi + +from prik.contracts import Addr, Arg, Float64, native_call + +@native_call([Addr(Arg(0))]) +def square(x: Float64) -> Float64: ... +``` + +Here `file=maths.pyi` is the contract for `module maths` inside `maths.f90`, +and the cell publishes `maths`. The leaf name must match the Fortran module, +whatever the file is called — a mismatch reaches the Fortran compiler as a +missing `.mod`, not a PRIK diagnostic. To expose a second module from the same +source, write another `%%pyi` cell naming that module, such as +`# prik: file=helpers.pyi`. + +List multiple source files in compilation order when needed: + +```ipython +%%pyi --native-fortran-sources maths.f90 helpers.f90 +``` + +```ipython +%%pyi --native-c-sources square.c +``` + +For C or standalone Fortran, omit the `# prik:` line entirely. Standalone +Fortran declarations still use `@standalone`. Relative source paths start from +the kernel's current working directory. + +## Compilers and flags + +```ipython +%%fortran --compiler ifx --native-compile-flags="-O3 -march=native" +``` + +- `--native-compile-flags` applies to the cell's own Fortran or C source. A + cell has one native language, so there is a single option here; the command + line splits the same setting into `--native-compile-flags` for Fortran and + `--native-c-compile-flags` for C. +- `--wrapper-fortran-flags` applies to generated Fortran bridge source. +- `--wrapper-c-flags` applies to generated C binding source and extension + linking. +- `--compiler-arg` adds one preprocessing argument and may be repeated. + +**A value that starts with a dash needs the equals form.** `--native-compile-flags -O3` +reads `-O3` as another option and fails; write `--native-compile-flags=-O3`. +The three flag options also accept several flags as one quoted group: + +```ipython +%%c --native-compile-flags="-O3 -march=native" +``` + +`--compiler-arg` is not split, so it carries exactly one argument. Repeat it +for more: + +```ipython +%%fortran --compiler-arg=-fdefault-real-8 --compiler-arg=-I/opt/include +``` + +## Cell cache + +PRIK reuses a compiled result when the cell, compiler, flags, and native files +are unchanged. Use `--force` to rebuild or `--verbose` to show build activity: + +```ipython +%%fortran --force +``` + +```ipython +%%fortran --verbose +``` + +The cache persists between sessions in `~/.cache/prik/jupyter`, or under +`$XDG_CACHE_HOME/prik/jupyter` or `$PRIK_CACHE_DIR/jupyter` when either is +set. Each distinct cell text gets its own entry holding that cell's source, +generated wrapper, and compiled extension, so editing a cell repeatedly leaves +one entry per version. Nothing is evicted automatically. Delete the directory +to reclaim the space: + +```bash +rm -rf ~/.cache/prik/jupyter +``` + +The next run of each cell rebuilds it. + +## Limitations + +Source magics compile one self-contained cell. Use PRIK's CLI or Python build +API for larger projects, external libraries, mixed-language builds, and +advanced link configuration. diff --git a/docs/user/guide/wrapping-functions.md b/docs/user/guide/wrapping-functions.md index 4a58adc7a..2e107e5cc 100644 --- a/docs/user/guide/wrapping-functions.md +++ b/docs/user/guide/wrapping-functions.md @@ -19,7 +19,7 @@ only when their contract marks them as Python results. ## Basic Scalar Function -The `scale` function built in +The Fortran `scale` function built in [First Wrapped Function](../getting-started/first-wrapped-function.md) returns its direct `Float64` result as a NumPy `float64` scalar: diff --git a/docs/user/guide/wrapping-modules.md b/docs/user/guide/wrapping-modules.md index 43a39944a..e82449295 100644 --- a/docs/user/guide/wrapping-modules.md +++ b/docs/user/guide/wrapping-modules.md @@ -2,7 +2,7 @@ title: Wrapping Modules description: How PRIK exposes Fortran modules as Python namespaces with procedures, variables, and state audience: users -prerequisites: data types, first wrapped module +prerequisites: data types, first wrapped function related: wrapping-functions.md, memory-management.md, building-shared-library.md status: maintained publication: reviewed @@ -14,9 +14,55 @@ A Fortran `module` becomes a **child Python module** (namespace) inside the gene --- -## Basic Usage +## Source and Build + +Create `module_state.f90`: + +```fortran +module module_state + implicit none + private + + public :: nmax, counter, scale, saved_counter + public :: summarize, scaled_counter, next_local + + integer(4), parameter :: nmax = 12 + integer(4) :: counter = 3 + real(8) :: scale = 1.5d0 + integer(4), save :: saved_counter = 6 + integer(4) :: hidden_counter = 17 -After building `module_state.f90`: +contains + + integer(4) function summarize() result(value) + value = counter + nmax + end function summarize + + real(8) function scaled_counter() result(value) + value = real(counter, 8) * scale + end function scaled_counter + + integer(4) function next_local() result(value) + integer(4), save :: local_counter = 0 + local_counter = local_counter + 1 + value = local_counter + end function next_local + +end module module_state +``` + +Build it: + +```bash +python3 -m prik module_state.f90 --out-dir build/first-module +``` + +The extension is named `module_state`. Its Fortran module is available as the +child namespace `module_state.module_state`. + +--- + +## Basic Usage ```python import sys @@ -29,8 +75,6 @@ import module_state mod = module_state.module_state # child namespace ``` -See [First Wrapped Module](../getting-started/first-wrapped-module.md) for the complete source, build command, and usage examples. - --- ## Procedures diff --git a/docs/user/index.md b/docs/user/index.md index 17e29295b..d77137024 100644 --- a/docs/user/index.md +++ b/docs/user/index.md @@ -11,7 +11,7 @@ publication: reviewed PRIK is the Python Runtime Interop Kit. Use these pages to install PRIK, verify your environment, build Fortran and C wrappers, and understand the behavior of -generated Python extensions. Current C coverage is documented in C Support. +generated Python extensions. ## Start Here @@ -20,12 +20,11 @@ generated Python extensions. Current C coverage is documented in C Support. 3. [Design a Pythonic BLAS API](tutorials/pythonic-blas.md) 4. [Performance](performance.md) -Getting Started covers installation, environment verification, the first -standalone wrapper, the first module wrapper, and the beginner edit-build-test -loop. The User Guide covers supported Fortran wrapper features, runtime -behavior, and extension builds. The tutorial then shows how to turn a low-level -native interface into a small designed API. Performance presents the -reproducible PRIK and f2py comparison. +Getting Started covers installation, environment verification, matched +Fortran and C first-function paths, and the beginner edit-build-test loop. The +User Guide provides continuous Fortran and C paths plus shared build workflows. +The tutorial then shows how to turn a low-level native interface into a small +designed API. Performance presents the reproducible PRIK and f2py comparison. ## Then diff --git a/docs/user/language-support/c-support.md b/docs/user/language-support/c-support.md index fdb8b4c78..4e7c02db9 100644 --- a/docs/user/language-support/c-support.md +++ b/docs/user/language-support/c-support.md @@ -1,872 +1,77 @@ --- title: C Support -description: Build supported C APIs as NumPy-aware Python extensions. +description: What C PRIK wraps, where the detailed guides live, and which declarations become Python APIs audience: users -prerequisites: installation, basic Python and NumPy -related: index.md, feature-matrix.md, ../reference/cli-commands.md, ../reference/python-api.md, ../reference/pyi-contracts/calls-and-results.md +prerequisites: installation +related: index.md, feature-matrix.md, ../guide/c/index.md, ../reference/pyi-format.md, ../reference/pyi-contracts/index.md status: maintained publication: reviewed --- # C Support -PRIK builds a supported subset of C APIs as importable Python extensions. The -generated binding calls your exported C symbol without ABI conversion. Every -extension has a generated CPython binding translation unit. The only optional -additional C translation unit is the opt-in forwarder for a symbol that your -headers and `Python.h` both declare, described in [Symbols your binding's own -headers declare](#symbols-your-bindings-own-headers-declare). +This page is the entry point for **what C PRIK wraps**. Start with the wrapper +areas below and follow the linked guide for complete Python behavior, +constraints, and examples. Use the [language feature +matrix](feature-matrix.md) for the exact status, evidence, and limitation of an +individual feature. Use [`.pyi` Format](../reference/pyi-format.md) for the +contract language and [Editing Contracts](../reference/pyi-contracts/index.md) +when shaping the Python API. -C wrapping is best for standalone numerical functions with primitive values, -NumPy buffers, and explicit output storage. It is deliberately fail-closed: -parsing a declaration does not promise that it can be wrapped, and an unsupported -form stops the build before native compilation. +## Supported Wrapper Areas -## Requirements - -Install PRIK and NumPy, then make sure a C compiler and the development headers -for the Python that will import the extension are available. `cc` is the default -compiler; use `--compiler` when the native project requires another one. - -To see the C types and NumPy dtypes selected for a particular compiler target, -run: - -```bash -python3 -m prik probe --language c --compiler cc -``` - -## Build a scalar C function - -This first example is source-driven: PRIK reads the C declaration, builds the -extension, and writes an editable contract alongside it. - -
-
- - - -
- -
- -Create `native_math.c`: - -```c -double add(double left, double right) { - return left + right; -} -``` - -Build it with an explicit language selection: - -```bash -python3 -m prik --language c native_math.c \ - --compiler cc \ - --out native_math \ - --out-dir build -``` - -
- -
- -To inspect the contract without compiling, write `native_math.pyi`: - -```bash -python3 -m prik generate --pyi --language c native_math.c --out native_math.pyi -``` - -The file contains: - -```python -from prik.contracts import Float64 - -def add(left: Float64, right: Float64) -> Float64: ... -``` - -C contract extraction writes one file rather than the Fortran package layout; -the [`.pyi` format reference](../reference/pyi-format.md#source-to-contract-layout) -shows both forms. - -
- -
- -Then import and call the extension: - -```python -import sys - -import numpy as np - -sys.path.insert(0, "build") -import native_math - -print(native_math.add(np.float64(3.0), np.float64(2.5))) -``` - -```text -5.5 -``` - -
-
- -PRIK validates arithmetic arguments at the native boundary. Pass the matching -NumPy scalar—for example, `np.float64` for a C `double`. - -The source build writes an editable semantic `.pyi` contract beside the -extension. Use that contract when a pointer needs a more precise Python meaning -than the C declaration can express. - -## Author a contract for pointers and arrays - -C syntax cannot tell whether `double *` means one scalar or the first element -of an array. A source-generated contract therefore starts conservatively. When -the parameter is a NumPy buffer, state the shape and native call order in an -authored `.pyi` contract. - -
-
- - - -
- -
- -Create `scale.c`: - -```c -#include - -void scale(size_t count, double *values) { - for (size_t index = 0; index < count; ++index) { - values[index] *= 2.0; - } -} -``` - -
- -
- -Create `scale.pyi`: - -```python -from prik.contracts import Arg, Float64, native_call - -@native_call([Arg(0).shape[0], Arg(0)]) -def scale(values: Float64[:]) -> None: ... -``` - -`Arg(0).shape[0]` provides `count`; `Arg(0)` passes the NumPy buffer to -`double *values`. - -```bash -python3 -m prik --language c scale.pyi \ - --native-c-sources scale.c \ - --compiler cc \ - --out scale \ - --out-dir build -``` - -
- -
- -```python -import sys - -import numpy as np - -sys.path.insert(0, "build") -import scale - -values = np.array([1.0, 2.0, 3.0], dtype=np.float64) -scale.scale(values) -print(values) -``` - -```text -[2. 4. 6.] -``` - -
-
- -Supported arrays have ranks 1 through 15, primitive non-Boolean elements, and -C-contiguous NumPy storage. PRIK validates dtype, rank, shape, layout, and -writeability before calling C. - -Use `Float64[()]` when the caller should provide one writable scalar slot. - -### Choose the pointer contract - -`Arg(i)` uses the annotation's normal C representation: a bare numeric scalar -crosses by value, while rank-zero and array storage cross by address. Use -`Addr(Arg(i))` only when a bare scalar must become a C pointer. - -| C parameter | Python contract | `@native_call` entry | Native effect | -| --- | --- | --- | --- | -| `double value` | `value: Float64` | `Arg(0)` (or omit `@native_call`) | Passes `double` by value. | -| `double *value` | `value: Float64` | `Addr(Arg(0))` | Passes the address of call-local scalar storage; mutation is discarded unless returned. | -| `double *value` | `value: Float64[()]` | `Arg(0)` (or omit `@native_call`) | Passes the caller's zero-dimensional NumPy storage address; mutation is visible in place. | -| `double *values` | `values: Float64[:]`, `Float64[4]`, or `Float64[n]` | `Arg(0)` | Passes the validated C-contiguous NumPy data address. | - -For an authored scalar read-back, write the address projection and return the -call-local value explicitly: - -```python -from prik.contracts import Addr, Arg, Float64, Returns, native_call - -@native_call([Addr(Arg(0))]) -def scale_scalar(value: Float64) -> Returns["value", Float64]: ... -``` - -A source-generated contract for `double *value` already contains this -`Addr(Arg(0))` projection. Do not wrap `Float64[()]` or an array in `Addr(...)`: -their normal native representation is already an address. - -Do not leave a pointer as a scalar when C indexes it as an array. A generated -source contract is conservative; promote the parameter to a shaped NumPy array -before calling a buffer API. - -An authored contract is authoritative. If the source C declaration is -`const T *`, do not author writable storage or write-back through it: writing -through a const-qualified C pointer is undefined behavior. - -## Rename, reorder, and address arguments - -An authored contract can present an existing C ABI under a better Python name -and argument order. It names the real C symbol, then states each native -argument explicitly. - -When the Python declaration and C symbol have the same name, omit `@bind`: -that name is the default native target. Use `@bind("native_name")` only for a -different C symbol. The same default applies to Fortran semantic contracts. - -
-
- - - -
- -
- -Create `projected.c`: - -```c -int combine_native(int right, int *left, int bias) { - return 100 * right + 10 * *left + bias; -} - -void read_status(int value, int *output) { - *output = value + 1; -} -``` - -
- -
- -Create `projected.pyi`: - -```python -from prik.contracts import Addr, Arg, Int32, Return, bind, native_call - -@bind("combine_native") -@native_call([Arg(1), Addr(Arg(0)), Int32(5)]) -def combine(left: Int32, right: Int32) -> Int32: ... - -@bind("read_status") -@native_call([Arg(0), Return("output", 0)]) -def status(value: Int32) -> Int32: ... -``` - -`combine` is the Python name, `combine_native` is the linked C symbol, -`Addr(Arg(0))` passes the address of `left`, and `Int32(5)` supplies the literal -third native argument. `Return(...)` turns the output pointer into the Python -result. - -```bash -python3 -m prik --language c projected.pyi \ - --native-c-sources projected.c \ - --compiler cc \ - --out projected \ - --out-dir build -``` - -
- -
- -```python -import sys - -import numpy as np - -sys.path.insert(0, "build") -import projected - -print(projected.combine(np.int32(2), np.int32(3))) -print(projected.status(np.int32(7))) -``` - -```text -325 -8 -``` - -
-
- -## Return several C outputs - -Use a named `Return(...)` slot for every native output pointer that should -become part of the Python return value. - -
-
- - - -
- -
- -Create `stats.c`: - -```c -#include - -void stats_compute(size_t count, const double *values, double *mean, double *total) { - double sum = 0.0; - for (size_t index = 0; index < count; ++index) { - sum += values[index]; - } - *total = sum; - *mean = count ? sum / (double)count : 0.0; -} -``` - -
- -
- -Create `stats.pyi`: - -```python -from prik.contracts import Arg, Float64, Return, Returns, bind, native_call - -@bind("stats_compute") -@native_call([Arg(0).shape[0], Arg(0), Return("mean", 0), Return("total", 1)]) -def summarize(values: Float64[:]) -> tuple[Returns["mean", Float64], Returns["total", Float64]]: ... -``` - -```bash -python3 -m prik --language c stats.pyi \ - --native-c-sources stats.c \ - --compiler cc \ - --out stats \ - --out-dir build -``` - -
- -
- -```python -import sys - -import numpy as np - -sys.path.insert(0, "build") -import stats - -mean, total = stats.summarize(np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float64)) -print(mean, total) -``` - -```text -2.5 10.0 -``` - -
-
- -See [Calls and Results](../reference/pyi-contracts/calls-and-results.md) for -the full shared contract vocabulary. - -## Pass C strings - -Choose the string contract from what the C function does with the pointer: - -| C parameter | Contract | Python value | +| Area | Supported wrapping surface | Detailed guide | | --- | --- | --- | -| Read-only `const char *` | `String` | Python `str` | -| Writable `char *` | `String[n][()]` or `String[...][()]` | Rank-zero NumPy `S` array | - -`String` borrows the UTF-8 buffer of the Python `str`, which CPython -NUL-terminates. The native function must not write through it. For writable -storage, use a caller-owned NumPy bytes array. A stated capacity such as -`String[32][()]` also checks the array itemsize; `String[...][()]` accepts the -itemsize the caller supplies. - -
-
- - - -
- -
- -Create `text.c`: - -```c -#include -#include - -int name_length(const char *text) { - return (int)strlen(text); -} - -void shout(const char *text, char *out) { - size_t index = 0; - for (; text[index]; ++index) { - char value = text[index]; - out[index] = (value >= 'a' && value <= 'z') ? (char)(value - 32) : value; - } - out[index] = '\0'; -} -``` - -
- -
- -Create `text.pyi`: - -```python -from prik.contracts import Int32, String - -def name_length(text: String) -> Int32: ... - -def shout(text: String, out: String[32][()]) -> None: ... -``` - -```bash -python3 -m prik --language c text.pyi \ - --native-c-sources text.c \ - --compiler cc \ - --out text \ - --out-dir build -``` - -
- -
- -```python -import sys - -import numpy as np - -sys.path.insert(0, "build") -import text - -print(text.name_length("hello")) -buffer = np.array(b"", dtype="S32") -text.shout("hello", buffer) -print(buffer[()]) -``` - -```text -5 -b'HELLO' -``` - -
-
- -When C uses an explicit byte length, pass it with `Len(Arg(i))` in -`@native_call(...)`. The contract does not impose a terminator convention of -its own. - -## Hide native outputs and raise Python exceptions - -Use `Hidden(name, T)` for C output storage that Python never returns. This is -particularly useful for status values and diagnostic messages consumed by -`@raises`. - -
-
- - - -
- -
- -Create `checked.c`: - -```c -#include - -void checked_sqrt(double value, double *root, int *status, char *message) { - if (value < 0.0) { - *status = -1; - *root = 0.0; - strcpy(message, "value must not be negative"); - return; - } - *status = 0; - message[0] = '\0'; - *root = value == 4.0 ? 2.0 : value; -} -``` - -
- -
- -Create `checked.pyi`: - -```python -from prik.contracts import Arg, Float64, Hidden, Int32, Return, Returns, String, native_call, raises - -@raises(status="status", message="message", success=0) -@native_call([Arg(0), Return("root", 0), Hidden("status", Int32), Hidden("message", String[64])]) -def checked_sqrt(value: Float64) -> Returns["root", Float64]: ... -``` - -```bash -python3 -m prik --language c checked.pyi \ - --native-c-sources checked.c \ - --compiler cc \ - --out checked \ - --out-dir build -``` - -
- -
- -```python -import sys - -import numpy as np - -sys.path.insert(0, "build") -import checked - -print(checked.checked_sqrt(np.float64(4.0))) -try: - checked.checked_sqrt(np.float64(-1.0)) -except RuntimeError as error: - print(error) -``` - -```text -2.0 -value must not be negative -``` - -
-
- -The function returns only `root`; `status` and `message` become a -`RuntimeError` on failure. A hidden message needs a fixed capacity because PRIK -allocates the native buffer. - -A visible message buffer is also valid when the caller owns it: - -```python -@raises(status="status", message="message", success=0) -@native_call([Arg(0), Arg(1), Hidden("status", Int32)]) -def checked(value: Float64, message: String[64][()]) -> None: ... -``` - -Here `message` is a rank-zero `np.ndarray` with dtype `S64`; the caller can -inspect it after the exception. `String` can also name a visible message when -the C API declares `const char *`; that borrows a Python `str`. If that native -code writes through the borrowed pointer, handling that unsafe contract is the -C API author's responsibility. Prefer NumPy storage for a writable message. - -## Present several C symbols as one Python name - -An authored contract can dispatch supported dtype/rank variants behind one -Python name. Mark the concrete candidates `@private`, then name them with -`@overload(...)`. - -
-
- - - -
- -
- -Create `overloads.c`: - -```c -int scale_integer(int value) { return value * 2; } - -double scale_real(double value) { return value * 2.0; } -``` - -
- -
- -Create `overloads.pyi`: - -```python -from prik.contracts import Float64, Int32, overload, private - -@private -def scale_integer(value: Int32) -> Int32: ... - -@private -def scale_real(value: Float64) -> Float64: ... - -@overload("scale_integer") -def scale(value: Int32) -> Int32: ... - -@overload("scale_real") -def scale(value: Float64) -> Float64: ... -``` - -```bash -python3 -m prik --language c overloads.pyi \ - --native-c-sources overloads.c \ - --compiler cc \ - --out overloads \ - --out-dir build -``` - -
- -
- -```python -import sys - -import numpy as np - -sys.path.insert(0, "build") -import overloads - -print(overloads.scale(np.int32(21))) -print(overloads.scale(np.float64(1.5))) -print([name for name in dir(overloads) if not name.startswith("_")]) -``` - -```text -42 -3.0 -['scale'] -``` - -
-
- -Candidates must remain distinguishable by their supported dtype and rank. - -## What is supported - -- Externally linked functions with `void`, arithmetic scalars, and C99 complex - values whose ABI the selected compiler can probe. -- One-level primitive pointer parameters, expressed as a scalar address, - rank-zero NumPy storage, a projected result, or a C-contiguous NumPy array. -- Rank-zero C string inputs and storage, hidden outputs, status projection, - symbol renaming, reordered arguments, typed literals, and derived lengths or - shapes. -- Overload sets whose candidates are distinguishable by supported dtype and - rank. -- `@nogil` calls that do not access Python state. -- Ordinary compiler preprocessing, including standard includes and macros. - -## Qualifiers and compiler attributes - -Use C qualifiers as constraints when authoring a contract: `const T *` must not -be presented as writable NumPy storage. `const` and `restrict` themselves do -not add a separate Python type or calling convention. - -Common non-ABI attributes, such as `deprecated` and `warn_unused_result`, do -not change a wrapper. An attribute that may change the ABI, symbol identity, or -layout—such as a calling convention or alignment attribute—stops the build -instead of being ignored. - -Compiler-preprocessed system headers may define an unavailable extended -floating spelling, such as `_Float32`, through a compatibility `typedef`. -PRIK accepts those declarations as parsing context so that an unrelated private -header declaration does not block a reviewed public surface. This tolerance -does not add direct-wrapper support for the extended floating type itself. -Prototype parameters may also omit their names: a declaration such as -`long rinttol(double)` remains a modern prototype and is not treated as a K&R -definition. Actual K&R definitions remain unsupported. - -## Exact native scalar identities - -Generated C contracts are target-specific and representation-based. Distinct C -types such as `long` and `long long` may therefore use the same public NumPy -contract type. When their exact identity matters to the call, generation keeps -it as a sparse operator inside `@native_call(...)`: - -```python -from prik.contracts import Arg, CLongLong, Float64, Int64, Return, native_call - -@native_call([Arg(0)], result=CLongLong(Return(0))) -def llround(value: Float64) -> Int64: ... -``` - -The public signature continues to use ordinary NumPy contract types. Scalars -and scalar addresses accept exactly that public dtype and are converted -directionally; the native C spelling does not add a second accepted Python -scalar type. Ranked arguments instead require the corresponding exact NumPy -element storage so the pointer path remains zero-copy. See -[Calls and Results: Preserve an Exact C Scalar at the Native -Call](../reference/pyi-contracts/calls-and-results.md#preserve-an-exact-c-scalar-at-the-native-call) -for arguments, addresses, results, arrays, and the supported exact-storage -rules. - -## Symbols your binding's own headers declare - -Exact native scalar identities make compatible duplicate declarations harmless, but -they cannot resolve a genuine identifier collision: a header included by the -binding may already declare the same name for a different API. Name that symbol -to isolate it from `Python.h`: - -```bash -python3 -m prik --language c vendor.pyi \ - --native-library vendor \ - --collision-adapter evaluate \ - --out vendor_api --out-dir build -``` - -The build writes a separate adapter translation unit that includes no Python -header. Its signature is reconstructed from the completed exact native C types -and it only forwards to the original symbol: - -```c -long long evaluate(double x); - -long long prik_collision_adapter_evaluate(double x) { - return (evaluate)(x); -} -``` - -The adapter targets a real function symbol; PRIK does not expose macros. The -forwarder has hidden visibility, so it is not part of the extension's exported -ABI. It is correct with or without the shared `--lto` build optimization. Use -`--collision-adapter-all` to adapt every eligible function instead of naming -each one. Only functions declared by C source inputs are eligible. - -This isolates a declaration collision inside the binding translation unit. It -does not choose between two different linked libraries that both export the -same external symbol; normal target linker and loader resolution must already -select the intended implementation. - -A source-free `.pyi` must preserve every exact native scalar identity needed by -the declaration. A target-generated contract does this automatically; an -edited contract uses the same `@native_call` operators explicitly. See [CLI -Commands](../reference/cli-commands.md#wrapper-builds) for complete selection, -validation, and LTO behavior. - -## Current limits - -PRIK rejects these forms rather than guessing their ABI or memory contract: - -- callbacks and function pointers; `struct`, `union`, and C global-state - wrappers; and enum constants; -- variadic functions, `static` symbols, unsupported calling conventions, - `volatile`, and `_Atomic` values; -- pointer results, multi-level pointers, raw or nullable pointers, and APIs - with retained or ownership-sensitive pointers; -- arrays of strings, Boolean arrays, native C array declarators, arrays outside - ranks 1–15, and Fortran-ordered C arrays. - -For a feature-by-feature view, see the [language support -matrix](feature-matrix.md). The C parser can inspect a broader set of -declarations than the supported wrapper subset; use its output to understand -source, not as a build promise. - -## Build and inspect APIs - -The examples above use the CLI. For application and test code, use -`build_c_extension()` for source builds or `build_pyi_extension()` for authored -contracts, then import the returned `WrapperBuildResult`. See the -[Python API](../reference/python-api.md) for those calls and [CLI -Commands](../reference/cli-commands.md) for build, generation, Makefile, and -inspection options. - -### Native dependencies - -Pass public C source files as positional inputs. Add implementation-only C -files with `--native-c-sources`, compiler flags with -`--native-c-compile-flags`, existing objects with `--native-objects`, and -libraries with `--native-library` and `--native-library-dir`. These complete -the native link without becoming Python API declarations. - -For headers and conditional source, pass the same preprocessing information as -the native project: `-I`, `-D`, `--std`, and, when available, -`--compile-commands build/compile_commands.json`. - -To wrap a reviewed subset of a broad or system header, keep included files -private and select the exact reachable functions from a file: - -```bash -python3 -m prik generate --pyi --language c api_probe.h \ - --include-exposure roots-only \ - --export-symbols reviewed_functions.txt \ - --out contracts/api.pyi -``` - -The export file names the reviewed functions that become public, including -functions declared by an otherwise-private system header. Every unlisted -declaration is excluded. This selects the semantic API rather than linker -exports: selected functions still need native link inputs and a signature -supported by the C wrapper. See [CLI Commands: C include -exposure](../reference/cli-commands.md#c-include-exposure) for the file format -and fail-closed validation rules. - -The Python build API accepts the already-resolved names instead of a CLI text -file: - -```python -build = build_c_extension( - "api_probe.c", - export_symbols=("evaluate", "normalize"), - native_libraries=("vendor",), -) -``` - -### Inspect a broader C API - -The C parser and contract generator accept more syntax than the supported -wrapper subset. Use them to examine declarations, not as a promise that each -declaration can be built: - -```bash -python3 -m prik parse --language c include/library.h --json -python3 -m prik semantics --language c include/library.h -python3 -m prik generate --pyi --language c include/library.h --out contracts/library.pyi -``` - -For a project header that needs its normal preprocessing configuration: - -```bash -python3 -m prik parse --language c include/library.h \ - -I include \ - -D LIBRARY_ENABLE_FAST=1 \ - --std c11 \ - --compile-commands build/compile_commands.json -``` - -Only declarations in the wrapped translation unit become a source build's -public API; headers supply declarations and preprocessing context. For the -broader Fortran wrapper surface, start with the [User Guide](../guide/index.md). +| Functions and primitive values | Externally linked functions with `void`, target-probed arithmetic scalars, and C99 complex values. | [Functions and Scalars](../guide/c/functions-and-scalars.md) | +| Pointers and arrays | One-level primitive pointer parameters start as runtime-rank NumPy storage accepting ranks 0 through 15 with any strides, and can be narrowed to contiguous storage, scalar addresses, exact ranks, or projected results. | [Pointers, Arrays, and Strings](../guide/c/pointers-arrays-and-strings.md) | +| Strings | Rank-zero C string inputs and caller-owned string storage. | [Pointers, Arrays, and Strings](../guide/c/pointers-arrays-and-strings.md#pass-c-strings) | +| Outputs and errors | Returned or hidden output storage, status projection, and Python exception construction. | [Outputs and Errors](../guide/c/outputs-and-errors.md) | +| Names and overloads | Symbol renaming, reordered arguments, typed literals, derived lengths and shapes, and overloads distinguishable by supported dtype or rank. | [Functions and Scalars](../guide/c/functions-and-scalars.md#rename-and-reorder-arguments), [Symbols, Headers, and Dependencies](../guide/c/symbols-headers-and-dependencies.md#present-several-c-symbols-as-one-python-name) | +| Projects and native builds | C source and header preprocessing, explicit source, object, and library dependencies, selected collision forwarders, and `@nogil` calls that do not access Python state. | [Symbols, Headers, and Dependencies](../guide/c/symbols-headers-and-dependencies.md), [Building the Shared Library](../guide/building-shared-library.md) | +| Editable Python interfaces | Generated or authored semantic `.pyi` contracts can select, rename, reorder, and reshape the documented C wrapper surface. | [Editing `.pyi` Contracts](../reference/pyi-contracts/index.md) | + +## Important Boundaries + +The table above names supported areas, not blanket support for every related C +declaration: + +- C pointer syntax does not identify one value, an array, caller-owned + storage, or an output. The semantic `.pyi` contract supplies that meaning. +- A `const T *` declaration must not be authored as writable storage. + Attributes that can change ABI, symbol identity, layout, or calling + convention stop the build instead of being ignored. +- Pointer results, multi-level pointers, raw or nullable pointers, and APIs + with retained or ownership-sensitive pointers are unsupported. +- Callbacks, function pointers, `struct`, `union`, global-state wrappers, enum + constants, variadic functions, `static` symbols, `volatile`, and `_Atomic` + values are unsupported. +- Arrays of strings, Boolean arrays, native C array declarators, and arrays + above rank 15 are unsupported. An explicitly shaped array requires + C-contiguous storage; a Fortran-ordered or strided actual is accepted only + through stride-agnostic `T[...]` storage. +- Parser and contract-generation coverage is broader than wrapper coverage; + successfully inspecting a declaration is not a build promise. + +The [unsupported and blocked forms](feature-matrix.md#unsupported-or-blocked-forms) +table gives the complete feature-by-feature boundary and links to its evidence. +See [C wrapper diagnostics](../reference/diagnostic-codes.md#c-wrapper-diagnostics) +for build rejections. + +## Source Files And Public Entry Points + +C source and header inputs require `--language c`. A source build exposes +supported externally linked functions from the selected translation units at +the Python extension root. C has no native module namespace, so a generated C +contract is one `.pyi` file rather than the Fortran contract-package layout. + +Headers provide declarations and preprocessing context. Included declarations +remain private unless the selected include-exposure policy makes them public. +Implementation-only sources, objects, archives, and libraries must be supplied +explicitly; PRIK does not discover native dependencies. + +For broad or system headers, select the reviewed public functions before +building. The [Symbols, Headers, and +Dependencies](../guide/c/symbols-headers-and-dependencies.md) guide shows source, +library, preprocessing, and API-inspection workflows. The [C User +Guide](../guide/c/index.md) provides the complete path from a C declaration to +an imported Python extension. diff --git a/docs/user/language-support/feature-matrix.md b/docs/user/language-support/feature-matrix.md index 058cacaf1..4e423566a 100644 --- a/docs/user/language-support/feature-matrix.md +++ b/docs/user/language-support/feature-matrix.md @@ -11,8 +11,8 @@ publication: reviewed This matrix is the user-facing support index for native-language features. It points each feature to its user guide, evidence, and limitations. Start with -[C Support](c-support.md) for the complete C workflow, or the [User -Guide](../guide/index.md) for the broader Fortran workflow. +the [User Guide](../guide/index.md) for Fortran and C workflows, and use [C +Support](c-support.md) for the exact C boundary. A row may claim support only when the linked evidence proves that behavior in the current repository. Runtime wrapper support requires compiled, imported, @@ -29,7 +29,7 @@ exact C boundary. | Capability | Fortran | C | | --- | --- | --- | | Primitive arguments and results | Supported for all documented kinds | Supported for target-probed arithmetic and C99 complex types, including `void` results | -| NumPy arrays | Supported with documented rank, shape, layout, stride, and mutation contracts | Supported for ranks 1–15 with primitive non-Boolean elements and C-contiguous storage | +| NumPy arrays | Supported with documented rank, shape, layout, stride, and mutation contracts | Supported for ranks 0–15 with primitive non-Boolean elements; explicit shapes require C-contiguous storage and `T[...]` accepts any strides | | Strings | Supported for scalars and fixed-width arrays | Supported for rank-zero inputs and caller-owned storage | | Procedures and API shaping | Functions, subroutines, modules, state, optional arguments, generics, and defined operators | Externally linked functions, output and status projection, renaming, argument reordering, and dtype/rank overloads | | Pointers and managed storage | Allocatable arrays are supported; pointer arrays are partially supported | One-level primitive pointer parameters support scalar addresses, rank-zero storage, projected results, and arrays | @@ -83,12 +83,12 @@ where they apply. | Feature | Status | User docs | Evidence | Limitations | | --- | --- | --- | --- | --- | -| Primitive scalar calls | Supported | [Build a scalar C function](c-support.md#build-a-scalar-c-function) | [C scalar runtime](../../../tests/c/primitive_scalars/end_to_end/test_direct_c_scalar_matrix.py) | Covers target-probed arithmetic and C99 complex scalars, including `void` results. Public NumPy dtypes remain canonical while the native boundary preserves exact C scalar identities. | -| One-level primitive pointer parameters | Supported | [Choose the pointer contract](c-support.md#choose-the-pointer-contract) | [Pointer contracts](../../../tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py) | Covers scalar addresses, rank-zero storage, projected results, and C-contiguous arrays. Pointer results, multi-level pointers, and nullable or ownership-sensitive pointers remain unsupported. | -| NumPy array arguments | Supported | [Author a contract for pointers and arrays](c-support.md#author-a-contract-for-pointers-and-arrays) | [Pointer contracts](../../../tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py) | Covers ranks 1–15 with primitive non-Boolean elements and C-contiguous storage. PRIK validates dtype, rank, shape, layout, and writeability before the call. | -| Rank-zero C strings | Supported | [Pass C strings](c-support.md#pass-c-strings) | [String contracts](../../../tests/c/primitive_strings/end_to_end/test_direct_c_strings.py) | Covers string inputs and caller-owned rank-zero storage. Arrays of strings remain unsupported. | -| Output, status, naming, and overload projection | Supported | [Rename and reorder arguments](c-support.md#rename-reorder-and-address-arguments), [return several outputs](c-support.md#return-several-c-outputs), [raise Python exceptions](c-support.md#hide-native-outputs-and-raise-python-exceptions), [overload sets](c-support.md#present-several-c-symbols-as-one-python-name) | [Projection and overload evidence](../../../tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py), [hidden-output and status evidence](../../../tests/c/primitive_strings/end_to_end/test_direct_c_strings.py) | Covers hidden outputs, status projection, symbol renaming, argument reordering, typed literals, derived lengths and shapes, and overload sets distinguishable by dtype and rank. | -| C source, header, and semantic-contract builds | Supported | [Build and inspect APIs](c-support.md#build-and-inspect-apis) | [C build pipeline](../../../tests/c/infrastructure/building/pipeline/test_c_build_cli.py), [collision forwarder](../../../tests/c/symbol_collisions/end_to_end/test_collision_adapter_runtime.py) | Supports ordinary compiler preprocessing, including standard includes and macros, plus explicit native dependencies. A selected collision adapter isolates a binding-header name conflict; it is not an ABI fallback. | +| Primitive scalar calls | Supported | [Build a scalar C function](../guide/c/functions-and-scalars.md#build-a-scalar-c-function) | [C scalar runtime](../../../tests/c/primitive_scalars/end_to_end/test_direct_c_scalar_matrix.py) | Covers target-probed arithmetic and C99 complex scalars, including `void` results. Public NumPy dtypes remain canonical while the native boundary preserves exact C scalar identities. | +| One-level primitive pointer parameters | Supported | [Choose the pointer contract](../guide/c/pointers-arrays-and-strings.md#choose-the-pointer-contract) | [Pointer contracts](../../../tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py) | Generated primitive pointers use stride-agnostic runtime-rank storage. Contracts can narrow them to contiguous storage, scalar addresses, exact ranks, rank-zero storage, or projected results. Pointer results, multi-level pointers, and nullable or ownership-sensitive pointers remain unsupported. | +| NumPy array arguments | Supported | [Author a contract for pointers and arrays](../guide/c/pointers-arrays-and-strings.md#author-a-contract-for-pointers-and-arrays) | [Pointer contracts](../../../tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py) | Runtime-rank pointer storage accepts ranks 0–15 with any strides; explicit positive-rank arrays accept ranks 1–15 and require C-contiguous storage. Elements are primitive and non-Boolean, and PRIK validates dtype, rank, declared shape, layout, and writeability before the call. | +| Rank-zero C strings | Supported | [Pass C strings](../guide/c/pointers-arrays-and-strings.md#pass-c-strings) | [String contracts](../../../tests/c/primitive_strings/end_to_end/test_direct_c_strings.py) | Covers string inputs and caller-owned rank-zero storage. Arrays of strings remain unsupported. | +| Output, status, naming, and overload projection | Supported | [Rename and reorder arguments](../guide/c/functions-and-scalars.md#rename-and-reorder-arguments), [return several outputs](../guide/c/outputs-and-errors.md#return-several-c-outputs), [raise Python exceptions](../guide/c/outputs-and-errors.md#hide-native-outputs-and-raise-python-exceptions), [overload sets](../guide/c/symbols-headers-and-dependencies.md#present-several-c-symbols-as-one-python-name) | [Projection and overload evidence](../../../tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py), [hidden-output and status evidence](../../../tests/c/primitive_strings/end_to_end/test_direct_c_strings.py) | Covers hidden outputs, status projection, symbol renaming, argument reordering, typed literals, derived lengths and shapes, and overload sets distinguishable by dtype and rank. | +| C source, header, and semantic-contract builds | Supported | [Build and inspect APIs](../guide/c/symbols-headers-and-dependencies.md#build-and-inspect-apis) | [C build pipeline](../../../tests/c/infrastructure/building/pipeline/test_c_build_cli.py), [collision forwarder](../../../tests/c/symbol_collisions/end_to_end/test_collision_adapter_runtime.py) | Supports ordinary compiler preprocessing, including standard includes and macros, plus explicit native dependencies. A selected collision adapter isolates a binding-header name conflict; it is not an ABI fallback. | ## Inspection And Contract Support @@ -105,7 +105,7 @@ where they apply. | Feature | Status | User docs | Evidence | Limitations | | --- | --- | --- | --- | --- | -| C parse, semantic IR, and `.pyi` inspection | Partially supported | [C Support](c-support.md#build-and-inspect-apis) | [C parser fixtures](../../../tests/c/infrastructure/parsing/test_c_fixture_suite.py), [C semantic tests](../../../tests/c/infrastructure/semantic_ir/semantics/) | Parser coverage is broader than the supported C wrapper subset; parser acceptance is not a runtime-support claim. | +| C parse, semantic IR, and `.pyi` inspection | Partially supported | [Inspect a broader C API](../guide/c/symbols-headers-and-dependencies.md#inspect-a-broader-c-api) | [C parser fixtures](../../../tests/c/infrastructure/parsing/test_c_fixture_suite.py), [C semantic tests](../../../tests/c/infrastructure/semantic_ir/semantics/) | Parser coverage is broader than the supported C wrapper subset; parser acceptance is not a runtime-support claim. | ### Shared @@ -137,8 +137,8 @@ documented diagnostic-stage exception below. | Feature | Status | User docs | Evidence | Limitations | | --- | --- | --- | --- | --- | -| Callbacks and function pointers | Unsupported | [C Support](c-support.md#current-limits) | [C policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [prebuild callback rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | C callback and function-pointer parameters may be parsed, but wrapper policy rejects them before planning. | -| Aggregates, global state, and enum constants | Unsupported | [C Support](c-support.md#current-limits) | [Aggregate, global-state, and enum rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | `struct` and `union` wrappers, native global state, and enum constants are not exposed by C wrappers. | -| Variadics, local symbols, qualifiers, and calling conventions | Unsupported | [C Support](c-support.md#current-limits) | [C policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [prebuild qualifier and declaration rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | Variadic functions, `static` symbols, `volatile` or `_Atomic` values, and unsupported calling conventions fail before wrapper planning. | -| Pointer results and ownership-sensitive pointers | Unsupported | [C Support](c-support.md#current-limits) | [Pointer policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [prebuild raw-address rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | Pointer results, multi-level pointers, raw or nullable pointers, and APIs with retained or ownership-sensitive pointers are unsupported. | -| Unsupported C array and string forms | Unsupported | [C Support](c-support.md#current-limits) | [Array policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [string contract blockers](../../../tests/c/primitive_strings/end_to_end/test_direct_c_strings.py) | Arrays of strings, Boolean arrays, native C array declarators, ranks outside 1–15, and non-C-contiguous arrays are unsupported. | +| Callbacks and function pointers | Unsupported | [C Support](c-support.md#important-boundaries) | [C policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [prebuild callback rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | C callback and function-pointer parameters may be parsed, but wrapper policy rejects them before planning. | +| Aggregates, global state, and enum constants | Unsupported | [C Support](c-support.md#important-boundaries) | [Aggregate, global-state, and enum rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | `struct` and `union` wrappers, native global state, and enum constants are not exposed by C wrappers. | +| Variadics, local symbols, qualifiers, and calling conventions | Unsupported | [C Support](c-support.md#important-boundaries) | [C policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [prebuild qualifier and declaration rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | Variadic functions, `static` symbols, `volatile` or `_Atomic` values, and unsupported calling conventions fail before wrapper planning. | +| Pointer results and ownership-sensitive pointers | Unsupported | [C Support](c-support.md#important-boundaries) | [Pointer policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [prebuild raw-address rejection](../../../tests/c/infrastructure/building/pipeline/test_c_direct_rejections.py) | Pointer results, multi-level pointers, raw or nullable pointers, and APIs with retained or ownership-sensitive pointers are unsupported. | +| Unsupported C array and string forms | Unsupported | [C Support](c-support.md#important-boundaries) | [Array policy blockers](../../../tests/c/primitive_scalars/policy/test_direct_c_policy.py), [string contract blockers](../../../tests/c/primitive_strings/end_to_end/test_direct_c_strings.py) | Arrays of strings, Boolean arrays, native C array declarators, and ranks above 15 are unsupported. A non-C-contiguous actual is accepted only through stride-agnostic `T[...]` storage. | diff --git a/docs/user/language-support/index.md b/docs/user/language-support/index.md index 0a236648d..1e950538b 100644 --- a/docs/user/language-support/index.md +++ b/docs/user/language-support/index.md @@ -14,8 +14,9 @@ publication: reviewed - [Fortran Support](fortran-support.md) maps supported wrapper areas to their detailed guides and records the source files and program units that become Python APIs. -- [C Support](c-support.md) is the complete workflow for C projects. Current - C wrapper coverage is the supported subset documented on that page. +- [C Support](c-support.md) records the supported C wrapper surface and its + boundaries. The [C User Guide](../guide/c/index.md) teaches the build and + contract workflow. - The [language feature matrix](feature-matrix.md) is the authoritative Fortran-and-C index for supported, partially supported, and unsupported features. diff --git a/docs/user/reference/cli-commands.md b/docs/user/reference/cli-commands.md index 92bd984df..1aa466746 100644 --- a/docs/user/reference/cli-commands.md +++ b/docs/user/reference/cli-commands.md @@ -65,7 +65,7 @@ Compiled wrapper builds support Fortran and the documented C subset — scalars, one-level primitive pointers, arrays, rank-zero strings, hidden outputs, and status projection. C paths require `--language c`; the parser accepts more C forms than that runtime subset, and those fail before wrapper -planning. [C Support](../language-support/c-support.md#what-is-supported) +planning. [C Support](../language-support/c-support.md#supported-wrapper-areas) records the exact boundary. Directories are expanded recursively in deterministic path order. Fortran @@ -384,6 +384,6 @@ which has a complete source, build, import, and result flow. ## Related pages - [Python API Reference](python-api.md) — the same workflows from Python. -- [C Support](../language-support/c-support.md) — the complete C - source, contract, build, and Python workflows. +- [C User Guide](../guide/c/index.md) — C source, contract, build, and Python + workflows. - [Editing `.pyi` Contracts](pyi-contracts/index.md) — supported contract edits. diff --git a/docs/user/reference/diagnostic-codes.md b/docs/user/reference/diagnostic-codes.md index 3a7b92cbf..b8c09a305 100644 --- a/docs/user/reference/diagnostic-codes.md +++ b/docs/user/reference/diagnostic-codes.md @@ -199,5 +199,5 @@ argument, or declaration. | `C_DIRECT_NATIVE_GLOBAL_STATE`, `C_DIRECT_ENUM_CONSTANT`, `C_DIRECT_MACRO_CONSTANT` | Native global state and constants are not exposed by C wrappers. | | `C_DIRECT_UNMODELED_DECLARATION` | A declaration would otherwise be omitted from a C wrapper build. | -See [C Support](../language-support/c-support.md#current-limits) for the +See [C Support](../language-support/c-support.md#important-boundaries) for the supported boundary and the repair choices. diff --git a/docs/user/reference/index.md b/docs/user/reference/index.md index b55634bd1..e6373325b 100644 --- a/docs/user/reference/index.md +++ b/docs/user/reference/index.md @@ -44,8 +44,10 @@ the changed path once. - [Fortran Support](../language-support/fortran-support.md) — the complete map of supported Fortran wrapper areas, limits, and detailed guides. -- [C Support](../language-support/c-support.md) — the complete C source, - contract, build, generated API, and support-boundary workflow. +- [C User Guide](../guide/c/index.md) — C source, contract, build, and generated + API workflows. +- [C Support](../language-support/c-support.md) — supported C features and + boundaries. - [Language feature matrix](../language-support/feature-matrix.md) — whether a feature is supported at all, with its evidence. - [Diagnostic codes](diagnostic-codes.md) — what a rejected wrapper is telling diff --git a/docs/user/reference/pyi-contracts/calls-and-results.md b/docs/user/reference/pyi-contracts/calls-and-results.md index 248449e4e..f585d2655 100644 --- a/docs/user/reference/pyi-contracts/calls-and-results.md +++ b/docs/user/reference/pyi-contracts/calls-and-results.md @@ -82,14 +82,42 @@ from prik.contracts import Arg, CLongLong, Float64, Int64, native_call def accumulate(count: Int64, scale: Float64) -> None: ... ``` -The public annotation is authoritative: the user passes a NumPy `int64`, not a -`numpy.longlong` merely because `CLongLong` appears in `@native_call`. The -binding extracts the public value into `int64_t`, then emits the native call as: +The public annotation is authoritative: the user passes a NumPy `int64` and +never has to spell `numpy.longlong` merely because `CLongLong` appears in +`@native_call`. The binding extracts the public value into `int64_t`, then +emits the native call as: ```c accumulate((long long)contract_count, contract_scale); ``` +A scalar crosses by value, so the binding converts it: either NumPy spelling +of that width is accepted. `np.int64` and `np.longlong` both satisfy a `long +long` parameter, and both satisfy a `long` parameter, whichever one the target +calls `int64_t`. + +An array is different. Its elements are handed to C as the buffer they already +are, and a buffer cannot be converted element by element, so an array argument +whose `@native_call` entry names an exact C identity accepts **only** that +element dtype: + +```python +@native_call([CLongLong(Arg(0)), Arg(1)]) +def increment(values: Int64[:], count: Int32) -> None: ... +``` + +```python +increment(np.array([1, 2, 3], dtype=np.longlong), np.int32(3)) # accepted +increment(np.array([1, 2, 3], dtype=np.int64), np.int32(3)) # TypeError +``` + +A generated array contract follows the same rule from the C source alone: the +accepted dtype is that of the declared C element type, so a `long long *` +buffer wants `numpy.longlong` and a `long *` buffer wants `numpy.int64` on +every target, whichever one that target calls `int64_t`. The generated +docstring names the exact dtype, so check it when a `long`/`long long` +distinction is in play. + The same sparse form records a native function result whose C identity was lost by width-based normalization: diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index cb90eff96..956acc089 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -590,8 +590,9 @@ Here Python receives `(a, b)` and returns `status`; the native procedure receive | `Int32(1)`, `Float64(0.5)`, `Bool(False)` | Typed hidden primitive literal. | | `String[1]("N")` | Typed hidden one-character literal. | | `Len(Arg(i))`, `Len(Return(i))`, `Len(Work("name"))` | Hidden native character length. | +| `Arg(i).size` | Hidden `SizeT` total element count for array argument `i`. | | `Arg(i).shape[d]`, `Return(i).shape[d]`, `Work("name").shape[d]` | Hidden `SizeT` extent for axis `d`. | -| `Arg(i).strides[d]`, `Return(i).strides[d]`, `Work("name").strides[d]` | Hidden `SizeT` stride for axis `d`. | +| `Arg(i).strides[d]`, `Return(i).strides[d]`, `Work("name").strides[d]` | Hidden `SizeT` stride for axis `d`, in bytes, as `ndarray.strides` reports it. | | `Int32(Arg(i).shape[d])` | The same extent, materialized as the named integer type. | | `IsPresent(Arg(i))` | Hidden optional-presence flag. | | `Work("name")` | Named hidden workspace. | @@ -622,12 +623,12 @@ The binding materializes the extent as that type directly, and the generated Fortran dummy becomes `integer(c_int32_t)`, so a default Fortran `integer` parameter no longer has to stay visible in the Python signature. -The same form applies to `strides[d]` and `Len(...)`. Fixed-width signed and -unsigned integer contract types, plus `SizeT`, are accepted. The unresolved -`Int` and `UInt` names and all real, Boolean, and character types are rejected -during policy completion. The conversion is not range-checked, so an extent -wider than the stated type wraps rather than raising. Use a type that matches -the native parameter. +The same form applies to `.size`, `strides[d]`, and `Len(...)`. Fixed-width +signed and unsigned integer contract types, plus `SizeT`, are accepted. The +unresolved `Int` and `UInt` names and all real, Boolean, and character types +are rejected during policy completion. The conversion is not range-checked, +so an extent wider than the stated type wraps rather than raising. Use a type +that matches the native parameter. A character length has a compiler-fixed ABI. Restate it only when the native declaration genuinely takes a different integer, not to change how PRIK passes @@ -717,7 +718,7 @@ stores or passes it: | `T[:]` | Rank-one array with runtime extent. | Shared. | | `T[:, :]` | Rank-two array with runtime extents. | Shared. | | `T[::]` | Rank-one stride-aware array. | Shared. | -| `T[...]` | Assumed-rank storage. | Accepted; build support is feature-specific. | +| `T[...]` | Runtime-rank NumPy storage. C accepts ranks 0–15 with any strides; Fortran assumed-rank support is feature-specific. | Shared. | | `T[Flat]`, `T[n, Flat]` | Contiguous assumed-size/flat-edge storage. | Primarily Fortran and flat-buffer APIs. | | `Addr(T)` | Python integer carrying a raw native address to `T`. | Shared low-level form. | | `Addr[n](T)` | Pointer depth greater than one. | Loaded for low-level declarations; callable support is limited. | @@ -781,7 +782,8 @@ completed native handle. For C, a pointer annotation alone cannot determine whether Python should see a scalar address, rank-zero storage, an array, a result, or an opaque address. Express that choice with `Addr(T)`, `T[()]`, `T[...]`, `Returns[...]`, and -`@native_call` as described in [C Support](../language-support/c-support.md). +`@native_call` as described in [C Pointers, Arrays, and +Strings](../guide/c/pointers-arrays-and-strings.md). ### Allocatable Array Handles @@ -831,7 +833,7 @@ type. Metadata falls into four groups. | --- | --- | --- | | `ORDER_C`, `ORDER_F`, `ORDER_ANY` | Array orientation. | Author only when it differs from the language default, or when either order is intended. | | `COPY_F` | Accept C-order Python storage, use an F-order temporary, and copy back visible mutation. | Focused multidimensional array support. | -| `Contiguous` | Native declaration promises contiguity. | Loaded source-provenance/compatibility fact. | +| `Contiguous` | Storage is contiguous. | Loaded source-provenance/compatibility fact; on `T[...]` it narrows stride-agnostic runtime-rank storage to the language's contiguous order. | | `Aliased` | Native storage is addressable or may be exposed as an alias. | Fortran `target` and borrowed native objects. | | `Immutable` | Python-visible value is replace-only, not mutated in place. | Requires a compatible replacement/copy policy for writable native storage. | | `Polymorphic` | Fortran native declaration is `class(T)`. | Fortran. | @@ -954,13 +956,13 @@ entry-contract imports and aliases shape the public Python namespace. | Native language selection | Inferred from normal Fortran `.pyi` generation/build context. | Pass `--language c` for a source-free contract. | | Native scope | Leaf filename is native module; `@standalone` marks external procedures. | Functions are external symbols; file/import structure organizes the contract and Python API. | | Default multidimensional order | `ORDER_F`. | `ORDER_C`. | -| Scalar reference input | Generated visible `T` plus `Addr(Arg(i))`, or explicit `T[()]` storage. | Pointer meaning must be authored as scalar address, storage, array, output, or raw address. | +| Scalar reference input | Generated visible `T` plus `Addr(Arg(i))`, or explicit `T[()]` storage. | A primitive `T *` starts as runtime-rank `T[...]` storage; edit it to a scalar address, exact rank, projected result, or raw address as needed. | | ABI marker | `@native_abi("c")` preserves Fortran `bind(C)`. | Invalid because C language identity already supplies the ABI. | | Exact native scalar identity | Usually carried by resolved Fortran type/kind. | C cast helpers inside `@native_call` preserve spelling such as `long long`. | | Classes | Wrapped derived types, fields, methods, inheritance, constructors, finalizers. | Struct/union/anonymous/opaque classes are currently inspection forms, not aggregate wrappers. | | Module variables | Supported Fortran module-state subset. | Native C globals are inspection-only. | | Callbacks/prototypes | Supported Fortran callback subset through `@prototype`. | C function pointers are not currently buildable callbacks. | -| Arrays and strings | Full documented Fortran wrapper mechanisms. | Primitive C-contiguous arrays and rank-zero strings in the documented C subset. | +| Arrays and strings | Full documented Fortran wrapper mechanisms. | Stride-agnostic runtime-rank storage, explicitly shaped C-contiguous arrays, and rank-zero strings in the documented C subset. | ### Fortran Wrapper Contracts @@ -1010,12 +1012,13 @@ valid and whether it is buildable. | Project layout | Entry plus native-module leaves; compact standalone entry when applicable. | One contract file per selected source or declaration owner. | | Primitive types | Compiler-resolved kinds and storage names. | Compiler-probed types plus exact C identity helpers when needed. | | Functions | Module and standalone procedures, results, outputs, callbacks, overloads. | Functions, primitive pointers, projections, renames, and overload candidates. | -| Arrays and strings | Shapes, order, striding, characters, allocatable and pointer descriptors. | C-order array facts and conservative pointer/string starter contracts. | +| Arrays and strings | Shapes, order, striding, characters, allocatable and pointer descriptors. | Stride-agnostic runtime-rank primitive pointer storage, C-order array facts, and conservative string starter contracts. | | Classes | Derived types, fields, methods, constructors, inheritance, abstract/final roles. | Struct, union, anonymous, and opaque inspection classes. | | Variables/constants | Module variables, parameters, and enum constants. | Global and enum/macro constant inspection declarations. | -Generated output is a conservative starting point. C pointer meaning and any -ownership or Pythonic result projection that native syntax cannot prove must be +Generated output is a conservative starting point. Runtime-rank `T[...]` +avoids choosing a fixed C pointer rank, but native extents, ownership, or +Pythonic result projection that source syntax cannot prove must still be authored explicitly. ## Rejected Or Not Yet Supported diff --git a/docs/user/reference/python-api.md b/docs/user/reference/python-api.md index 1d5b9387b..415ed1f76 100644 --- a/docs/user/reference/python-api.md +++ b/docs/user/reference/python-api.md @@ -81,8 +81,8 @@ print(native_math.add(np.float64(3.0), np.float64(2.5))) ``` For an authored C semantic contract, use `build_pyi_extension` with -`native_language="c"` and `native_c_sources=[...]`. The [C Support -guide](../language-support/c-support.md#author-a-contract-for-pointers-and-arrays) +`native_language="c"` and `native_c_sources=[...]`. [C Pointers, Arrays, and +Strings](../guide/c/pointers-arrays-and-strings.md#author-a-contract-for-pointers-and-arrays) shows the complete contract and build. ## Advanced package imports @@ -98,6 +98,7 @@ Reach past the root facade when you need a single stage rather than a build. | C semantic conversion | `prik.semantics.c2ir` | `CToIRConverter`, `c_file_to_semantic_module`, `c_file_to_semantic_modules` | | `.pyi` loading and stub emission | `prik.pipeline.pyi` | `pyi_*_to_semantic_module`, `emit_module_stubs` | | Build records and results | `prik.pipeline.build` | `WrapperBuildResult`, `NativeBuildPlan`, `NativeCompilationUnit`, `NativePrebuiltArtifact`, `NativeLinkItem` | +| IPython/Jupyter integration | `prik.jupyter` | `%load_ext prik.jupyter`, then `%%fortran`, `%%c`, or `%%pyi` | | Target type probing | `prik.preprocessing.probes.fortran_types` | probe source, requirements, expressions, report and error types | | C target type probing | `prik.preprocessing.probes.c_types` | `probe_c_standard_types`, `probe_c_standard_types_cached`, and C probe records/error type | | Runtime descriptor handles | `prik.runtime.handles` | `NativeArrayHandleBase`, `AllocatableArray`, `PointerArray` | @@ -112,15 +113,16 @@ Reach past the root facade when you need a single stage rather than a build. completion, planning, and generation are separate stages that can each reject input the parser accepted. - C source builds are limited to the documented supported subset recorded in - [C Support](../language-support/c-support.md#what-is-supported). Other + [C Support](../language-support/c-support.md#supported-wrapper-areas). Other parser-accepted C forms fail before wrapper planning rather than falling back to a generated ABI-conversion adapter. ## Related pages - [CLI Commands](cli-commands.md) — the same workflows from a shell. -- [C Support](../language-support/c-support.md) — C source, contract, CLI, and - Python workflows. +- [IPython and Jupyter Notebooks](../guide/notebooks.md) — compile source cells and publish their APIs in the session. +- [C User Guide](../guide/c/index.md) — C source, contract, CLI, and Python + workflows. - [Editing `.pyi` Contracts](pyi-contracts/index.md) — supported API-shaping edits. - [Package guides](../../developer/packages/index.md) — module responsibilities diff --git a/mkdocs.yml b/mkdocs.yml index cfff27da3..d693c45e9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -43,27 +43,35 @@ nav: - Installation: user/getting-started/installation.md - Verification: user/getting-started/verification.md - First Wrapped Function: user/getting-started/first-wrapped-function.md - - First Wrapped Module: user/getting-started/first-wrapped-module.md - Common Beginner Workflow: user/getting-started/beginner-workflow.md - User Guide: - Overview: user/guide/index.md - - Data Types: user/guide/data-types.md - - Arrays: user/guide/arrays.md - - Strings: user/guide/strings.md - - Wrapping Functions: user/guide/wrapping-functions.md - - Wrapping Subroutines: user/guide/wrapping-subroutines.md - - Wrapping Modules: user/guide/wrapping-modules.md - - Optional Arguments: user/guide/optional-arguments.md - - Generic Interfaces (Overloading): user/guide/generic-interfaces.md - - Wrapping Derived Types: user/guide/wrapping-derived-types.md - - Allocatables: user/guide/allocatables.md - - Pointers: user/guide/pointers.md - - Memory Management: user/guide/memory-management.md - - Callbacks: user/guide/callbacks.md - - Enumerations: user/guide/enumerations.md - - Raw Addresses: user/guide/raw-addresses.md - - Error Handling & Diagnostics: user/guide/error-handling.md - - Building the Shared Library: user/guide/building-shared-library.md + - Fortran: + - Data Types: user/guide/data-types.md + - Arrays: user/guide/arrays.md + - Strings: user/guide/strings.md + - Wrapping Functions: user/guide/wrapping-functions.md + - Wrapping Subroutines: user/guide/wrapping-subroutines.md + - Wrapping Modules: user/guide/wrapping-modules.md + - Optional Arguments: user/guide/optional-arguments.md + - Generic Interfaces (Overloading): user/guide/generic-interfaces.md + - Wrapping Derived Types: user/guide/wrapping-derived-types.md + - Allocatables: user/guide/allocatables.md + - Pointers: user/guide/pointers.md + - Memory Management: user/guide/memory-management.md + - Callbacks: user/guide/callbacks.md + - Enumerations: user/guide/enumerations.md + - Raw Addresses: user/guide/raw-addresses.md + - Error Handling & Diagnostics: user/guide/error-handling.md + - C: + - Overview: user/guide/c/index.md + - Functions and Scalars: user/guide/c/functions-and-scalars.md + - Pointers, Arrays, and Strings: user/guide/c/pointers-arrays-and-strings.md + - Outputs and Errors: user/guide/c/outputs-and-errors.md + - Symbols, Headers, and Dependencies: user/guide/c/symbols-headers-and-dependencies.md + - Build Workflows: + - Building the Shared Library: user/guide/building-shared-library.md + - IPython and Jupyter Notebooks: user/guide/notebooks.md - Tutorials: - Design a Pythonic BLAS API: user/tutorials/pythonic-blas.md - Examples: diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index 4bdcf0081..55c01ac5f 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -12,6 +12,7 @@ from collections.abc import Mapping from dataclasses import dataclass, replace import re +from typing import ClassVar from prik.utilities.declaration_expressions import declaration_extent_uses_power, render_declaration_extent from prik.policy.ownership import ( @@ -22,6 +23,7 @@ ) from prik.policy.models import ( ArgumentHandoffMode, + ArrayPythonLayout, CallbackABIKind, CallbackResultAction, CallbackTransferAction, @@ -5993,12 +5995,62 @@ def _visit_FunctionPlan(self, plan: FunctionPlan) -> CFunction: *self._native_output_declarations(plan, context), self._parse_statement(plan, context), *argument_body, + *self._projected_axis_guard_nodes(plan, context), *alias_body, *self._native_call_setup_nodes(plan, context), *output_nodes, ), ) + def _projected_axis_guard_nodes( + self, + plan: FunctionPlan, + context: _CFunctionContext, + ) -> tuple[CExpressionStatement, ...]: + """Require every projected axis to exist on a runtime-rank actual. + + A runtime-rank contract states no rank, so the rank an axis projection + needs is only known once the caller's array arrives. The guard reads + that rank before the projection reads the axis. + """ + guards = {} + for slot in sorted(plan.entrypoint.projected_slots, key=lambda item: item.native_position): + axis = self._projected_runtime_rank_axis(plan, slot) + if axis is None: + continue + argument = next(item for item in plan.arguments if item.python_position == slot.python_position) + names = context.arguments[argument.owner_path] + guards[(names.object_name, axis)] = CExpressionStatement( + CodeExpression( + f"if (PyArray_NDIM((PyArrayObject *){names.object_name}) <= {axis}) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {argument.binding.python_name} ' + f'has no axis {axis}"); return NULL; }}' + ) + ) + return tuple(guards.values()) + + @staticmethod + def _projected_runtime_rank_axis( + plan: FunctionPlan, + slot: NativeEntrypointProjectedSlotPlan, + ) -> int | None: + """Return the projected axis when its source array carries a runtime rank.""" + if slot.projection_action not in { + EntrypointProjectionAction.COMPUTED_SHAPE, + EntrypointProjectionAction.COMPUTED_STRIDE, + }: + return None + if slot.python_position is None or not isinstance(slot.literal_value, Mapping): + return None + argument = next( + (item for item in plan.arguments if item.python_position == slot.python_position), + None, + ) + if argument is None or argument.array is None or argument.array.rank is not None: + return None + axis = slot.literal_value.get("dim") + return axis if isinstance(axis, int) else None + def _function_argument_nodes(self, plan: FunctionPlan, context: _CFunctionContext) -> tuple: """Build function argument nodes from the supplied completed binding records; emitted nodes only project completed binding actions.""" return tuple( @@ -7091,22 +7143,20 @@ def _numeric_array_dtype_selectors(plan: ArgumentTransferPlan) -> tuple[str, str @staticmethod def _array_rank_bounds(handoff: ArrayHandoffPlan) -> tuple[int, int]: """Return inclusive runtime-rank bounds selected by array policy.""" - if handoff.rank is None: - return 1, 15 - if handoff.flatten_python_storage: - return handoff.rank, 15 - return handoff.rank, handoff.rank + return handoff.minimum_rank, handoff.maximum_rank - @staticmethod - def _array_layout_selector(handoff: ArrayHandoffPlan) -> str: - """Return the compact helper layout selector chosen by the plan.""" - if handoff.contiguous is False: - return "PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F" - if handoff.order == "ORDER_C": - return "PRIK_ARRAY_LAYOUT_C_CONTIGUOUS" - if handoff.order == "ORDER_F" or (handoff.rank is not None and handoff.rank > 1): - return "PRIK_ARRAY_LAYOUT_F_CONTIGUOUS" - return "PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS" + _ARRAY_LAYOUT_SELECTORS: ClassVar[dict[ArrayPythonLayout, str]] = { + ArrayPythonLayout.ANY_CONTIGUOUS: "PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS", + ArrayPythonLayout.C_CONTIGUOUS: "PRIK_ARRAY_LAYOUT_C_CONTIGUOUS", + ArrayPythonLayout.F_CONTIGUOUS: "PRIK_ARRAY_LAYOUT_F_CONTIGUOUS", + ArrayPythonLayout.POSITIVE_STRIDED_F: "PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F", + ArrayPythonLayout.ANY_STRIDED: "PRIK_ARRAY_LAYOUT_ANY_STRIDED", + } + + @classmethod + def _array_layout_selector(cls, handoff: ArrayHandoffPlan) -> str: + """Return the compact helper selector for the layout policy completed.""" + return cls._ARRAY_LAYOUT_SELECTORS[handoff.python_layout] def _array_shape_checks( self, @@ -10774,6 +10824,8 @@ def _projected_slot_values( return (f"({spelling}){names.length_name}",) if slot.projection_action is EntrypointProjectionAction.COMPUTED_PRESENCE: return (f"({names.nullable_name} != NULL)",) + if slot.projection_action is EntrypointProjectionAction.COMPUTED_SIZE: + return (f"({spelling})PyArray_SIZE((PyArrayObject *){names.object_name})",) if not isinstance(slot.literal_value, Mapping): raise ValueError(f"Projected array fact {slot.owner_path!r} has no axis metadata") axis = slot.literal_value.get("dim") @@ -11713,29 +11765,33 @@ def _method_table(self, module: ModulePlan, namespace: NamespacePlan) -> CMethod """Build method table from the supplied completed binding records; emitted nodes only project completed binding actions.""" return CMethodDefTable( f"{module.binding.owner_path}_{self._namespace_symbol(namespace)}_methods", - ( - *( - CMethodDefEntry( - function.binding.python_name, - self._binding_function_name(function), - self._binding_method_flags(function), - function.binding.docstring, - ) - for function in namespace.functions - ), - *self._overload_method_entries(namespace), - *( - CMethodDefEntry( - CBindingNames.class_create_method(surface), - CBindingNames.class_create_method(surface), - "METH_VARARGS", - "", - ) - for surface in namespace.classes - if surface.constructor.kind is not ClassConstructorKind.ABSENT - ), - *self._derived_private_method_entries(namespace), + self._method_entries(namespace), + ) + + def _method_entries(self, namespace: NamespacePlan) -> tuple[CMethodDefEntry, ...]: + """Return the exact callable definitions installed in one namespace.""" + return ( + *( + CMethodDefEntry( + function.binding.python_name, + self._binding_function_name(function), + self._binding_method_flags(function), + function.binding.docstring, + ) + for function in namespace.functions + ), + *self._overload_method_entries(namespace), + *( + CMethodDefEntry( + CBindingNames.class_create_method(surface), + CBindingNames.class_create_method(surface), + "METH_VARARGS", + "", + ) + for surface in namespace.classes + if surface.constructor.kind is not ClassConstructorKind.ABSENT ), + *self._derived_private_method_entries(namespace), ) @staticmethod diff --git a/prik/codegen/docstrings.py b/prik/codegen/docstrings.py index 7007fab34..654b71632 100644 --- a/prik/codegen/docstrings.py +++ b/prik/codegen/docstrings.py @@ -12,6 +12,7 @@ from prik.codegen.primitive_scalar_types import NativeCArrayStorageRegistry from prik.policy.ownership import OwnershipOwner, PythonBarrierAction, SetterAction, TransferMode from prik.policy.models import ( + ArrayPythonLayout, ClassConstructorKind, EntrypointOptionalityAction, ModuleGetterAction, @@ -905,9 +906,24 @@ def _base_type(self, transfer) -> str: return f"{prefix}[{scalar}]" if getattr(transfer, "array", None) is not None: element = "bytes" if transfer.semantic_type_name == "String" else scalar - return f"ndarray[{element}]" + return f"ndarray[{self._exact_array_element_label(transfer, element)}]" return scalar + @staticmethod + def _exact_array_element_label(transfer, element: str) -> str: + """Name the exact element dtype when policy requires that C identity. + + An array buffer is handed to C as it stands, so an argument bound to an + exact native C element type accepts only that dtype. The header names + it rather than the canonical spelling the element would otherwise use. + """ + binding = getattr(transfer, "binding", None) + c_type = getattr(binding, "native_array_element_c_type", None) + if c_type is None: + return element + storage = NativeCArrayStorageRegistry.type_for(c_type, transfer.semantic_type_name) + return storage.python_type_name.removeprefix("numpy.") + def _callback_type(self, callback: CallbackHandoffPlan | None) -> str: """Return the named completed prototype for one callback transfer. @@ -948,11 +964,20 @@ def _array_lines(array: ArrayHandoffPlan | None) -> tuple[str, ...]: display_shape = array.display_shape or array.shape if display_shape and all(str(extent) not in _UNKNOWN_EXTENTS for extent in display_shape): lines.append(f" Shape: ({', '.join(map(str, display_shape))})") - if (array.rank is None or array.rank > 1) and array.order in {"ORDER_C", "ORDER_F"}: - layout = "C-contiguous" if array.order == "ORDER_C" else "F-contiguous" + layout = WrapperDocstringBuilder._array_layout_label(array) + if layout is not None: lines.append(f" Layout: {layout}") return tuple(lines) + @staticmethod + def _array_layout_label(array: ArrayHandoffPlan) -> str | None: + """Render the layout every accepted actual must already have, if any.""" + if array.python_layout is ArrayPythonLayout.ANY_STRIDED: + return "Any strides" + if (array.rank is None or array.rank > 1) and array.order in {"ORDER_C", "ORDER_F"}: + return "C-contiguous" if array.order == "ORDER_C" else "F-contiguous" + return None + @staticmethod def _array_rank_line(array: ArrayHandoffPlan) -> str: """Render the rank sentence for ordinary or flattened Python storage. @@ -967,8 +992,8 @@ def _array_rank_line(array: ArrayHandoffPlan) -> str: return " Rank: 1..15, flattened to native rank 1" edge = "leading" if array.flat_axis == 0 else "final" return f" Rank: {native_rank}..15, flattened at {edge} Flat axis to native rank {native_rank}" - if array.rank is None: - return " Rank: 1..15" + if array.minimum_rank != array.maximum_rank: + return f" Rank: {array.minimum_rank}..{array.maximum_rank}" return f" Rank: {array.rank}" @staticmethod diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index 0ff6400cd..f1c9130c1 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -3891,6 +3891,7 @@ def _native_arguments( EntrypointProjectionAction.TYPED_LITERAL, EntrypointProjectionAction.COMPUTED_LENGTH, EntrypointProjectionAction.COMPUTED_PRESENCE, + EntrypointProjectionAction.COMPUTED_SIZE, EntrypointProjectionAction.COMPUTED_SHAPE, EntrypointProjectionAction.COMPUTED_STRIDE, EntrypointProjectionAction.WORK_STORAGE, diff --git a/prik/jupyter/__init__.py b/prik/jupyter/__init__.py new file mode 100644 index 000000000..4a0b80328 --- /dev/null +++ b/prik/jupyter/__init__.py @@ -0,0 +1,28 @@ +"""Optional IPython and Jupyter integration. + +Load this package with ``%load_ext prik.jupyter``. Importing it does not +require IPython until the extension registration hook is called. +""" + + +def load_ipython_extension(ipython) -> None: + """Register PRIK's cell magics without replacing another extension's names.""" + from IPython.core.error import UsageError + + from prik.jupyter.magic import PrikMagics + + find_cell_magic = getattr(ipython, "find_cell_magic", None) + conflicts: list[str] = [] + if callable(find_cell_magic): + for name in PrikMagics.magic_names: + existing = find_cell_magic(name) + owner = None if existing is None else getattr(existing, "__self__", None) + if existing is not None and not getattr(owner, "owns_prik_cell_magics", False): + conflicts.append(f"%%{name}") + if conflicts: + joined = ", ".join(conflicts) + raise UsageError(f"Cannot load PRIK because these cell magics are already registered: {joined}") + ipython.register_magics(PrikMagics) + + +__all__ = ("load_ipython_extension",) diff --git a/prik/jupyter/contracts.py b/prik/jupyter/contracts.py new file mode 100644 index 000000000..30c5754a6 --- /dev/null +++ b/prik/jupyter/contracts.py @@ -0,0 +1,437 @@ +"""Generate, identify, persist, and materialize editable notebook contracts.""" + +from __future__ import annotations + +from collections.abc import Mapping +from dataclasses import dataclass +import json +import os +from pathlib import Path, PurePosixPath +import shlex + +from IPython.core.error import UsageError + +from prik.preprocessing import PreprocessingConfig + + +_GENERATED_CONTRACT_SCHEMA_VERSION = 1 +_GENERATED_CONTRACT_RECORD_NAME = "contracts.json" +_EDITABLE_CONTRACT_PREFIX = "# prik:" + + +@dataclass(frozen=True) +class GeneratedContracts: + """Generated editable contracts associated with one exact native cell.""" + + language: str + source_digest: str + module_contracts: dict[str, str] + direct_contract: str | None + dependency_contracts: dict[str, str] + + +@dataclass(frozen=True) +class EditableContract: + """Notebook metadata plus the editable semantic contract text.""" + + source_digest: str | None + filename: str | None + text: str + + +def _contract_filename(module_name: str) -> str: + """Return the visible `.pyi` path that carries a native module namespace.""" + parts = module_name.split(".") + if not parts or any(not part.isidentifier() for part in parts): + raise ValueError(f"Cannot represent generated module name {module_name!r} as a .pyi filename") + return PurePosixPath(*parts).with_suffix(".pyi").as_posix() + + +def _validated_contract_filename(value: str) -> str: + """Validate one generated module contract path without accepting a package entry.""" + path = PurePosixPath(value) + invalid_part = any(part in {"", ".", ".."} for part in path.parts) + invalid_identifier = any(not part.isidentifier() for part in path.with_suffix("").parts) + if path.is_absolute() or path.suffix != ".pyi" or path.name == "__init__.pyi": + raise UsageError(f"Invalid editable .pyi filename {value!r}") + if invalid_part or invalid_identifier: + raise UsageError(f"Invalid editable .pyi filename {value!r}") + return path.as_posix() + + +def _read_json_record(path: Path) -> dict[str, object] | None: + try: + value = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + return value if isinstance(value, dict) else None + + +def _string_mapping(value: object, *, field_name: str) -> dict[str, str]: + """Validate a JSON mapping whose keys and values must both be strings.""" + if not isinstance(value, dict): + raise ValueError(f"Generated contract cache field {field_name!r} is invalid") + if any(not isinstance(key, str) or not isinstance(text, str) for key, text in value.items()): + raise ValueError(f"Generated contract cache field {field_name!r} is invalid") + return dict(value) + + +def generated_contract_record_path(entry_dir: Path, fingerprint: str) -> Path: + """Return the target-specific generated-contract record for one source.""" + return entry_dir / "generated-contracts" / fingerprint / _GENERATED_CONTRACT_RECORD_NAME + + +def write_generated_contracts(path: Path, contracts: GeneratedContracts) -> None: + """Atomically persist generated contracts needed by later editable cells.""" + record = { + "schema_version": _GENERATED_CONTRACT_SCHEMA_VERSION, + "language": contracts.language, + "source_digest": contracts.source_digest, + "module_contracts": contracts.module_contracts, + "direct_contract": contracts.direct_contract, + "dependency_contracts": contracts.dependency_contracts, + } + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_name(f".{path.name}.{os.getpid()}.tmp") + temporary.write_text(f"{json.dumps(record, sort_keys=True, indent=2)}\n", encoding="utf-8") + temporary.replace(path) + + +def _invalid_cache_message(*, invalid: bool) -> str: + state = "invalid" if invalid else "unavailable or incompatible" + return ( + f"The generated contract cache is {state} for these compiler and build options; " + "execute its %%fortran --pyi or %%c --pyi source cell again with the desired options" + ) + + +def read_generated_contracts( + path: Path, + *, + language: str, + source_digest: str, +) -> GeneratedContracts: + """Load and validate the generated contract bundle for one native cell.""" + record = _read_json_record(path) + identity = ( + None + if record is None + else ( + record.get("schema_version"), + record.get("language"), + record.get("source_digest"), + ) + ) + expected = (_GENERATED_CONTRACT_SCHEMA_VERSION, language, source_digest) + if identity != expected: + raise UsageError(_invalid_cache_message(invalid=False)) + assert record is not None + try: + module_contracts = _string_mapping(record.get("module_contracts"), field_name="module_contracts") + dependency_contracts = _string_mapping( + record.get("dependency_contracts"), + field_name="dependency_contracts", + ) + except ValueError as exc: + raise UsageError(_invalid_cache_message(invalid=True)) from exc + direct = record.get("direct_contract") + if direct is not None and not isinstance(direct, str): + raise UsageError(_invalid_cache_message(invalid=True)) + return GeneratedContracts( + language=language, + source_digest=source_digest, + module_contracts=module_contracts, + direct_contract=direct, + dependency_contracts=dependency_contracts, + ) + + +def _metadata_line(filename: str | None, source_digest: str) -> str: + parts = [_EDITABLE_CONTRACT_PREFIX] + if filename is not None: + parts.append(f"file={filename}") + parts.append(f"source-sha256={source_digest}") + return " ".join(parts) + + +def editable_cell_text( + contract: str, + *, + filename: str | None, + source_digest: str, + magic_command: str, +) -> str: + """Render one complete editable cell including its normal magic command.""" + return f"{magic_command}\n\n{_metadata_line(filename, source_digest)}\n\n{contract.rstrip()}\n" + + +def _metadata_values(marker: str) -> dict[str, str]: + try: + fields = shlex.split(marker.removeprefix(_EDITABLE_CONTRACT_PREFIX).strip()) + except ValueError as exc: + raise UsageError(f"Invalid editable .pyi metadata: {exc}") from exc + values: dict[str, str] = {} + for field in fields: + key, separator, value = field.partition("=") + if not separator or key in values or key not in {"file", "source-sha256"}: + raise UsageError(f"Invalid editable .pyi metadata field {field!r}") + values[key] = value + return values + + +def _source_digest(values: Mapping[str, str]) -> str | None: + digest = values.get("source-sha256") + if digest is None: + return None + if len(digest) != 64 or any(char not in "0123456789abcdef" for char in digest): + raise UsageError("Editable .pyi metadata requires a full lowercase source-sha256 digest") + return digest + + +def parse_editable_contract(cell: str) -> EditableContract | None: + """Recognize an editable contract carrying reserved notebook metadata.""" + lines = cell.splitlines(keepends=True) + marker_indices = [index for index, line in enumerate(lines) if line.strip().startswith(_EDITABLE_CONTRACT_PREFIX)] + if not marker_indices: + return None + if len(marker_indices) > 1: + raise UsageError("An editable .pyi cell must contain exactly one PRIK metadata line") + marker_index = marker_indices[0] + marker = lines[marker_index].strip() + values = _metadata_values(marker) + filename = values.get("file") + if filename is not None: + filename = _validated_contract_filename(filename) + del lines[marker_index] + return EditableContract( + source_digest=_source_digest(values), + filename=filename, + text="".join(lines), + ) + + +def _generated_contract_mapping(value: object, *, field_name: str) -> dict[str, str]: + """Convert semantic-pipeline module names into safe relative `.pyi` paths.""" + if not isinstance(value, Mapping): + raise ValueError(f"Generated semantic payload field {field_name!r} is invalid") + contracts: dict[str, str] = {} + for module_name, text in value.items(): + if not isinstance(module_name, str) or not isinstance(text, str): + raise ValueError(f"Generated semantic payload field {field_name!r} is invalid") + contracts[_contract_filename(module_name)] = text + return contracts + + +def _source_semantic_report(source: Path, options) -> Mapping[str, object]: + from prik.cli import _semantic_report + + preprocessing = PreprocessingConfig( + mode="compiler", + compiler=options.compiler, + compiler_args=[*options.compiler_args, *options.native_compile_flags], + ) + reports = _semantic_report([str(source)], preprocessing, language=options.language) + report = reports.get(str(source)) + if not isinstance(report, Mapping): + raise ValueError("PRIK did not generate a semantic contract for this source cell") + return report + + +def generate_contracts_from_source( + source: Path, + *, + source_digest: str, + options, +) -> GeneratedContracts: + """Reuse the CLI source-semantic route to extract editable contracts.""" + report = _source_semantic_report(source, options) + dependencies = _generated_contract_mapping( + report.get("pyi_dependencies", {}), + field_name="pyi_dependencies", + ) + if options.language == "c": + contract = report.get("pyi") + if not isinstance(contract, str) or not contract.strip(): + raise ValueError("PRIK did not generate any editable C declarations from this source cell") + return GeneratedContracts( + language=options.language, + source_digest=source_digest, + module_contracts={}, + direct_contract=contract, + dependency_contracts=dependencies, + ) + return _generated_fortran_contracts( + report, + language=options.language, + source_digest=source_digest, + dependencies=dependencies, + ) + + +def _generated_fortran_contracts( + report: Mapping[str, object], + *, + language: str, + source_digest: str, + dependencies: dict[str, str], +) -> GeneratedContracts: + from prik.cli import _source_root_stub + + modules = _generated_contract_mapping(report.get("pyi_modules", {}), field_name="pyi_modules") + external_sections = report.get("pyi_root_externals", ()) + if not isinstance(external_sections, list) or any(not isinstance(text, str) for text in external_sections): + raise ValueError("Generated semantic payload field 'pyi_root_externals' is invalid") + standalone = _source_root_stub([], external_sections) if external_sections else None + if not modules and not standalone: + raise ValueError("PRIK did not generate any editable Fortran declarations from this source cell") + return GeneratedContracts( + language=language, + source_digest=source_digest, + module_contracts=modules, + direct_contract=standalone, + dependency_contracts=dependencies, + ) + + +def generated_editable_cells( + contracts: GeneratedContracts, + *, + magic_command: str, +) -> list[str]: + """Render one editable notebook cell per visible generated contract.""" + cells = [ + editable_cell_text( + text, + filename=filename, + source_digest=contracts.source_digest, + magic_command=magic_command, + ) + for filename, text in contracts.module_contracts.items() + ] + if contracts.direct_contract is not None: + cells.append( + editable_cell_text( + contracts.direct_contract, + filename=None, + source_digest=contracts.source_digest, + magic_command=magic_command, + ) + ) + return cells + + +def insert_editable_cells(shell, cells: list[str]) -> tuple[str, ...]: + """Present editable cells and return any awaiting a terminal prompt.""" + if not cells: + raise UsageError("PRIK did not generate an editable .pyi cell") + set_next_input = getattr(shell, "set_next_input", None) + if not callable(set_next_input): + raise UsageError("This IPython frontend cannot insert an editable .pyi cell") + + first, *remaining = cells + set_next_input(first, replace=False) + if getattr(shell, "rl_next_input", None) == first: + return tuple(remaining) + + payload_manager = getattr(shell, "payload_manager", None) + write_payload = getattr(payload_manager, "write_payload", None) + if callable(write_payload): + for text in remaining: + write_payload( + {"source": "set_next_input", "text": text, "replace": False}, + single=False, + ) + return () + for text in remaining: + set_next_input(text, replace=False) + return () + + +def _write_contract_file(root: Path, relative_name: str, text: str) -> Path: + """Materialize one validated generated contract below a build-local package.""" + validated = _validated_contract_filename(relative_name) + target = root.joinpath(*PurePosixPath(validated).parts) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(f"{text.rstrip()}\n", encoding="utf-8") + return target + + +def _merge_contract_files(destination: dict[str, str], additions: Mapping[str, str]) -> None: + """Merge generated support contracts without silently changing a path.""" + for filename, text in additions.items(): + validated = _validated_contract_filename(filename) + previous = destination.get(validated) + if previous is not None and previous != text: + raise UsageError(f"Generated editable contract dependencies conflict at {validated}") + destination[validated] = text + + +def materialize_editable_contract( + entry_dir: Path, + editable: EditableContract, + generated: GeneratedContracts, +) -> Path: + """Create the hidden contract package consumed by `build_pyi_extension()`.""" + package = entry_dir / "contract" + package.mkdir(parents=True, exist_ok=True) + support_contracts: dict[str, str] = {} + _merge_contract_files(support_contracts, generated.dependency_contracts) + _merge_contract_files(support_contracts, generated.module_contracts) + + if editable.filename is None: + entry = _materialize_direct_entry(package, editable, generated) + else: + entry = _materialize_module_entry(package, editable, generated, support_contracts) + for filename, text in support_contracts.items(): + _write_contract_file(package, filename, text) + return entry + + +def materialize_handwritten_contract( + entry_dir: Path, + editable: EditableContract, + *, + native_language: str, +) -> Path: + """Create one handwritten cell contract for the ordinary `.pyi` build path.""" + if editable.source_digest is not None: + raise UsageError("A handwritten .pyi contract cannot carry generated source digest metadata") + package = entry_dir / "contract" + package.mkdir(parents=True, exist_ok=True) + if editable.filename is not None: + _write_contract_file(package, editable.filename, editable.text) + module_name = editable.filename.removesuffix(".pyi").replace("/", ".") + entry = package / "__init__.pyi" + entry.write_text(f"from . import {module_name}\n", encoding="utf-8") + return entry + entry = package / ("contract.pyi" if native_language == "c" else "__init__.pyi") + entry.write_text(f"{editable.text.rstrip()}\n", encoding="utf-8") + return entry + + +def _materialize_direct_entry( + package: Path, + editable: EditableContract, + generated: GeneratedContracts, +) -> Path: + if generated.direct_contract is None: + raise UsageError("This source cell did not generate an editable direct-declaration .pyi contract") + entry = package / ("contract.pyi" if generated.language == "c" else "__init__.pyi") + entry.write_text(f"{editable.text.rstrip()}\n", encoding="utf-8") + return entry + + +def _materialize_module_entry( + package: Path, + editable: EditableContract, + generated: GeneratedContracts, + support_contracts: dict[str, str], +) -> Path: + assert editable.filename is not None + if editable.filename not in generated.module_contracts: + raise UsageError(f"This source cell did not generate the editable module contract {editable.filename!r}") + support_contracts[editable.filename] = editable.text + module_name = editable.filename.removesuffix(".pyi").replace("/", ".") + entry = package / "__init__.pyi" + entry.write_text(f"from . import {module_name}\n", encoding="utf-8") + return entry diff --git a/prik/jupyter/magic.py b/prik/jupyter/magic.py new file mode 100644 index 000000000..169590971 --- /dev/null +++ b/prik/jupyter/magic.py @@ -0,0 +1,878 @@ +"""Compile Fortran and C notebook cells through PRIK's public build API.""" + +from __future__ import annotations + +import argparse +from collections.abc import Callable +from contextlib import suppress +from dataclasses import dataclass +import hashlib +import json +import os +from pathlib import Path +import shlex +import sys +import sysconfig +from types import ModuleType + +from filelock import FileLock +from IPython.core.error import UsageError +from IPython.core.magic import Magics, cell_magic, magics_class + +from prik import __version__ +from prik.jupyter import contracts as contract_cells +from prik.pipeline.build import ( + WrapperBuildResult, + build_c_extension, + build_fortran_extension, + build_pyi_extension, +) +from prik.preprocessing import PreprocessingConfig + + +_BUILD_CONFIGURATION_SCHEMA_VERSION = 1 +_BUILD_CACHE_RECORD_SCHEMA_VERSION = 2 +_SOURCE_RECORD_SCHEMA_VERSION = 1 +_CACHE_RECORD_NAME = "cell-build.json" +_SOURCE_RECORD_NAME = "source.json" + + +# Options whose value normally starts with a dash, which argparse would +# otherwise read as the next option rather than as this option's value. Only +# the flag groups are split, so only they may carry several flags at once. +_DASH_VALUE_OPTIONS = ("--compiler-arg",) +_DASH_VALUE_FLAG_GROUPS = ( + "--native-compile-flags", + "--wrapper-fortran-flags", + "--wrapper-c-flags", +) + + +class _MagicArgumentParser(argparse.ArgumentParser): + """Raise an IPython usage error instead of terminating the kernel.""" + + def error(self, message: str) -> None: + raise UsageError(self._with_dash_value_hint(message)) + + @staticmethod + def _with_dash_value_hint(message: str) -> str: + """Name the equals form when argparse read a flag value as an option.""" + if "expected one argument" not in message: + return message + group = next((name for name in _DASH_VALUE_FLAG_GROUPS if name in message), None) + if group is not None: + return ( + f"{message}; use the equals form for a dash-prefixed value ({group}=-O3) " + f'and one quoted group for several flags ({group}="-O3 -march=native")' + ) + option = next((name for name in _DASH_VALUE_OPTIONS if name in message), None) + if option is None: + return message + return f"{message}; use the equals form for a dash-prefixed value ({option}=-fopenmp)" + + +@dataclass(frozen=True) +class _CellOptions: + """Normalized build-affecting and execution-only magic options.""" + + language: str + compiler: str + compiler_explicit: bool + compiler_args: tuple[str, ...] + native_compile_flags: tuple[str, ...] + wrapper_fortran_flags: tuple[str, ...] + wrapper_c_flags: tuple[str, ...] + generate_pyi: bool + force: bool + verbose: bool + + +@dataclass(frozen=True) +class _PendingEditableCells: + """Generated contracts awaiting terminal IPython's next-input prompt.""" + + source_digest: str + cells: tuple[str, ...] + + +@dataclass(frozen=True) +class _ManualNativeSources: + """Existing native source paths supplied by one handwritten contract cell.""" + + language: str + paths: tuple[Path, ...] + + +def _argument_parser( + magic_name: str, + *, + supports_pyi_generation: bool, + supports_native_sources: bool = False, +) -> _MagicArgumentParser: + parser = _MagicArgumentParser( + prog=f"%%{magic_name}", + add_help=False, + description="Compile and publish one notebook cell through PRIK.", + ) + parser.add_argument("-h", "--help", action="store_true", help="Show this help and do not compile the cell") + if supports_pyi_generation: + parser.add_argument( + "--pyi", + action="store_true", + help="Persist this source and insert editable semantic .pyi cells", + ) + if supports_native_sources: + parser.add_argument( + "--native-fortran-sources", + action="extend", + nargs="+", + default=[], + metavar="PATH", + help="Existing Fortran implementation sources for a handwritten contract", + ) + parser.add_argument( + "--native-c-sources", + action="extend", + nargs="+", + default=[], + metavar="PATH", + help="Existing C implementation sources for a handwritten contract", + ) + parser.add_argument("--compiler", help="Exact Fortran or C compiler executable") + parser.add_argument( + "--compiler-arg", + action="append", + default=[], + metavar="ARG", + help="Additional preprocessing argument; repeat as needed", + ) + parser.add_argument( + "--native-compile-flags", + action="append", + default=[], + metavar="FLAGS", + help="Quoted compiler flags for the selected native sources; repeat as needed", + ) + parser.add_argument( + "--wrapper-fortran-flags", + action="append", + default=[], + metavar="FLAGS", + help="Quoted compiler flags for generated Fortran bridge source", + ) + parser.add_argument( + "--wrapper-c-flags", + action="append", + default=[], + metavar="FLAGS", + help="Quoted compiler and link flags for generated C binding source", + ) + parser.add_argument("--force", action="store_true", help="Recompile even when this exact cell is cached") + parser.add_argument("--verbose", action="store_true", help="Print PRIK build commands or cache reuse") + return parser + + +def _split_flag_groups(groups: list[str], *, option_name: str) -> tuple[str, ...]: + flags: list[str] = [] + for group in groups: + try: + flags.extend(shlex.split(group)) + except ValueError as exc: + raise UsageError(f"Invalid {option_name} value {group!r}: {exc}") from exc + return tuple(flags) + + +def _options_from_namespace( + parsed: argparse.Namespace, + *, + language: str, + generate_pyi: bool, +) -> _CellOptions: + """Normalize one successfully parsed magic invocation.""" + compiler = parsed.compiler or ("gfortran" if language == "fortran" else "cc") + return _CellOptions( + language=language, + compiler=compiler, + compiler_explicit=parsed.compiler is not None, + compiler_args=tuple(parsed.compiler_arg), + native_compile_flags=_split_flag_groups( + parsed.native_compile_flags, + option_name="--native-compile-flags", + ), + wrapper_fortran_flags=_split_flag_groups( + parsed.wrapper_fortran_flags, + option_name="--wrapper-fortran-flags", + ), + wrapper_c_flags=_split_flag_groups( + parsed.wrapper_c_flags, + option_name="--wrapper-c-flags", + ), + generate_pyi=generate_pyi, + force=parsed.force, + verbose=parsed.verbose, + ) + + +def _parse_arguments(line: str, parser: _MagicArgumentParser) -> argparse.Namespace | None: + try: + arguments = shlex.split(line) + except ValueError as exc: + raise UsageError(f"Invalid {parser.prog} arguments: {exc}") from exc + if "-h" in arguments or "--help" in arguments: + print(parser.format_help()) + return None + return parser.parse_args(arguments) + + +def _parse_source_options(line: str, *, language: str) -> _CellOptions | None: + parser = _argument_parser(language, supports_pyi_generation=True) + parsed = _parse_arguments(line, parser) + if parsed is None: + return None + if parsed.pyi and parsed.force: + parser.error("--pyi only generates editable cells; do not pass --force") + return _options_from_namespace(parsed, language=language, generate_pyi=parsed.pyi) + + +def _manual_native_sources( + parsed: argparse.Namespace, + parser: _MagicArgumentParser, +) -> _ManualNativeSources | None: + """Resolve the one native language selected by handwritten-contract options.""" + fortran_values = tuple(parsed.native_fortran_sources) + c_values = tuple(parsed.native_c_sources) + if fortran_values and c_values: + parser.error("%%pyi cannot mix --native-fortran-sources and --native-c-sources") + values = fortran_values or c_values + if not values: + return None + language = "fortran" if fortran_values else "c" + paths: list[Path] = [] + for value in values: + try: + path = Path(value).expanduser().resolve(strict=True) + except (OSError, RuntimeError) as exc: + raise UsageError(f"Native source path {value!r} is unavailable: {exc}") from exc + if not path.is_file(): + raise UsageError(f"Native source path {value!r} is not a file") + paths.append(path) + return _ManualNativeSources(language=language, paths=tuple(paths)) + + +def _cell_digest(language: str, cell: str) -> str: + """Return the documented digest of the language followed by exact cell text.""" + return hashlib.sha256(f"{language}{cell}".encode()).hexdigest() + + +def _default_cache_root() -> Path: + """Return the persistent notebook cache root without creating it.""" + if root := os.getenv("PRIK_CACHE_DIR"): + return Path(root) / "jupyter" + if root := os.getenv("XDG_CACHE_HOME"): + return Path(root) / "prik" / "jupyter" + return Path.home() / ".cache" / "prik" / "jupyter" + + +def _source_path(entry_dir: Path, language: str) -> Path: + """Return the native cell path owned by one source-digest entry.""" + return entry_dir / ("cell.f90" if language == "fortran" else "cell.c") + + +def _editable_magic_command(options: _CellOptions) -> str: + """Render the readable build options copied into an inserted contract cell.""" + arguments = ["%%pyi"] + if options.compiler_explicit: + arguments.extend(("--compiler", options.compiler)) + arguments.extend(f"--compiler-arg={value}" for value in options.compiler_args) + flag_groups = ( + ("--native-compile-flags", options.native_compile_flags), + ("--wrapper-fortran-flags", options.wrapper_fortran_flags), + ("--wrapper-c-flags", options.wrapper_c_flags), + ) + for option_name, flags in flag_groups: + if flags: + arguments.append(f"{option_name}={shlex.join(flags)}") + if options.verbose: + arguments.append("--verbose") + return shlex.join(arguments) + + +def _build_configuration(options: _CellOptions) -> dict[str, object]: + """Return the compatibility facts validated inside one source digest entry.""" + return { + "schema_version": _BUILD_CONFIGURATION_SCHEMA_VERSION, + "prik_version": __version__, + "python_cache_tag": sys.implementation.cache_tag, + "python_soabi": sysconfig.get_config_var("SOABI"), + "language": options.language, + "compiler": options.compiler, + "compiler_args": list(options.compiler_args), + "native_compile_flags": list(options.native_compile_flags), + "wrapper_fortran_flags": list(options.wrapper_fortran_flags), + "wrapper_c_flags": list(options.wrapper_c_flags), + } + + +def _file_sha256(path: Path) -> str: + """Hash one explicit native source without retaining its contents in memory.""" + digest = hashlib.sha256() + try: + with path.open("rb") as source: + while chunk := source.read(1024 * 1024): + digest.update(chunk) + except OSError as exc: + raise UsageError(f"Cannot read native source {path}: {exc}") from exc + return digest.hexdigest() + + +def _build_fingerprint( + options: _CellOptions, + *, + native_sources: tuple[Path, ...] = (), +) -> str: + configuration = _build_configuration(options) + if native_sources: + configuration["native_sources"] = [{"path": str(path), "sha256": _file_sha256(path)} for path in native_sources] + payload = json.dumps(configuration, sort_keys=True, separators=(",", ":")) + return hashlib.sha256(payload.encode()).hexdigest() + + +def _extension_module_name(language: str, digest: str, fingerprint: str, generation: int) -> str: + language_marker = "f" if language == "fortran" else "c" + return f"_prik_{language_marker}_{digest[:12]}_{fingerprint[:8]}_{generation}" + + +def _read_cache_record(path: Path) -> dict[str, object] | None: + try: + value = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + return value if isinstance(value, dict) else None + + +def _write_source_record(path: Path, *, digest: str, language: str) -> None: + """Persist the language identity needed by a later `%%pyi` cell.""" + record = { + "schema_version": _SOURCE_RECORD_SCHEMA_VERSION, + "source_digest": digest, + "language": language, + } + temporary = path.with_name(f".{path.name}.{os.getpid()}.tmp") + temporary.write_text(f"{json.dumps(record, sort_keys=True, indent=2)}\n", encoding="utf-8") + temporary.replace(path) + + +def _source_language(path: Path, *, digest: str) -> str: + """Load the native language owned by one generated source-cache entry.""" + record = _read_cache_record(path) + if record is None or record.get("schema_version") != _SOURCE_RECORD_SCHEMA_VERSION: + raise UsageError( + "The native source for this editable .pyi is unavailable; " + "execute its %%fortran --pyi or %%c --pyi source cell again" + ) + language = record.get("language") + if record.get("source_digest") != digest or language not in {"fortran", "c"}: + raise UsageError( + "The cached native source identity does not match this editable .pyi; " + "execute its %%fortran --pyi or %%c --pyi source cell again" + ) + assert isinstance(language, str) + return language + + +def _generation(record: dict[str, object] | None) -> int: + value = None if record is None else record.get("generation") + return value if isinstance(value, int) and not isinstance(value, bool) and 0 <= value < 1_000_000 else -1 + + +def _compatible_generation( + record: dict[str, object] | None, + *, + digest: str, + fingerprint: str, +) -> int | None: + if record is None or record.get("schema_version") != _BUILD_CACHE_RECORD_SCHEMA_VERSION: + return None + if record.get("digest") != digest or record.get("build_fingerprint") != fingerprint: + return None + generation = _generation(record) + return generation if generation >= 0 else None + + +def _recorded_shared_library(record: dict[str, object], *, entry_dir: Path, module_name: str) -> Path | None: + name = record.get("shared_library") + if not isinstance(name, str) or Path(name).name != name or not name.startswith(f"{module_name}."): + return None + shared_library = entry_dir / "build" / name + return shared_library if shared_library.is_file() else None + + +def _cached_result( + record: dict[str, object] | None, + *, + entry_dir: Path, + sources: tuple[Path, ...], + digest: str, + fingerprint: str, + language: str, +) -> WrapperBuildResult | None: + generation = _compatible_generation(record, digest=digest, fingerprint=fingerprint) + if record is None or generation is None: + return None + module_name = _extension_module_name(language, digest, fingerprint, generation) + if record.get("module_name") != module_name: + return None + shared_library = _recorded_shared_library(record, entry_dir=entry_dir, module_name=module_name) + if shared_library is None: + return None + return WrapperBuildResult( + sources=sources, + module_name=module_name, + output_dir=entry_dir / "build", + shared_library=shared_library, + build_makefile=None, + compiled=True, + generated_sources=(), + generated_files=(), + ) + + +def _write_cache_record( + path: Path, + *, + digest: str, + fingerprint: str, + generation: int, + result: WrapperBuildResult, +) -> None: + record = { + "schema_version": _BUILD_CACHE_RECORD_SCHEMA_VERSION, + "digest": digest, + "build_fingerprint": fingerprint, + "generation": generation, + "module_name": result.module_name, + "shared_library": result.shared_library.name, + } + temporary = path.with_name(f".{path.name}.{os.getpid()}.tmp") + temporary.write_text(f"{json.dumps(record, sort_keys=True, indent=2)}\n", encoding="utf-8") + temporary.replace(path) + + +def _build_cell( + source: Path, + *, + output_dir: Path, + output_name: str, + options: _CellOptions, +) -> WrapperBuildResult: + preprocessing = PreprocessingConfig( + mode="compiler", + compiler=options.compiler, + compiler_args=list(options.compiler_args), + ) + common = { + "output_dir": output_dir, + "output_name": output_name, + "preprocessing": preprocessing, + "verbose": options.verbose, + "wrapper_fortran_flags": options.wrapper_fortran_flags, + "wrapper_c_flags": options.wrapper_c_flags, + } + if options.language == "fortran": + return build_fortran_extension( + source, + native_fortran_flags=options.native_compile_flags, + **common, + ) + return build_c_extension( + source, + input_c_compiler=options.compiler, + native_c_flags=options.native_compile_flags, + **common, + ) + + +def _build_editable_contract( + contract: Path, + native_sources: tuple[Path, ...], + *, + output_dir: Path, + output_name: str, + options: _CellOptions, +) -> WrapperBuildResult: + """Build one semantic contract against its selected native sources.""" + common = { + "output_dir": output_dir, + "output_name": output_name, + "native_language": options.language, + "verbose": options.verbose, + "wrapper_fortran_flags": options.wrapper_fortran_flags, + "wrapper_c_flags": options.wrapper_c_flags, + } + if options.language == "fortran": + return build_pyi_extension( + contract, + input_compiler=options.compiler, + native_fortran_sources=native_sources, + native_fortran_flags=options.native_compile_flags, + **common, + ) + return build_pyi_extension( + contract, + input_c_compiler=options.compiler, + native_c_sources=native_sources, + native_c_flags=options.native_compile_flags, + **common, + ) + + +def _editable_pyi_contract( + cell: str, + manual_sources: _ManualNativeSources | None, +) -> contract_cells.EditableContract: + """Resolve generated metadata or construct one handwritten cell contract.""" + if not cell.strip(): + raise UsageError("%%pyi requires a non-empty semantic .pyi cell") + editable = contract_cells.parse_editable_contract(cell) + if editable is None: + if manual_sources is None: + raise UsageError( + "%%pyi requires generated source metadata or explicit --native-fortran-sources/--native-c-sources" + ) + return contract_cells.EditableContract( + source_digest=None, + filename=None, + text=cell, + ) + if editable.source_digest is not None and manual_sources is not None: + raise UsageError("%%pyi cannot combine generated source-sha256 metadata with explicit native sources") + if editable.source_digest is None and manual_sources is None: + raise UsageError("A handwritten %%pyi cell requires --native-fortran-sources or --native-c-sources") + return editable + + +def _pyi_native_language( + editable: contract_cells.EditableContract, + manual_sources: _ManualNativeSources | None, + *, + cache_root: Path, +) -> str: + """Return the explicit manual language or recover one generated language.""" + if manual_sources is not None: + return manual_sources.language + source_digest = editable.source_digest + assert source_digest is not None + source_entry_dir = cache_root / source_digest + return _source_language( + source_entry_dir / _SOURCE_RECORD_NAME, + digest=source_digest, + ) + + +def _public_bindings(module: ModuleType) -> dict[str, object]: + """Return public root declarations and namespaces named as the notebook shows them. + + A cell's extension is imported under a private cache module name, which + would otherwise surface in ``repr()``, ``help()``, and ``__module__``. The + published objects are restated under the names the user actually binds, + leaving the private name to ``sys.modules`` alone. + """ + bindings = {name: value for name, value in vars(module).items() if not name.startswith("_")} + for name, value in bindings.items(): + if isinstance(value, ModuleType): + _restate_namespace(value, name) + else: + # Published directly into the session, so it has no public owner. + _set_owner_module(value, None) + return bindings + + +def _restate_namespace(namespace: ModuleType, public_name: str) -> None: + """Rename one published namespace and every member it owns.""" + namespace.__name__ = public_name + # A namespace carrying module variables is an instance of a generated heap + # type whose name also embeds the private root. + _set_owner_module(type(namespace), public_name) + for member_name, member in vars(namespace).items(): + if member_name.startswith("_"): + continue + if isinstance(member, ModuleType): + _restate_namespace(member, f"{public_name}.{member_name}") + else: + _set_owner_module(member, public_name) + + +def _set_owner_module(value: object, owner: str | None) -> None: + """Set one object's owning-module name, ignoring an immutable target.""" + with suppress(AttributeError, TypeError): + value.__module__ = owner + + +@magics_class +class PrikMagics(Magics): + """Own PRIK's source, contract, cache, import, and publication magics.""" + + magic_names = ("fortran", "c", "pyi") + owns_prik_cell_magics = True + + def __init__(self, shell=None, *, cache_dir: str | Path | None = None) -> None: + super().__init__(shell=shell) + self.cache_root = Path(cache_dir) if cache_dir is not None else _default_cache_root() + self._pending_editable_cells: _PendingEditableCells | None = None + + @cell_magic + def fortran(self, line: str, cell: str) -> None: + """Compile Fortran source or generate its editable contract cells.""" + self._run_source_magic(line, cell, language="fortran") + + @cell_magic + def c(self, line: str, cell: str) -> None: + """Compile C source or generate its editable contract cell.""" + self._run_source_magic(line, cell, language="c") + + @cell_magic + def pyi(self, line: str, cell: str) -> None: + """Compile one generated or handwritten semantic contract.""" + parser = _argument_parser( + "pyi", + supports_pyi_generation=False, + supports_native_sources=True, + ) + parsed = _parse_arguments(line, parser) + if parsed is None: + return + manual_sources = _manual_native_sources(parsed, parser) + editable = _editable_pyi_contract(cell, manual_sources) + language = _pyi_native_language(editable, manual_sources, cache_root=self.cache_root) + options = _options_from_namespace(parsed, language=language, generate_pyi=False) + if manual_sources is None: + module, reused = self._load_or_build_editable(cell, editable, options) + else: + module, reused = self._load_or_build_handwritten( + cell, + editable, + manual_sources, + options, + ) + self._publish(module, reused=reused, cell=cell, options=options) + if editable.source_digest is not None: + self._present_next_editable_cell(editable.source_digest) + + def _run_source_magic(self, line: str, cell: str, *, language: str) -> None: + """Execute the shared native-source workflow selected by its magic name.""" + options = _parse_source_options(line, language=language) + if options is None: + return + if not cell.strip(): + raise UsageError(f"%%{language} requires a non-empty {language.capitalize()} source cell") + + if options.generate_pyi: + self._generate_pyi_cells(cell, options) + return + module, reused = self._load_or_build_source(cell, options) + self._publish(module, reused=reused, cell=cell, options=options) + + def _publish(self, module: ModuleType, *, reused: bool, cell: str, options: _CellOptions) -> None: + """Publish one built extension's public API into the notebook namespace.""" + if reused and options.verbose: + print(f">> Reuse cached PRIK cell: {_cell_digest(options.language, cell)}") + self.shell.push(_public_bindings(module)) + + def _generate_pyi_cells(self, cell: str, options: _CellOptions) -> None: + source_digest = _cell_digest(options.language, cell) + fingerprint = _build_fingerprint(options) + entry_dir = self.cache_root / source_digest + source = _source_path(entry_dir, options.language) + contracts_path = contract_cells.generated_contract_record_path(entry_dir, fingerprint) + + self.cache_root.mkdir(parents=True, exist_ok=True) + with FileLock(str(self.cache_root / f"{source_digest}.lock")): + entry_dir.mkdir(parents=True, exist_ok=True) + source.write_text(cell, encoding="utf-8") + contracts = contract_cells.generate_contracts_from_source( + source, + source_digest=source_digest, + options=options, + ) + contract_cells.write_generated_contracts(contracts_path, contracts) + _write_source_record( + entry_dir / _SOURCE_RECORD_NAME, + digest=source_digest, + language=options.language, + ) + + cells = contract_cells.generated_editable_cells( + contracts, + magic_command=_editable_magic_command(options), + ) + remaining = contract_cells.insert_editable_cells(self.shell, cells) + self._pending_editable_cells = ( + _PendingEditableCells(source_digest=source_digest, cells=remaining) if remaining else None + ) + + def _present_next_editable_cell(self, source_digest: str) -> None: + """Advance one matching terminal-IPython editable-contract queue.""" + pending = self._pending_editable_cells + if pending is None or pending.source_digest != source_digest: + return + next_cell, *remaining = pending.cells + contract_cells.insert_editable_cells(self.shell, [next_cell]) + self._pending_editable_cells = ( + _PendingEditableCells(source_digest=source_digest, cells=tuple(remaining)) if remaining else None + ) + + def _load_or_build_source(self, cell: str, options: _CellOptions) -> tuple[ModuleType, bool]: + digest = _cell_digest(options.language, cell) + fingerprint = _build_fingerprint(options) + entry_dir = self.cache_root / digest + source = _source_path(entry_dir, options.language) + + def build(build_dir: Path, module_name: str) -> WrapperBuildResult: + source.write_text(cell, encoding="utf-8") + return _build_cell( + source, + output_dir=build_dir, + output_name=module_name, + options=options, + ) + + return self._load_or_build_cached( + digest=digest, + fingerprint=fingerprint, + sources=(source,), + options=options, + build=build, + ) + + def _load_or_build_editable( + self, + cell: str, + editable: contract_cells.EditableContract, + options: _CellOptions, + ) -> tuple[ModuleType, bool]: + digest = _cell_digest(options.language, cell) + fingerprint = _build_fingerprint(options) + source_digest = editable.source_digest + assert source_digest is not None + source_entry_dir = self.cache_root / source_digest + source = _source_path(source_entry_dir, options.language) + contracts_path = contract_cells.generated_contract_record_path(source_entry_dir, fingerprint) + + self.cache_root.mkdir(parents=True, exist_ok=True) + with FileLock(str(self.cache_root / f"{source_digest}.lock")): + try: + source_text = source.read_text(encoding="utf-8") + except OSError as exc: + raise UsageError( + "The native source for this editable .pyi is unavailable; " + "execute its %%fortran --pyi or %%c --pyi source cell again" + ) from exc + if _cell_digest(options.language, source_text) != source_digest: + raise UsageError( + "The cached native source does not match this editable .pyi; " + "execute its %%fortran --pyi or %%c --pyi source cell again" + ) + generated = contract_cells.read_generated_contracts( + contracts_path, + language=options.language, + source_digest=source_digest, + ) + + def build(build_dir: Path, module_name: str) -> WrapperBuildResult: + entry_dir = build_dir.parent + contract = contract_cells.materialize_editable_contract(entry_dir, editable, generated) + return _build_editable_contract( + contract, + (source,), + output_dir=build_dir, + output_name=module_name, + options=options, + ) + + return self._load_or_build_cached( + digest=digest, + fingerprint=fingerprint, + sources=(source,), + options=options, + build=build, + ) + + def _load_or_build_handwritten( + self, + cell: str, + editable: contract_cells.EditableContract, + native_sources: _ManualNativeSources, + options: _CellOptions, + ) -> tuple[ModuleType, bool]: + """Build one independent handwritten contract against existing files.""" + digest = _cell_digest(options.language, cell) + fingerprint = _build_fingerprint(options, native_sources=native_sources.paths) + + def build(build_dir: Path, module_name: str) -> WrapperBuildResult: + contract = contract_cells.materialize_handwritten_contract( + build_dir.parent, + editable, + native_language=options.language, + ) + return _build_editable_contract( + contract, + native_sources.paths, + output_dir=build_dir, + output_name=module_name, + options=options, + ) + + return self._load_or_build_cached( + digest=digest, + fingerprint=fingerprint, + sources=native_sources.paths, + options=options, + build=build, + ) + + def _load_or_build_cached( + self, + *, + digest: str, + fingerprint: str, + sources: tuple[Path, ...], + options: _CellOptions, + build: Callable[[Path, str], WrapperBuildResult], + ) -> tuple[ModuleType, bool]: + """Reuse one validated notebook artifact or execute its selected build path.""" + entry_dir = self.cache_root / digest + record_path = entry_dir / _CACHE_RECORD_NAME + self.cache_root.mkdir(parents=True, exist_ok=True) + with FileLock(str(self.cache_root / f"{digest}.lock")): + entry_dir.mkdir(parents=True, exist_ok=True) + record = _read_cache_record(record_path) + if not options.force: + cached = _cached_result( + record, + entry_dir=entry_dir, + sources=sources, + digest=digest, + fingerprint=fingerprint, + language=options.language, + ) + if cached is not None: + return cached.import_module(), True + + generation = _generation(record) + 1 + module_name = _extension_module_name(options.language, digest, fingerprint, generation) + while module_name in sys.modules: + generation += 1 + module_name = _extension_module_name(options.language, digest, fingerprint, generation) + result = build(entry_dir / "build", module_name) + module = result.import_module() + _write_cache_record( + record_path, + digest=digest, + fingerprint=fingerprint, + generation=generation, + result=result, + ) + return module, False + + +__all__ = ("PrikMagics",) diff --git a/prik/pipeline/wrapper.py b/prik/pipeline/wrapper.py index 0699dcd27..6a5f29611 100644 --- a/prik/pipeline/wrapper.py +++ b/prik/pipeline/wrapper.py @@ -37,6 +37,7 @@ from prik.policy.models import ( ArgumentHandoffMode, ArrayLogicalABI, + ArrayPythonLayout, ArrayWritebackABI, BridgeDataAction, CallbackABIKind, @@ -3597,7 +3598,7 @@ def _array_shape_diagnostics( return self._raw_array_shape_diagnostics(plan) diagnostics = [] if array.rank is None: - diagnostics.extend(self._assumed_rank_array_diagnostics(plan)) + diagnostics.extend(self._runtime_rank_array_diagnostics(plan)) else: diagnostics.extend(self._concrete_rank_array_diagnostics(plan)) diagnostics.extend(self._array_layout_role_diagnostics(plan)) @@ -3811,21 +3812,42 @@ def _concrete_rank_array_diagnostics( diagnostics.append(self._diagnostic(plan.owner_path, "inconsistent-array-rank", array.rank)) if len(array.extent_roles) != array.rank or array.runtime_rank_role is not None: diagnostics.append(self._diagnostic(plan.owner_path, "invalid-array-rank-roles", array.extent_roles)) + expected_maximum = 15 if array.flatten_python_storage else array.rank + if array.minimum_rank != array.rank or array.maximum_rank != expected_maximum: + diagnostics.append( + self._diagnostic( + plan.owner_path, + "invalid-array-rank-bounds", + (array.minimum_rank, array.maximum_rank), + ) + ) return tuple(diagnostics) - def _assumed_rank_array_diagnostics( + def _runtime_rank_array_diagnostics( self, plan: ArgumentTransferPlan, ) -> tuple[WrapperPlanDiagnostic, ...]: - """Validate the one-through-fifteen runtime-rank ABI.""" + """Validate a completed language-specific runtime-rank ABI.""" array = plan.array if array is None or array.rank is not None: return () diagnostics = [] - if array.category != "assumed_rank" or array.shape != ("...",): - diagnostics.append(self._diagnostic(plan.owner_path, "invalid-assumed-rank-array", array.shape)) + bounds = { + "assumed_rank": (1, 15), + "runtime_rank": (0, 15), + } + if array.category not in bounds or array.shape != ("...",): + diagnostics.append(self._diagnostic(plan.owner_path, "invalid-runtime-rank-array", array.shape)) + elif (array.minimum_rank, array.maximum_rank) != bounds[array.category]: + diagnostics.append( + self._diagnostic( + plan.owner_path, + "invalid-runtime-rank-bounds", + (array.minimum_rank, array.maximum_rank), + ) + ) if len(array.extent_roles) != 15 or array.runtime_rank_role is None: - diagnostics.append(self._diagnostic(plan.owner_path, "invalid-assumed-rank-roles", array.extent_roles)) + diagnostics.append(self._diagnostic(plan.owner_path, "invalid-runtime-rank-roles", array.extent_roles)) return tuple(diagnostics) def _array_layout_role_diagnostics( @@ -3869,14 +3891,17 @@ def _array_order_diagnostics(self, plan: ArgumentTransferPlan) -> tuple[WrapperP return tuple(diagnostics) def _array_axis_mode_diagnostics(self, plan: ArgumentTransferPlan) -> tuple[WrapperPlanDiagnostic, ...]: - """Validate dense versus stride-aware axis markers.""" + """Validate dense, stride-aware, and stride-agnostic axis markers.""" array = plan.array if array is None: return () if array.order not in {None, "ORDER_F", "ORDER_C"}: return () - if array.contiguous not in {True, False}: + stride_agnostic = array.python_layout is ArrayPythonLayout.ANY_STRIDED + if (array.contiguous is None) is not stride_agnostic: return (self._diagnostic(plan.owner_path, "invalid-array-contiguity", array.contiguous),) + if stride_agnostic: + return () if array.contiguous is True and any(axis != "dense" for axis in array.axes): return (self._diagnostic(plan.owner_path, "invalid-array-axis-modes", array.axes),) if array.contiguous is False and "strided" not in array.axes: diff --git a/prik/planning/models.py b/prik/planning/models.py index cc4df12e9..b0d5cb33b 100644 --- a/prik/planning/models.py +++ b/prik/planning/models.py @@ -34,6 +34,7 @@ ArgumentConversionPhase, ArgumentHandoffMode, ArrayLogicalABI, + ArrayPythonLayout, ArrayWritebackABI, BridgeDataAction, CallbackABIKind, @@ -476,6 +477,9 @@ class ArrayHandoffPlan(StageRecord): order: str | None native_order: str | None contiguous: bool | None + python_layout: ArrayPythonLayout + minimum_rank: int + maximum_rank: int flatten_python_storage: bool flat_axis: int | None itemsize: int | None diff --git a/prik/planning/planner.py b/prik/planning/planner.py index b114d0b8e..b681ef79f 100644 --- a/prik/planning/planner.py +++ b/prik/planning/planner.py @@ -2382,6 +2382,9 @@ def _array_plan( order=policy.order, native_order=policy.native_order, contiguous=policy.contiguous, + python_layout=policy.python_layout, + minimum_rank=policy.minimum_rank, + maximum_rank=policy.maximum_rank, flatten_python_storage=policy.flatten_python_storage, flat_axis=policy.flat_axis, itemsize=policy.itemsize, diff --git a/prik/policy/construction.py b/prik/policy/construction.py index 1f9cb798c..86657f67d 100644 --- a/prik/policy/construction.py +++ b/prik/policy/construction.py @@ -26,6 +26,7 @@ ADDRESS_ROLE_METADATA, ADDRESS_ROLE_RAW, BIND_TARGET_METADATA, + NATIVE_C_ARRAY_ELEMENT_IDENTITY_METADATA, NATIVE_C_SCALAR_IDENTITY_METADATA, NULLABLE_ANNOTATION_METADATA, SCALAR_STORAGE_CATEGORY, @@ -80,6 +81,7 @@ ArrayWritebackABI, ScalarLogicalABI, ArrayLogicalABI, + ArrayPythonLayout, WritebackPhase, LifecycleOperation, TransformationLayer, @@ -1839,6 +1841,33 @@ def _complete_function_entrypoint_route( return arguments, slots, entrypoint_action, entrypoint_symbol, entrypoint_diagnostics +def _c_direct_array_element_c_type( + argument: ArgumentPolicy, + slot: NativeCallSlotPolicy | None, + semantic_argument: models.SemanticArgument | None, +) -> str | None: + """Complete the exact C element type one array buffer must already hold. + + A scalar crosses by value and is converted, so its canonical width is + enough. An array is handed to C as the buffer it already is, so its + elements must be the declared C type rather than another type of the same + width. Which of ``long`` and ``long long`` a target happens to call + ``int64_t`` therefore does not change what the buffer must contain, and one + C source keeps one accepted dtype on every target. + + An authored contract entry such as ``CLongLong(Arg(0))`` still wins, since + it states the identity the native call itself requires. + """ + if argument.ownership.kind is not ObjectKind.NUMPY_ARRAY: + return None + if slot is not None and slot.native_scalar_c_type is not None: + return slot.native_scalar_c_type + if semantic_argument is None: + return None + declared = semantic_argument.semantic_type.metadata.get(NATIVE_C_ARRAY_ELEMENT_IDENTITY_METADATA) + return declared if isinstance(declared, str) else None + + def _normalize_c_direct_scalar_identities( function: models.SemanticFunction, arguments: list[ArgumentPolicy], @@ -1874,10 +1903,10 @@ def _normalize_c_direct_scalar_identities( else None ) ), - native_array_element_c_type=( - slots_by_name[argument.name].native_scalar_c_type - if argument.ownership.kind is ObjectKind.NUMPY_ARRAY and argument.name in slots_by_name - else None + native_array_element_c_type=_c_direct_array_element_c_type( + argument, + slots_by_name.get(argument.name), + semantic_by_name.get(argument.name), ), # A C payload is bytes plus whatever length the contract passes. # Refusing an embedded NUL would impose a terminator convention @@ -2104,6 +2133,7 @@ def _complete_entrypoint_slot_policies( EntrypointProjectionAction.TYPED_LITERAL, EntrypointProjectionAction.COMPUTED_LENGTH, EntrypointProjectionAction.COMPUTED_PRESENCE, + EntrypointProjectionAction.COMPUTED_SIZE, EntrypointProjectionAction.COMPUTED_SHAPE, EntrypointProjectionAction.COMPUTED_STRIDE, }: @@ -2157,6 +2187,7 @@ def _entrypoint_projection_action(slot: NativeCallSlotPolicy) -> EntrypointProje "value": EntrypointProjectionAction.ARGUMENT_VALUE, "is_present": EntrypointProjectionAction.COMPUTED_PRESENCE, "len": EntrypointProjectionAction.COMPUTED_LENGTH, + "size": EntrypointProjectionAction.COMPUTED_SIZE, "shape": EntrypointProjectionAction.COMPUTED_SHAPE, "stride": EntrypointProjectionAction.COMPUTED_STRIDE, "work": EntrypointProjectionAction.WORK_STORAGE, @@ -2200,9 +2231,10 @@ def _direct_c_abi_ineligibility( def _direct_c_array_ineligibility(argument: ArgumentPolicy) -> tuple[str, ...]: """Validate the selected one-level C-pointer NumPy-array mechanism.""" reasons = [] - if argument.rank < 1 or argument.rank > 15: + array = argument.array + if array is None or array.minimum_rank < 0 or array.maximum_rank > 15 or array.maximum_rank < array.minimum_rank: reasons.append(f"C_DIRECT_ARRAY_RANK:{argument.name}") - if argument.handoff_mode is not ArgumentHandoffMode.ARRAY_BUFFER or argument.array is None: + if argument.handoff_mode is not ArgumentHandoffMode.ARRAY_BUFFER or array is None: reasons.append(f"C_DIRECT_ARRAY_CONTRACT:{argument.name}") if argument.entrypoint_passing is not EntrypointPassingConvention.POINTER_REFERENCE: reasons.append(f"C_DIRECT_ARRAY_PASSING:{argument.name}") @@ -2212,7 +2244,7 @@ def _direct_c_array_ineligibility(argument: ArgumentPolicy) -> tuple[str, ...]: reasons.append(f"C_DIRECT_ARRAY_TRANSFORMATION:{argument.name}") if argument.entrypoint_optionality is not EntrypointOptionalityAction.REQUIRED: reasons.append(f"C_DIRECT_NULLABLE_POINTER:{argument.name}") - if argument.rank > 1 and argument.array is not None and argument.array.order != "ORDER_C": + if array is not None and array.maximum_rank > 1 and array.order != "ORDER_C": reasons.append(f"C_DIRECT_ARRAY_ORDER:{argument.name}") return tuple(dict.fromkeys(reasons)) @@ -3674,8 +3706,13 @@ def _projected_native_call_slot_policy( if mapping.value_kind == "literal": slot, blockers = _literal_native_call_slot_policy(mapping, owner_path, native_position) return slot, None, blockers - if mapping.value_kind in {"len", "is_present", "shape", "stride", "work"}: - slot, blockers = _computed_native_call_slot_policy(mapping, owner_path, native_position) + if mapping.value_kind in {"len", "is_present", "size", "shape", "stride", "work"}: + slot, blockers = _computed_native_call_slot_policy( + mapping, + owner_path, + native_position, + visible_arguments, + ) return slot, None, blockers python_position = mapping.python_position if mapping.result_position is not None and python_position is None: @@ -3701,14 +3738,20 @@ def _computed_native_call_slot_policy( mapping: models.ProjectionMapping, owner_path: str, native_position: int, + visible_arguments: tuple[models.SemanticArgument, ...], ) -> tuple[NativeCallSlotPolicy, tuple[str, ...]]: - """Complete one binding-owned length, presence, shape, stride, or work slot.""" + """Complete one binding-owned length, presence, size, shape, stride, or work slot.""" value_kind = mapping.value_kind source_position = _projection_value_argument_position(mapping.value) semantic_type_name, cast_blockers = _computed_slot_cast_type(mapping, native_position) blockers = list(cast_blockers) if value_kind != "work" and source_position is None: blockers.append(f"native-call {value_kind} slot {native_position} has no argument source") + if value_kind == "size" and source_position is not None: + if not 0 <= source_position < len(visible_arguments): + blockers.append(f"native-call size slot {native_position} references argument position {source_position}") + elif _array_handoff_policy(visible_arguments[source_position].semantic_type) is None: + blockers.append(f"native-call size slot {native_position} requires an array argument") if value_kind == "work": blockers.append(f"native-call work slot {native_position} has no completed typed storage policy") return ( @@ -7323,20 +7366,27 @@ def _array_handoff_policy(semantic_type: models.SemanticType) -> ArrayHandoffPol return None if semantic_type.name == "String" and array.category == SCALAR_STORAGE_CATEGORY: return None - assumed_rank = array.category == "assumed_rank" - rank = _array_handoff_rank(semantic_type, array.rank, assumed_rank) + runtime_rank = array.category in {"assumed_rank", "runtime_rank"} + rank = _array_handoff_rank(semantic_type, array.rank, runtime_rank) if rank is not None and rank <= 0 and not (rank == 0 and array.category == SCALAR_STORAGE_CATEGORY): return None shape = tuple(str(item) for item in (array.shape or semantic_type.shape)) axes = tuple(str(item) for item in array.axes) + flatten_python_storage = _array_handoff_flattens_python_storage(array) + minimum_rank, maximum_rank = _array_handoff_rank_bounds(rank, array.category, flatten_python_storage) + order = _array_handoff_order(array.order, array.category) + contiguous = _array_handoff_contiguous(array.contiguous, array.category) return ArrayHandoffPolicy( rank=rank, shape=shape, axes=axes, - order=_array_handoff_order(array.order, assumed_rank), - native_order=_array_handoff_native_order(array.order, array.copy_order, assumed_rank), - contiguous=_array_handoff_contiguous(array.contiguous, assumed_rank, array.category), - flatten_python_storage=_array_handoff_flattens_python_storage(array), + order=order, + native_order=_array_handoff_native_order(array.order, array.copy_order, array.category), + contiguous=contiguous, + python_layout=_array_handoff_python_layout(order, contiguous, rank), + minimum_rank=minimum_rank, + maximum_rank=maximum_rank, + flatten_python_storage=flatten_python_storage, flat_axis=_array_handoff_flat_axis(array), itemsize=_array_handoff_itemsize(semantic_type), character=semantic_type.name == "String", @@ -7348,39 +7398,82 @@ def _array_handoff_policy(semantic_type: models.SemanticType) -> ArrayHandoffPol def _array_handoff_rank( semantic_type: models.SemanticType, storage_rank: int | None, - assumed_rank: bool, + runtime_rank: bool, ) -> int | None: - """Return the concrete rank, leaving assumed-rank selection explicit.""" - if assumed_rank: + """Return the concrete rank, leaving runtime-rank selection explicit.""" + if runtime_rank: return None return int(storage_rank or semantic_type.rank or 0) -def _array_handoff_order(order: str | None, assumed_rank: bool) -> str | None: - """Default assumed-rank buffers to native Fortran layout.""" - if assumed_rank and order is None: +def _array_handoff_order(order: str | None, category: str | None) -> str | None: + """Complete the Python layout for each runtime-rank contract.""" + if category == "assumed_rank" and order is None: return "ORDER_F" + if category == "runtime_rank" and order is None: + return "ORDER_C" return order def _array_handoff_native_order( order: str | None, copy_order: str | None, - assumed_rank: bool, + category: str | None, ) -> str | None: """Return the completed native-copy layout independently of input layout.""" - if assumed_rank and order is None: + if category == "assumed_rank" and order is None: return "ORDER_F" + if category == "runtime_rank" and order is None: + return "ORDER_C" return copy_order if copy_order is not None else order -def _array_handoff_contiguous(contiguous: bool | None, assumed_rank: bool, category: str | None) -> bool | None: - """Default assumed-rank handoff to one contiguous native buffer.""" - if category == SCALAR_STORAGE_CATEGORY and contiguous is None: - return True - if assumed_rank and contiguous is None: +def _array_handoff_contiguous(contiguous: bool | None, category: str | None) -> bool | None: + """Complete the contiguity a contract asserts, leaving C runtime rank open. + + ``None`` states that the contract asserts nothing about layout: the caller's + own strides reach the native call. Fortran assumed rank keeps its contiguous + descriptor default, and every C ``T[...]`` that did not spell ``Contiguous`` + stays stride-agnostic. + """ + if contiguous is not None: + return contiguous + if category in {SCALAR_STORAGE_CATEGORY, "assumed_rank"}: return True - return contiguous + return None + + +def _array_handoff_python_layout( + order: str | None, + contiguous: bool | None, + rank: int | None, +) -> ArrayPythonLayout: + """Select the layout every accepted Python array actual must already have.""" + if contiguous is None: + return ArrayPythonLayout.ANY_STRIDED + if contiguous is False: + return ArrayPythonLayout.POSITIVE_STRIDED_F + if order == "ORDER_C": + return ArrayPythonLayout.C_CONTIGUOUS + if order == "ORDER_F" or (rank is not None and rank > 1): + return ArrayPythonLayout.F_CONTIGUOUS + return ArrayPythonLayout.ANY_CONTIGUOUS + + +def _array_handoff_rank_bounds( + rank: int | None, + category: str | None, + flatten_python_storage: bool, +) -> tuple[int, int]: + """Complete inclusive Python rank bounds before wrapper planning.""" + if category == "runtime_rank": + return 0, 15 + if category == "assumed_rank": + return 1, 15 + concrete_rank = int(rank or 0) + if flatten_python_storage: + return concrete_rank, 15 + return concrete_rank, concrete_rank def _array_handoff_flattens_python_storage(array: models.SemanticArrayContract) -> bool: @@ -7479,6 +7572,9 @@ def _raw_array_handoff_policy(semantic_type: models.SemanticType) -> ArrayHandof order=order, native_order=order, contiguous=True, + python_layout=_array_handoff_python_layout(order, True, rank), + minimum_rank=rank, + maximum_rank=rank, itemsize=_character_length(semantic_type) if semantic_type.name == "String" else None, character=semantic_type.name == "String", category="raw_address", diff --git a/prik/policy/models.py b/prik/policy/models.py index 468e6d959..345682357 100644 --- a/prik/policy/models.py +++ b/prik/policy/models.py @@ -122,6 +122,7 @@ class EntrypointProjectionAction(str, Enum): TYPED_LITERAL = "typed_literal" COMPUTED_LENGTH = "computed_length" COMPUTED_PRESENCE = "computed_presence" + COMPUTED_SIZE = "computed_size" COMPUTED_SHAPE = "computed_shape" COMPUTED_STRIDE = "computed_stride" WORK_STORAGE = "work_storage" @@ -130,6 +131,21 @@ class EntrypointProjectionAction(str, Enum): BLOCKED = "blocked" +class ArrayPythonLayout(str, Enum): + """Completed memory layout one Python array actual must already have. + + Policy selects the constraint; a backend only enforces it. ``ANY_STRIDED`` + states that the contract constrains neither ordering nor contiguity, so the + caller's own strides reach the native call unchanged. + """ + + ANY_CONTIGUOUS = "any_contiguous" + C_CONTIGUOUS = "c_contiguous" + F_CONTIGUOUS = "f_contiguous" + POSITIVE_STRIDED_F = "positive_strided_f" + ANY_STRIDED = "any_strided" + + class ArgumentHandoffMode(str, Enum): """Completed binding-to-bridge ABI shape for one argument.""" @@ -948,6 +964,9 @@ class ArrayHandoffPolicy: order: str | None native_order: str | None contiguous: bool | None + python_layout: ArrayPythonLayout + minimum_rank: int + maximum_rank: int flatten_python_storage: bool = False flat_axis: int | None = None itemsize: int | None = None @@ -1391,6 +1410,9 @@ class FunctionWrapperPolicy: order="F", native_order="F", contiguous=True, + python_layout=ArrayPythonLayout.F_CONTIGUOUS, + minimum_rank=2, + maximum_rank=2, ) example_lifecycle = LifecyclePolicy( owner_path="math.scale.values", diff --git a/prik/printers/pyi.py b/prik/printers/pyi.py index 4eeb221f2..b7f066ef9 100644 --- a/prik/printers/pyi.py +++ b/prik/printers/pyi.py @@ -2377,6 +2377,9 @@ def _native_projection_value( f"{context.contract('Len')}({self._native_value_ref(mapping.value, context)})", context, ) + if mapping.value_kind == "size": + producer = f"{self._native_value_ref(mapping.value, context)}.size" + return self._cast_projection(mapping, producer, context) if mapping.value_kind == "shape": producer = f"{self._native_value_ref(mapping.value['value'], context)}.shape[{mapping.value['dim']}]" return self._cast_projection(mapping, producer, context) diff --git a/prik/runtime/native_support/prik_binding.h b/prik/runtime/native_support/prik_binding.h index 4b1868f7e..b4f96011f 100644 --- a/prik/runtime/native_support/prik_binding.h +++ b/prik/runtime/native_support/prik_binding.h @@ -396,6 +396,7 @@ PRIK_NO_INLINE static int prik_array_actual_unpack( #define PRIK_ARRAY_LAYOUT_C_CONTIGUOUS 1 #define PRIK_ARRAY_LAYOUT_F_CONTIGUOUS 2 #define PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F 3 +#define PRIK_ARRAY_LAYOUT_ANY_STRIDED 4 /* * Validate mechanics shared by every ordinary NumPy-array argument. The @@ -420,7 +421,7 @@ static inline int prik_array_validate_ndarray( if (minimum_rank < 0 || maximum_rank < minimum_rank || maximum_rank > PRIK_MAX_ARRAY_RANK || layout < PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS - || layout > PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F + || layout > PRIK_ARRAY_LAYOUT_ANY_STRIDED || python_type == NULL || argument_name == NULL) { PyErr_SetString(PyExc_RuntimeError, "prik generated invalid NumPy-array validation selectors"); return -1; @@ -435,7 +436,9 @@ static inline int prik_array_validate_ndarray( Py_TYPE((PyObject *)array)->tp_name); return -1; } - if (layout == PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F) { + if (layout == PRIK_ARRAY_LAYOUT_ANY_STRIDED) { + /* The plan accepts whatever strides the caller's array already has. */ + } else if (layout == PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F) { for (axis = 0; axis < rank; axis++) { npy_intp stride = PyArray_STRIDE(array, axis); if ((stride % PyArray_ITEMSIZE(array)) != 0 @@ -548,7 +551,8 @@ static inline int prik_array_validate( * minimum_rank smallest accepted runtime rank; equals `rank` unless * the plan flattens Python storage * maximum_rank largest accepted runtime rank - * layout PRIK_ARRAY_LAYOUT_* ordering the plan requires + * layout PRIK_ARRAY_LAYOUT_* ordering the plan requires; + * PRIK_ARRAY_LAYOUT_ANY_STRIDED requires none * require_contiguous non-zero when the plan requires contiguous storage * require_writeable non-zero when the plan may write through this argument * python_type public dtype name used in diagnostics, "numpy.float64" @@ -705,20 +709,28 @@ static inline int prik_int32_unpack_exact(PyObject *value, int32_t *destination) return 0; } +/* + * C long and long long are distinct NumPy scalar types even where both are + * 64 bits wide, and which one a target calls int64_t varies. A scalar crosses + * by value, so either spelling is accepted here and converted to the exact + * native storage. An array buffer cannot be converted element by element, so + * array validation stays exact. + */ static inline int prik_int64_unpack_exact(PyObject *value, int64_t *destination) { #if NPY_SIZEOF_LONG == 8 - if (!PyArray_IsScalar(value, Long)) { - return -1; + if (PyArray_IsScalar(value, Long)) { + *destination = (int64_t)PyArrayScalar_VAL(value, Long); + return 0; } - *destination = (int64_t)PyArrayScalar_VAL(value, Long); -#else - if (!PyArray_IsScalar(value, Int64)) { - return -1; +#endif +#if NPY_SIZEOF_LONGLONG == 8 + if (PyArray_IsScalar(value, LongLong)) { + *destination = (int64_t)PyArrayScalar_VAL(value, LongLong); + return 0; } - *destination = (int64_t)PyArrayScalar_VAL(value, Int64); #endif - return 0; + return -1; } static inline int prik_float32_unpack_exact(PyObject *value, float *destination) @@ -1142,20 +1154,22 @@ static inline PyObject *prik_uint32_to_numpy(const uint32_t *value) return result; } +/* Accepts either 64-bit spelling; see prik_int64_unpack_exact. */ static inline int prik_uint64_unpack_exact(PyObject *value, uint64_t *destination) { #if NPY_SIZEOF_LONG == 8 - if (!PyArray_IsScalar(value, ULong)) { - return -1; + if (PyArray_IsScalar(value, ULong)) { + *destination = (uint64_t)PyArrayScalar_VAL(value, ULong); + return 0; } - *destination = (uint64_t)PyArrayScalar_VAL(value, ULong); -#else - if (!PyArray_IsScalar(value, ULongLong)) { - return -1; +#endif +#if NPY_SIZEOF_LONGLONG == 8 + if (PyArray_IsScalar(value, ULongLong)) { + *destination = (uint64_t)PyArrayScalar_VAL(value, ULongLong); + return 0; } - *destination = (uint64_t)PyArrayScalar_VAL(value, ULongLong); #endif - return 0; + return -1; } static inline int prik_uint64_unpack(PyObject *value, uint64_t *destination) @@ -1188,25 +1202,28 @@ static inline PyObject *prik_uint64_to_numpy(const uint64_t *value) return result; } +/* Accepts every unsigned spelling of the target's pointer width. */ static inline int prik_uintp_unpack_exact(PyObject *value, size_t *destination) { #if NPY_SIZEOF_LONG == NPY_SIZEOF_INTP - if (!PyArray_IsScalar(value, ULong)) { - return -1; + if (PyArray_IsScalar(value, ULong)) { + *destination = (size_t)PyArrayScalar_VAL(value, ULong); + return 0; } - *destination = (size_t)PyArrayScalar_VAL(value, ULong); -#elif NPY_SIZEOF_INTP == 8 - if (!PyArray_IsScalar(value, ULongLong)) { - return -1; +#endif +#if NPY_SIZEOF_LONGLONG == NPY_SIZEOF_INTP + if (PyArray_IsScalar(value, ULongLong)) { + *destination = (size_t)PyArrayScalar_VAL(value, ULongLong); + return 0; } - *destination = (size_t)PyArrayScalar_VAL(value, ULongLong); -#else - if (!PyArray_IsScalar(value, UInt)) { - return -1; +#endif +#if NPY_SIZEOF_INT == NPY_SIZEOF_INTP + if (PyArray_IsScalar(value, UInt)) { + *destination = (size_t)PyArrayScalar_VAL(value, UInt); + return 0; } - *destination = (size_t)PyArrayScalar_VAL(value, UInt); #endif - return 0; + return -1; } static inline int prik_uintp_unpack(PyObject *value, size_t *destination) diff --git a/prik/semantics/c2ir.py b/prik/semantics/c2ir.py index a6ecd53b5..0af9d2759 100644 --- a/prik/semantics/c2ir.py +++ b/prik/semantics/c2ir.py @@ -15,8 +15,16 @@ from typing import Any from prik.contracts import NATIVE_C_SCALAR_IDENTITIES -from prik.semantics.metadata import EXPLICIT_C_EXPORT_METADATA, NATIVE_C_SCALAR_IDENTITY_METADATA -from prik.semantics.scalar_types import BOOLEAN_STORAGE_BITS +from prik.semantics.metadata import ( + EXPLICIT_C_EXPORT_METADATA, + NATIVE_C_ARRAY_ELEMENT_IDENTITY_METADATA, + NATIVE_C_SCALAR_IDENTITY_METADATA, +) +from prik.semantics.scalar_types import ( + BOOLEAN_STORAGE_BITS, + SEMANTIC_SCALAR_TYPE_NAMES, + is_boolean_semantic_type_name, +) from prik.parsers.c.models import ( CArray, @@ -160,6 +168,13 @@ for primitive, c_spelling in _PRIMITIVE_TYPE_FACT_NAMES.items() } +# NumPy assigns NPY_INT/NPY_UINT to the first C type of that width in +# these orders, mirroring the scan in its own npy_common.h. +_NUMPY_CANONICAL_INTEGER_SCAN = { + False: ("signed char", "short", "int", "long", "long long"), + True: ("unsigned char", "unsigned short", "unsigned int", "unsigned long", "unsigned long long"), +} + _CANONICAL_C_TYPE_FACT_NAMES = { "Bool": "_Bool", "Bool8": "_Bool", @@ -454,6 +469,7 @@ def _visit_CParameter( name = parameter.name or f"arg{position}" source_type = parameter.declared_type or parameter.type semantic_type = self.visit(source_type, owner=f"{owner or ''}.{name}", as_type=True) + semantic_type = self._runtime_rank_pointer_parameter(semantic_type) metadata: dict[str, Any] = {"native_position": position} if parameter.callback_candidate: semantic_type = self._callback_placeholder(source_type) @@ -822,6 +838,9 @@ def _primitive_type(self, type_: CType, *, owner: str | None) -> SemanticType: native_c_identity = self._required_native_c_identity(type_, dtype) if native_c_identity is not None: metadata[NATIVE_C_SCALAR_IDENTITY_METADATA] = native_c_identity + array_element = self._array_element_c_spelling(type_, dtype) + if array_element is not None: + metadata[NATIVE_C_ARRAY_ELEMENT_IDENTITY_METADATA] = array_element return SemanticType( name=semantic_name, dtype=dtype, @@ -846,6 +865,45 @@ def _required_native_c_identity(self, type_: CType, semantic_name: str) -> str | canonical_spelling = self._underlying_c_type(canonical_name) return None if self._compatible_c_scalar_spelling(source_spelling, canonical_spelling) else native_c_identity + def _array_element_c_spelling(self, type_: CType, semantic_name: str) -> str | None: + """Return the C element spelling an array of this type must hold. + + An array buffer reaches C unchanged, so its elements must be the + declared C type. The canonical spelling of a fixed-width integer is a + ```` typedef, and the C type behind that typedef need not be + the one NumPy assigns to the same width: a target whose ``int64_t`` is + ``long long`` still gives ``NPY_INT64`` to ``long`` when ``long`` is 64 + bits. Recording the declared spelling whenever it differs from NumPy's + choice keeps one accepted dtype per C spelling on every target. + """ + declared = _PRIMITIVE_TYPE_FACT_NAMES.get(type(type_)) + if declared is None: + return None + canonical = self._numpy_canonical_integer_primitive(semantic_name) + if canonical is None: + # Real and complex widths are already canonical C spellings, so the + # ordinary identity rule alone decides those. + required = self._required_native_c_identity(type_, semantic_name) + return declared if required is not None else None + return None if declared == canonical else declared + + def _numpy_canonical_integer_primitive(self, semantic_name: str) -> str | None: + """Return the C type NumPy assigns to one fixed-width integer here. + + NumPy defines ``NPY_INT64`` and friends by scanning C integer types in + width order and taking the first match, so the answer is target data. + """ + match = re.fullmatch(r"(U?)Int(8|16|32|64)", semantic_name) + if match is None: + return None + candidates = _NUMPY_CANONICAL_INTEGER_SCAN[bool(match.group(1))] + bits = int(match.group(2)) + for candidate in candidates: + fact = self.standard_type_facts.get(candidate) + if isinstance(fact, dict) and fact.get("bits") == bits: + return candidate + return None + def _underlying_c_type(self, name: str) -> str: fact = self.standard_type_facts.get(name) if isinstance(fact, dict): @@ -1051,7 +1109,7 @@ def _pointer_type( """Apply C pointer depth, qualifiers, and aliasing facts to a pointee type in place. The returned object is ``pointee`` with borrowed reference/pointer - storage. Pointee ``const`` controls mutability and ``restrict`` controls + storage. Pointee ``const`` controls mutability and ``restrict`` controls aliasing; no ownership-transfer policy is inferred. """ pointer_depth = len(pointer_components) @@ -1076,6 +1134,41 @@ def _pointer_type( pointee.ownership.aliasing = not restrict return pointee + @staticmethod + def _runtime_rank_pointer_parameter(pointee: SemanticType) -> SemanticType: + """Select the source-generated runtime-rank contract for primitive parameter ``T *``.""" + storage = pointee.storage + scalar_name = pointee.dtype or pointee.name + if not ( + storage is not None + and storage.kind == "reference" + and storage.pointer_depth == 1 + and scalar_name in SEMANTIC_SCALAR_TYPE_NAMES + and pointee.name != "String" + and not is_boolean_semantic_type_name(pointee.name) + and not is_boolean_semantic_type_name(pointee.dtype) + ): + return pointee + pointee.rank = 1 + pointee.shape = ["..."] + pointee.storage = SemanticStorageContract( + kind="array", + read_only=storage.read_only, + mutable=storage.mutable, + pointer_depth=storage.pointer_depth, + ownership=storage.ownership, + array=SemanticArrayContract( + rank=1, + shape=["..."], + source_shape=["..."], + category="runtime_rank", + order="ORDER_C", + axes=["dense"], + ), + metadata=dict(storage.metadata), + ) + return pointee + def _array_type( self, element: SemanticType, diff --git a/prik/semantics/metadata.py b/prik/semantics/metadata.py index d5a4050ce..be65dea8b 100644 --- a/prik/semantics/metadata.py +++ b/prik/semantics/metadata.py @@ -14,6 +14,9 @@ CONSTRUCTOR_SPECIFIC_METADATA = "constructor_specific" NATIVE_PROJECTION_METADATA = "native_projection" NATIVE_C_SCALAR_IDENTITY_METADATA = "native_c_scalar_identity" +# Exact C element spelling an array buffer must hold, recorded whenever it +# differs from the C type NumPy assigns to that width on this target. +NATIVE_C_ARRAY_ELEMENT_IDENTITY_METADATA = "native_c_array_element_identity" EXPLICIT_C_EXPORT_METADATA = "explicit_c_export" NATIVE_ARRAY_DESCRIPTOR_METADATA = "native_array_descriptor" NATIVE_ARRAY_HANDLE_POLICY_METADATA = "native_array_handle_policy" diff --git a/prik/semantics/pyi2ir.py b/prik/semantics/pyi2ir.py index a35ed37e9..adde496e8 100644 --- a/prik/semantics/pyi2ir.py +++ b/prik/semantics/pyi2ir.py @@ -97,7 +97,7 @@ _FLAT_DIMENSION_SENTINEL = "@prik.Flat" _STRIDED_DIMENSION_SENTINEL = "@prik.Strided" # Computed producers whose materialized type a contract may state explicitly. -_CASTABLE_PROJECTION_KINDS = frozenset({"shape", "stride", "len"}) +_CASTABLE_PROJECTION_KINDS = frozenset({"size", "shape", "stride", "len"}) # Contract-conversion state and public entrypoints @@ -1532,10 +1532,13 @@ def _validated_generic_override( def native_projection_entry(self, node: ast.AST, native_position: int) -> ProjectionMapping: """Convert one ``native_call`` list item into a positioned projection mapping. - Shape, address, descriptor, typed-literal, and named helper forms are + Array facts, address, descriptor, typed-literal, and named helper forms are dispatched here. ``native_position`` is preserved in the returned mapping; unsupported or untyped expressions fail closed. """ + size_mapping = self.native_size_projection_entry(node, native_position) + if size_mapping is not None: + return size_mapping shape_mapping = self.native_shape_projection_entry(node, native_position) if shape_mapping is not None: return shape_mapping @@ -1765,7 +1768,7 @@ def native_cast_projection_entry( native_position: int, value_cast: str, ) -> ProjectionMapping: - """Parse ``Int32(Arg(i).shape[d])``: one computed projection cast to a named type. + """Parse one computed projection cast to a named integer type. The producer is unchanged; only the type the binding materializes is stated here. Literal expressions have already been handled with @@ -1773,7 +1776,7 @@ def native_cast_projection_entry( """ mapping = self.native_projection_entry(node, native_position) if mapping.value_kind not in _CASTABLE_PROJECTION_KINDS: - raise ValueError(f"{value_cast} accepts a literal value or a shape, stride, or length projection") + raise ValueError(f"{value_cast} accepts a literal value or a size, shape, stride, or length projection") mapping.value_cast = value_cast return mapping @@ -1847,6 +1850,23 @@ def native_shape_projection_entry( }, ) + def native_size_projection_entry( + self, + node: ast.AST, + native_position: int, + ) -> ProjectionMapping | None: + """Parse ``Arg(i).size`` into a hidden total-element-count projection.""" + if not isinstance(node, ast.Attribute) or node.attr != "size": + return None + value = self.native_value_ref(node.value) + if value["kind"] != "arg": + raise ValueError("size projection expects Arg(i)") + return ProjectionMapping( + native_position=native_position, + value_kind="size", + value=value, + ) + def native_value_ref( self, node: ast.AST, @@ -2170,19 +2190,22 @@ def _array_type_from_dimensions( dims, category, source_shape, lower_bounds, upper_bounds = _PyiAstParser._flat_array_dimensions(dims) if not dims: category = SCALAR_STORAGE_CATEGORY - if dims == ["..."]: - category = "assumed_rank" - source_shape = [".."] + runtime_rank = dims == ["..."] + if runtime_rank: + category = "runtime_rank" if self.native_language == "c" else "assumed_rank" + source_shape = ["..."] if category == "runtime_rank" else [".."] if category is None and self.native_language == "fortran" and source_shape: category = "explicit_shape" - rank = 1 if category == "assumed_rank" else len(dims) + rank = 1 if category in {"assumed_rank", "runtime_rank"} else len(dims) array = SemanticArrayContract( rank=rank, shape=list(dims), order=self._array_order_for_dimensions(category, rank, source_shape), axes=["strided" if strided else "dense" for strided in strided_axes], - contiguous=not any(strided_axes), + # ``T[...]`` states no rank, so it also states no layout. Policy + # completes the language default; ``Contiguous`` still asserts one. + contiguous=None if runtime_rank else not any(strided_axes), category=category, source_shape=source_shape, lower_bounds=lower_bounds, @@ -2475,7 +2498,7 @@ def _validate_array_copy_metadata(semantic_type: SemanticType) -> None: raise ValueError("COPY_F requires a concrete multidimensional array rank") if array.copy_order != "ORDER_F" or array.order != "ORDER_C": raise ValueError("COPY_F requires a C-order Python array and targets Fortran order") - if array.category in {"assumed_size", "assumed_rank"} or array.contiguous is not True: + if array.category in {"assumed_size", "assumed_rank", "runtime_rank"} or array.contiguous is not True: raise ValueError("COPY_F initially supports only dense concrete-shape arrays") if semantic_type.name == "String" or native_array_descriptor_kind(semantic_type) is not None: raise ValueError("COPY_F does not apply to character arrays or native descriptor handles") diff --git a/pyproject.toml b/pyproject.toml index 2d6210491..5fadbcd30 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -51,6 +51,9 @@ dependencies = [ ] [project.optional-dependencies] +jupyter = [ + "ipython>=8.0", +] pretty = [ "rich>=13.7", "rich-argparse>=1.4", @@ -64,6 +67,7 @@ qa = [ "bandit[toml]==1.9.4", "coverage[toml]>=7.10", "hypothesis>=6.100", + "ipython>=8.0", "pytest>=8.0", "pytest-randomly>=3.15", "pyperf==2.10.0", diff --git a/tests/c/README.md b/tests/c/README.md index 666fc4098..47e87478d 100644 --- a/tests/c/README.md +++ b/tests/c/README.md @@ -22,13 +22,14 @@ The active owners are: | `data_types//` | C scalar type facts and compiler type probes | | `functions//` | C function declarations and their semantic projection | | `primitive_scalars//` | Direct-C primitive value policy, exact ABI identities, codegen, and compiled calls | -| `primitive_pointers//` | Authored one-level pointer, rank-zero storage, result, and C-contiguous array contracts | +| `primitive_pointers//` | One-level pointer, runtime-rank storage, rank-zero storage, result, and explicitly shaped array contracts | | `primitive_strings//` | Rank-zero string storage, hidden outputs, status projection, and compiled calls | | `symbol_collisions//` | Opt-in collision-forwarder planning, generated artifacts, and runtime behavior | | `records//` | C structs, unions, and typedefs | | `enumerations//` | C enum syntax and semantic projection | | `infrastructure/cli/` | C-input command dispatch and C-specific argument/output contracts | | `infrastructure/building/` | Direct-C build selection, artifacts, manifests, rejections, and CLI integration | +| `infrastructure/jupyter/` | Optional cell-magic routing, root-name publication, cache reuse, and compiled C-cell integration | | `infrastructure/parsing/` | C lexer, parser, project, corpus, fixture, and public-entrypoint behavior | | `infrastructure/preprocessing/` | C recipes, dependencies, mappings, execution, and diagnostics | | `infrastructure/semantic_ir/` | C parser-model conversion to semantic IR | diff --git a/tests/c/data_types/semantics/test_types_and_constants.py b/tests/c/data_types/semantics/test_types_and_constants.py index 3103ad132..176a195a3 100644 --- a/tests/c/data_types/semantics/test_types_and_constants.py +++ b/tests/c/data_types/semantics/test_types_and_constants.py @@ -58,7 +58,7 @@ ) -def test_c2ir_maps_const_and_mutable_pointers_to_storage_contracts(): +def test_c2ir_maps_primitive_pointer_parameters_to_runtime_rank_storage_contracts(): parsed = parse_c_file( "void copy(const double *src, double *dst);\n", filename="copy.h", @@ -68,21 +68,35 @@ def test_c2ir_maps_const_and_mutable_pointers_to_storage_contracts(): src, dst = copy.arguments assert src.semantic_type.name == "Float64" - assert src.semantic_type.storage.kind == "reference" + assert src.semantic_type.storage.kind == "array" assert src.semantic_type.storage.read_only is True assert dst.semantic_type.name == "Float64" - assert dst.semantic_type.storage.kind == "reference" + assert dst.semantic_type.storage.kind == "array" assert dst.semantic_type.storage.read_only is False assert asdict(src.semantic_type.ownership) == {"ownership": "borrowed", "mutable": False, "aliasing": True} assert asdict(dst.semantic_type.ownership) == {"ownership": "borrowed", "mutable": True, "aliasing": True} assert asdict(src.semantic_type.storage) == { - "kind": "reference", + "kind": "array", "read_only": True, "mutable": False, "pointer_depth": 1, "ownership": "borrowed", - "array": None, + "array": { + "rank": 1, + "shape": ["..."], + "lower_bounds": [], + "upper_bounds": [], + "source_shape": ["..."], + "category": "runtime_rank", + "order": "ORDER_C", + "copy_order": None, + "axes": ["dense"], + "contiguous": None, + "allocatable": False, + "pointer": False, + "metadata": {}, + }, "calling_convention": None, "metadata": { "c_pointer_qualifiers": [[]], @@ -91,12 +105,26 @@ def test_c2ir_maps_const_and_mutable_pointers_to_storage_contracts(): }, } assert asdict(dst.semantic_type.storage) == { - "kind": "reference", + "kind": "array", "read_only": False, "mutable": True, "pointer_depth": 1, "ownership": "borrowed", - "array": None, + "array": { + "rank": 1, + "shape": ["..."], + "lower_bounds": [], + "upper_bounds": [], + "source_shape": ["..."], + "category": "runtime_rank", + "order": "ORDER_C", + "copy_order": None, + "axes": ["dense"], + "contiguous": None, + "allocatable": False, + "pointer": False, + "metadata": {}, + }, "calling_convention": None, "metadata": { "c_pointer_qualifiers": [[]], diff --git a/tests/c/fixtures/pyi/general/basic_array_update.pyi b/tests/c/fixtures/pyi/general/basic_array_update.pyi index 8d2cdf6d7..2a6af576a 100644 --- a/tests/c/fixtures/pyi/general/basic_array_update.pyi +++ b/tests/c/fixtures/pyi/general/basic_array_update.pyi @@ -1,13 +1,12 @@ -from prik.contracts import Addr, Arg, Float64, Int, native_call +from prik.contracts import Float64, Int def add1( n: Int, x: Float64[1] ) -> None: ... -@native_call([Arg(0), Addr(Arg(1)), Arg(2)]) def add1_strided( n: Int, - x: Float64, + x: Float64[...], incx: Int ) -> None: ... diff --git a/tests/c/fixtures/pyi/general/math_api.pyi b/tests/c/fixtures/pyi/general/math_api.pyi index 4c302f62f..eefc4bd71 100644 --- a/tests/c/fixtures/pyi/general/math_api.pyi +++ b/tests/c/fixtures/pyi/general/math_api.pyi @@ -1,4 +1,4 @@ -from prik.contracts import Addr, Arg, Float64, Int, native_call +from prik.contracts import Float64, Int def norm2( n: Int, @@ -11,11 +11,10 @@ def scale( x: Float64[1] ) -> None: ... -@native_call([Arg(0), Addr(Arg(1)), Addr(Arg(2))]) def dot( n: Int, - x: Float64, - y: Float64 + x: Float64[...], + y: Float64[...] ) -> Float64: ... def fill_identity3( diff --git a/tests/c/fixtures/pyi/general/name_reuse.pyi b/tests/c/fixtures/pyi/general/name_reuse.pyi index 478fe1261..0188ececc 100644 --- a/tests/c/fixtures/pyi/general/name_reuse.pyi +++ b/tests/c/fixtures/pyi/general/name_reuse.pyi @@ -1,4 +1,4 @@ -from prik.contracts import Addr, Arg, Bool, CStruct, Complex128, Float32, Int, Int8, native_call +from prik.contracts import Bool, CStruct, Complex128, Float32, Int, Int8 class same_name(CStruct): payload: Int @@ -13,9 +13,8 @@ same_name_c: Complex128 same_name_s: Int8[8] -@native_call([Addr(Arg(0))]) def do_work_i( - same_name: Int + same_name: Int[...] ) -> None: ... def do_work_r( @@ -36,7 +35,6 @@ def convert_to_string( shared: Int8[16] ) -> Int: ... -@native_call([Addr(Arg(0))]) def convert_to_logical( - same_name: Int8 + same_name: Int8[...] ) -> Bool: ... diff --git a/tests/c/infrastructure/jupyter/end_to_end/test_c_magic_runtime.py b/tests/c/infrastructure/jupyter/end_to_end/test_c_magic_runtime.py new file mode 100644 index 000000000..29c50a2f8 --- /dev/null +++ b/tests/c/infrastructure/jupyter/end_to_end/test_c_magic_runtime.py @@ -0,0 +1,126 @@ +"""Compiled C-cell evidence for the IPython magic.""" + +from __future__ import annotations + +from pathlib import Path +import shutil + +import numpy as np +import pytest +from IPython.lib.pretty import pretty + +import prik.jupyter.magic as magic_module +from prik.jupyter.magic import PrikMagics + + +class _Shell: + def __init__(self) -> None: + self.user_ns: dict[str, object] = {} + self.next_inputs: list[str] = [] + + def push(self, values: dict[str, object]) -> None: + self.user_ns.update(values) + + def set_next_input(self, text: str, *, replace: bool = False) -> None: + assert replace is False + self.next_inputs.append(text) + + +@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") +def test_c_cell_compiles_once_and_publishes_direct_function(tmp_path: Path, monkeypatch): + build_calls = 0 + build_c_extension = magic_module.build_c_extension + + def counting_build(*args, **kwargs): + nonlocal build_calls + build_calls += 1 + return build_c_extension(*args, **kwargs) + + monkeypatch.setattr(magic_module, "build_c_extension", counting_build) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + cell = "double square(double x) { return x * x; }\n" + + magic.c("", cell) + first_function = shell.user_ns["square"] + assert first_function(np.float64(4.0)) == np.float64(16.0) + assert first_function.__module__ is None + assert pretty(first_function) == "" + + magic.c("", cell) + assert build_calls == 1 + assert shell.user_ns["square"] is first_function + + +@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") +def test_generated_c_contract_can_be_edited_then_compiled_once(tmp_path: Path, monkeypatch): + build_calls = 0 + build_pyi_extension = magic_module.build_pyi_extension + + def counting_build(*args, **kwargs): + nonlocal build_calls + build_calls += 1 + return build_pyi_extension(*args, **kwargs) + + monkeypatch.setattr(magic_module, "build_pyi_extension", counting_build) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + source = "double square(double x) { return x * x; }\n" + + magic.c("--pyi", source) + + assert len(shell.next_inputs) == 1 + magic_line, contract = shell.next_inputs[0].split("\n", 1) + contract = contract.replace( + "from prik.contracts import Float64", + "from prik.contracts import Float64, bind", + ) + contract = contract.replace("def square(", '@bind("square")\ndef squared(') + line = magic_line.removeprefix("%%pyi").strip() + + magic.pyi(line, contract) + first_function = shell.user_ns["squared"] + assert first_function(np.float64(4.0)) == np.float64(16.0) + assert first_function.__module__ is None + assert pretty(first_function) == "" + assert "square" not in shell.user_ns + + magic.pyi(line, contract) + assert build_calls == 1 + assert shell.user_ns["squared"] is first_function + + +@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") +def test_handwritten_c_contract_builds_existing_source_and_reuses_exact_cell( + tmp_path: Path, + monkeypatch, +): + build_calls = 0 + build_pyi_extension = magic_module.build_pyi_extension + + def counting_build(*args, **kwargs): + nonlocal build_calls + build_calls += 1 + return build_pyi_extension(*args, **kwargs) + + monkeypatch.setattr(magic_module, "build_pyi_extension", counting_build) + source = tmp_path / "square.c" + source.write_text("double square(double value) { return value * value; }\n", encoding="utf-8") + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + contract = """from prik.contracts import Float64 + +def square(value: Float64) -> Float64: ... +""" + line = f"--native-c-sources {source}" + + magic.pyi(line, contract) + + first_function = shell.user_ns["square"] + assert first_function(np.float64(4.0)) == np.float64(16.0) + assert first_function.__module__ is None + assert pretty(first_function) == "" + + magic.pyi(line, contract) + assert build_calls == 1 + assert shell.user_ns["square"] is first_function diff --git a/tests/c/infrastructure/jupyter/test_c_magic.py b/tests/c/infrastructure/jupyter/test_c_magic.py new file mode 100644 index 000000000..fc5c11d4b --- /dev/null +++ b/tests/c/infrastructure/jupyter/test_c_magic.py @@ -0,0 +1,189 @@ +"""Notebook magic contracts owned by C source cells.""" + +from __future__ import annotations + +import hashlib +from pathlib import Path +from types import ModuleType + +import prik.jupyter.contracts as contract_cells +import prik.jupyter.magic as magic_module +from prik.jupyter.magic import PrikMagics +from prik.pipeline.build import WrapperBuildResult + + +class _Shell: + def __init__(self) -> None: + self.user_ns: dict[str, object] = {} + self.next_inputs: list[tuple[str, bool]] = [] + + def push(self, values: dict[str, object]) -> None: + self.user_ns.update(values) + + def set_next_input(self, text: str, *, replace: bool = False) -> None: + self.next_inputs.append((text, replace)) + + +def test_c_magic_routes_compiler_flags_and_publishes_direct_declarations(tmp_path: Path, monkeypatch): + modules: dict[str, ModuleType] = {} + calls: list[tuple[Path, dict[str, object]]] = [] + + def build(source: Path, **kwargs) -> WrapperBuildResult: + calls.append((source, kwargs)) + output_dir = Path(kwargs["output_dir"]) + output_dir.mkdir(parents=True, exist_ok=True) + module_name = str(kwargs["output_name"]) + shared_library = output_dir / f"{module_name}.so" + shared_library.write_bytes(b"mock extension") + extension = ModuleType(module_name) + + def square(value): + return value * value + + extension.square = square + modules[module_name] = extension + return WrapperBuildResult( + sources=(source,), + module_name=module_name, + output_dir=output_dir, + shared_library=shared_library, + build_makefile=None, + compiled=True, + generated_sources=(), + generated_files=(), + ) + + monkeypatch.setattr(magic_module, "build_c_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + + magic.c( + '--compiler clang --native-compile-flags="-O3 -std=c11"', + "double square(double x) { return x * x; }\n", + ) + + source, kwargs = calls[0] + assert source.name == "cell.c" + assert kwargs["input_c_compiler"] == "clang" + assert kwargs["preprocessing"].compiler == "clang" + assert kwargs["native_c_flags"] == ("-O3", "-std=c11") + assert shell.user_ns["square"](4.0) == 16.0 + assert "cell" not in shell.user_ns + assert all(not name.startswith("_prik_") for name in shell.user_ns) + + +def test_generated_c_contract_has_no_artificial_filename_and_builds_against_cached_source( + tmp_path: Path, + monkeypatch, +): + modules: dict[str, ModuleType] = {} + calls: list[tuple[Path, dict[str, object]]] = [] + + def generate(path: Path, *, source_digest: str, options) -> contract_cells.GeneratedContracts: + return contract_cells.GeneratedContracts( + language="c", + source_digest=source_digest, + module_contracts={}, + direct_contract="def square(value: Float64) -> Float64: ...", + dependency_contracts={}, + ) + + def build(contract: Path, **kwargs) -> WrapperBuildResult: + calls.append((contract, kwargs)) + output_dir = Path(kwargs["output_dir"]) + output_dir.mkdir(parents=True, exist_ok=True) + module_name = str(kwargs["output_name"]) + shared_library = output_dir / f"{module_name}.so" + shared_library.write_bytes(b"mock extension") + extension = ModuleType(module_name) + extension.square = lambda value: value * value + modules[module_name] = extension + return WrapperBuildResult( + sources=(contract,), + module_name=module_name, + output_dir=output_dir, + shared_library=shared_library, + build_makefile=None, + compiled=True, + generated_sources=(), + generated_files=(), + ) + + monkeypatch.setattr(contract_cells, "generate_contracts_from_source", generate) + monkeypatch.setattr(magic_module, "build_pyi_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + source = "double square(double value) { return value * value; }\n" + + magic.c( + '--pyi --compiler clang --native-compile-flags="-O3 -std=c11"', + source, + ) + inserted = shell.next_inputs[0][0] + magic_line, editable_cell = inserted.split("\n", 1) + digest = hashlib.sha256(f"c{source}".encode()).hexdigest() + assert f"# prik: source-sha256={digest}" in editable_cell + assert " file=" not in editable_cell + + magic.pyi(magic_line.removeprefix("%%pyi").strip(), editable_cell) + + contract, kwargs = calls[0] + assert kwargs["native_language"] == "c" + assert kwargs["input_c_compiler"] == "clang" + assert kwargs["native_c_sources"] == (tmp_path / "cache" / digest / "cell.c",) + assert kwargs["native_c_flags"] == ("-O3", "-std=c11") + assert contract.name == "contract.pyi" + assert shell.user_ns["square"](4.0) == 16.0 + + +def test_handwritten_c_contract_builds_an_existing_source_without_generated_metadata( + tmp_path: Path, + monkeypatch, +): + modules: dict[str, ModuleType] = {} + calls: list[tuple[Path, dict[str, object]]] = [] + + def build(contract: Path, **kwargs) -> WrapperBuildResult: + calls.append((contract, kwargs)) + output_dir = Path(kwargs["output_dir"]) + output_dir.mkdir(parents=True, exist_ok=True) + module_name = str(kwargs["output_name"]) + shared_library = output_dir / f"{module_name}.so" + shared_library.write_bytes(b"mock extension") + extension = ModuleType(module_name) + extension.square = lambda value: value * value + modules[module_name] = extension + return WrapperBuildResult( + sources=(contract,), + module_name=module_name, + output_dir=output_dir, + shared_library=shared_library, + build_makefile=None, + compiled=True, + generated_sources=(), + generated_files=(), + ) + + monkeypatch.setattr(magic_module, "build_pyi_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + source = tmp_path / "square.c" + source.write_text("double square(double value) { return value * value; }\n", encoding="utf-8") + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + cell = "from prik.contracts import Float64\n\ndef square(value: Float64) -> Float64: ...\n" + + magic.pyi( + f'--native-c-sources {source} --compiler clang --native-compile-flags="-O3 -std=c11"', + cell, + ) + + contract, kwargs = calls[0] + assert contract.name == "contract.pyi" + assert contract.read_text(encoding="utf-8") == cell + assert kwargs["native_language"] == "c" + assert kwargs["input_c_compiler"] == "clang" + assert kwargs["native_c_sources"] == (source,) + assert kwargs["native_c_flags"] == ("-O3", "-std=c11") + assert shell.user_ns["square"](4.0) == 16.0 diff --git a/tests/c/primitive_scalars/codegen/test_direct_c_codegen.py b/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py similarity index 56% rename from tests/c/primitive_scalars/codegen/test_direct_c_codegen.py rename to tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py index 34ca13b27..d2fdf705a 100644 --- a/tests/c/primitive_scalars/codegen/test_direct_c_codegen.py +++ b/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py @@ -1,13 +1,14 @@ -"""C declaration provenance is consumed by direct binding generation.""" +"""Direct C pointer lowering consumes completed runtime-rank policy.""" from prik.parsers.c import parse_c_file from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner from prik.policy.completion import complete_semantic_policies +from prik.policy.models import ArrayPythonLayout from prik.semantics.c2ir import c_file_to_semantic_module -def test_direct_c_binding_keeps_qualifiers_pointer_depth_and_user_symbol(): +def test_direct_c_binding_keeps_pointer_abi_and_uses_completed_runtime_rank_bounds(): module = c_file_to_semantic_module( parse_c_file("double native_read(const double *input) { return *input; }", filename="read.c") ) @@ -15,9 +16,15 @@ def test_direct_c_binding_keeps_qualifiers_pointer_depth_and_user_symbol(): plan = WrapperPlanner().build(module) generated = WrapperGenerator().generate(plan) binding = next(source.text for source in generated.sources if source.path.suffix == ".c") + function = plan.namespaces[0].functions[0] + array = function.arguments[0].array assert plan.bridge is None assert plan.entrypoint.native_languages == ("c",) + assert array.rank is None + assert (array.minimum_rank, array.maximum_rank) == (0, 15) assert "double native_read(const double * input);" in binding - assert "native_read(&bound_input)" in binding + assert array.python_layout is ArrayPythonLayout.ANY_STRIDED + assert ("prik_array_validate(bound_input_obj, NPY_FLOAT64, 0, 15, PRIK_ARRAY_LAYOUT_ANY_STRIDED, 0, 0") in binding + assert "result = native_read(bound_input);" in binding assert "bind_c_read_wrapper" not in binding diff --git a/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py b/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py index b337f0a66..939feff12 100644 --- a/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py +++ b/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_contracts.py @@ -57,7 +57,7 @@ def test_generated_c_int_array_uses_its_probed_primitive_storage(tmp_path: Path) @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_c_pointer_supports_default_scalar_reference_and_edited_c_array_contracts(tmp_path: Path): +def test_c_pointer_supports_explicit_scalar_reference_and_exact_array_contracts(tmp_path: Path): contract = tmp_path / "pointers.pyi" contract.write_text( """from prik.contracts import Addr, Arg, Float64, Int32, Returns, native_call @@ -107,6 +107,110 @@ def scale_matrix(values: Float64[2, 2]) -> None: ... module.scale_matrix(np.asfortranarray(matrix)) +@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") +def test_runtime_rank_c_pointer_uses_total_size_for_rank_zero_and_ranked_storage(tmp_path: Path): + contract = tmp_path / "runtime_rank.pyi" + contract.write_text( + """from prik.contracts import Arg, Float64, native_call + +@native_call([Arg(0).size, Arg(0)]) +def scale(values: Float64[...]) -> None: ... +""", + encoding="utf-8", + ) + source = tmp_path / "runtime_rank.c" + source.write_text( + """#include +void scale(size_t count, double *values) { + for (size_t index = 0; index < count; ++index) values[index] *= 2.0; +} +""", + encoding="utf-8", + ) + + result = build_pyi_extension( + contract, + native_language="c", + native_c_sources=[source], + output_dir=tmp_path / "build_runtime_rank", + output_name="runtime_rank", + ) + module = sole_native_module(result.import_module()) + binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + + assert "void scale(size_t size_0, double * values);" in binding + assert "(size_t)PyArray_SIZE((PyArrayObject *)bound_values_obj)" in binding + + zero = np.array(3.0, dtype=np.float64) + vector = np.array([1.0, 2.0, 3.0], dtype=np.float64) + matrix = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float64, order="C") + empty = np.empty((2, 0), dtype=np.float64) + for values in (zero, vector, matrix, empty): + expected = values.copy() * 2.0 + assert module.scale(values) is None + np.testing.assert_allclose(values, expected) + + # Runtime-rank storage constrains neither rank nor strides, so a + # Fortran-ordered actual reaches the same contiguous buffer. + fortran = np.asfortranarray(matrix) + expected = fortran.copy() * 2.0 + assert module.scale(fortran) is None + np.testing.assert_allclose(fortran, expected) + + with pytest.raises(TypeError, match=r"numpy\.ndarray"): + module.scale(np.float64(3.0)) + with pytest.raises(TypeError, match=r"compatible numpy\.ndarray"): + module.scale(np.ones((1,) * 16, dtype=np.float64)) + + +@pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") +def test_runtime_rank_c_pointer_passes_a_strided_view_with_its_projected_layout(tmp_path: Path): + """``T[...]`` states no layout, so projected extents and strides carry it.""" + contract = tmp_path / "strided_rank.pyi" + contract.write_text( + """from prik.contracts import Arg, Float64, Int64, native_call + +@native_call([Arg(0).shape[0], Int64(Arg(0).strides[0]), Arg(0)]) +def scale(values: Float64[...]) -> None: ... +""", + encoding="utf-8", + ) + source = tmp_path / "strided_rank.c" + source.write_text( + """#include +void scale(size_t count, long long stride_bytes, double *values) { + char *base = (char *)values; + for (size_t index = 0; index < count; ++index) { + *(double *)(base + (ptrdiff_t)index * (ptrdiff_t)stride_bytes) *= 2.0; + } +} +""", + encoding="utf-8", + ) + + result = build_pyi_extension( + contract, + native_language="c", + native_c_sources=[source], + output_dir=tmp_path / "build_strided_rank", + output_name="strided_rank", + ) + module = sole_native_module(result.import_module()) + binding = next(path.read_text(encoding="utf-8") for path in result.generated_sources if path.suffix == ".c") + + assert "PRIK_ARRAY_LAYOUT_ANY_STRIDED" in binding + assert "(int64_t)PyArray_STRIDE((PyArrayObject *)bound_values_obj, 0)" in binding + + base = np.arange(6, dtype=np.float64) + assert module.scale(base[::2]) is None + np.testing.assert_allclose(base, np.array([0.0, 1.0, 4.0, 3.0, 8.0, 5.0])) + + # A projected axis cannot exist on rank-zero storage, so the caller is told + # instead of the binding reading past the actual's shape. + with pytest.raises(TypeError, match="has no axis 0"): + module.scale(np.array(1.0, dtype=np.float64)) + + @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") def test_edited_c_array_contract_can_derive_the_native_extent_from_its_shape(tmp_path: Path): """The documented promotion hides the count behind ``Arg(0).shape[0]``. @@ -189,6 +293,11 @@ def increment_zero(value: Int64[()]) -> None: ... scalar = module.increment_scalar(np.int64(4)) assert scalar == np.int64(5) assert scalar.dtype == np.dtype(np.int64) + # A scalar is converted rather than aliased, so either 64-bit spelling is + # accepted and cast to the exact native storage the call needs. + exact = module.increment_scalar(np.longlong(4)) + assert exact == np.int64(5) + assert exact.dtype == np.dtype(np.int64) values = np.array([1, 2, 3], dtype=np.longlong) assert module.increment(values, np.int32(values.size)) is None diff --git a/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_matrix.py b/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_matrix.py index 5a572c87b..00333d829 100644 --- a/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_matrix.py +++ b/tests/c/primitive_pointers/end_to_end/test_direct_c_pointer_matrix.py @@ -1,4 +1,4 @@ -"""Compiled all-primitive evidence for the conservative C pointer contracts.""" +"""Compiled all-primitive evidence for source-generated C pointer contracts.""" import shutil from pathlib import Path @@ -10,7 +10,13 @@ from prik.parsers.c import parse_c_file from prik.preprocessing import PreprocessingConfig from prik.preprocessing.probes.c_types import probe_c_standard_types +from prik.codegen.primitive_scalar_types import NativeCArrayStorageRegistry +from prik.contracts import NATIVE_C_SCALAR_IDENTITIES from prik.semantics.c2ir import c_file_to_semantic_module +from prik.semantics.metadata import ( + NATIVE_C_ARRAY_ELEMENT_IDENTITY_METADATA, + NATIVE_C_SCALAR_IDENTITY_METADATA, +) from tests.c._support.runtime import sole_native_module @@ -55,6 +61,27 @@ } +def _accepted_pointer_dtype(semantic_type, value) -> np.dtype: + """Return the NumPy storage one generated pointer argument accepts. + + An array buffer reaches C as it already is, so its elements are the + declared C type rather than another type of the same width. That keeps one + accepted dtype per C spelling on every target, however the target resolves + ``int64_t``. Only an element with no probed C primitive falls back to the + canonical storage of the semantic type PRIK resolved. + """ + identity = semantic_type.metadata.get(NATIVE_C_SCALAR_IDENTITY_METADATA) + declared = ( + NATIVE_C_SCALAR_IDENTITIES[identity] + if identity is not None + else semantic_type.metadata.get(NATIVE_C_ARRAY_ELEMENT_IDENTITY_METADATA) + ) + if not isinstance(declared, str): + return np.asarray(value).dtype + exact = NativeCArrayStorageRegistry.type_for(declared, semantic_type.name) + return np.dtype(getattr(np, exact.python_type_name.removeprefix("numpy."))) + + def _pointer_source() -> str: declarations = ["#include ", "#include "] for name, c_type in _C_PRIMITIVES: @@ -68,13 +95,14 @@ def _pointer_source() -> str: @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_every_c_primitive_pointer_defaults_to_one_call_local_scalar(tmp_path: Path): - """Source C retains ``T *``/``const T *`` as scalar ``Addr(Arg(i))`` calls.""" +def test_non_boolean_c_primitive_pointers_default_to_runtime_rank_numpy_storage(tmp_path: Path): + """Source ``T *`` accepts rank-zero and ranked storage without selecting a fixed rank.""" source = tmp_path / "pointer_defaults.c" source.write_text(_pointer_source(), encoding="utf-8") report = probe_c_standard_types(PreprocessingConfig(mode="compiler", compiler="cc")) semantic = c_file_to_semantic_module(parse_c_file(source), standard_type_report=report) result_types = {function.name: function.return_type.dtype for function in semantic.functions} + functions = {function.name: function for function in semantic.functions} # A typedef spelling such as ``size_t`` is declared through the builtin the # probe resolved it to, because the binding writes the prototype itself. declared_types = { @@ -88,13 +116,22 @@ def test_every_c_primitive_pointer_defaults_to_one_call_local_scalar(tmp_path: P for name, c_type in _C_PRIMITIVES: value = _VALUES[result_types[f"pointer_read_{name}"]] for prefix in ("pointer_read", "const_pointer_read"): - output = getattr(module, f"{prefix}_{name}")(value) - if type(value) is bool: - assert type(output) is bool - assert output is value + function_name = f"{prefix}_{name}" + function = functions[function_name] + call = getattr(module, function_name) + if name == "bool": + assert function.arguments[0].semantic_type.storage.kind == "reference" + output = call(value) + assert type(output) is bool and output is value else: - assert output.dtype == np.asarray(value).dtype - assert output == value + array = function.arguments[0].semantic_type.storage.array + assert array.category == "runtime_rank" + assert array.shape == ["..."] + dtype = _accepted_pointer_dtype(function.arguments[0].semantic_type, value) + for storage in (np.array(value, dtype=dtype), np.array([value], dtype=dtype)): + output = call(storage) + assert output.dtype == np.asarray(value).dtype + assert output == value declared = declared_types[c_type] assert f"{declared} pointer_read_{name}({declared} * value);" in binding assert f"{declared} const_pointer_read_{name}(const {declared} * value);" in binding diff --git a/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py b/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py new file mode 100644 index 000000000..e5b4ec126 --- /dev/null +++ b/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py @@ -0,0 +1,53 @@ +"""Completed-policy evidence for runtime-rank C pointer storage.""" + +from prik.pipeline.pyi import pyi_text_to_semantic_module +from prik.policy.completion import complete_semantic_policies +from prik.policy.models import ArrayPythonLayout, EntrypointPassingConvention, EntrypointProjectionAction +from prik.semantics.native_contract import validate_pyi_native_contract + + +def test_c_runtime_rank_and_total_size_are_complete_before_planning(): + module = pyi_text_to_semantic_module( + """from prik.contracts import Arg, Float64, native_call +@native_call([Arg(0).size, Arg(0)]) +def scale(values: Float64[...]) -> None: ... +""", + module_name="runtime_rank", + native_language="c", + ) + validate_pyi_native_contract([module]) + complete_semantic_policies(module) + + policy = module.functions[0].metadata["resolved_function_wrapper_policy"] + array = policy.arguments[0].array + size_slot = policy.native_call_slots[0] + + assert array.rank is None + assert (array.minimum_rank, array.maximum_rank) == (0, 15) + assert array.order == "ORDER_C" + assert array.native_order == "ORDER_C" + assert array.contiguous is None + assert array.python_layout is ArrayPythonLayout.ANY_STRIDED + assert size_slot.semantic_type_name == "SizeT" + assert size_slot.projection_action is EntrypointProjectionAction.COMPUTED_SIZE + assert size_slot.entrypoint_passing is EntrypointPassingConvention.C_VALUE + + +def test_contiguous_narrows_runtime_rank_storage_to_the_c_order_layout(): + """``T[...]`` states no layout, so ``Contiguous`` is what asserts one.""" + module = pyi_text_to_semantic_module( + """from prik.contracts import Annotated, Contiguous, Float64 +def scale(values: Annotated[Float64[...], Contiguous]) -> None: ... +""", + module_name="contiguous_rank", + native_language="c", + ) + validate_pyi_native_contract([module]) + complete_semantic_policies(module) + + array = module.functions[0].metadata["resolved_function_wrapper_policy"].arguments[0].array + + assert array.rank is None + assert (array.minimum_rank, array.maximum_rank) == (0, 15) + assert array.contiguous is True + assert array.python_layout is ArrayPythonLayout.C_CONTIGUOUS diff --git a/tests/c/primitive_pointers/semantics/test_starter_contracts.py b/tests/c/primitive_pointers/semantics/test_starter_contracts.py index bb9e74e21..a6e020b2a 100644 --- a/tests/c/primitive_pointers/semantics/test_starter_contracts.py +++ b/tests/c/primitive_pointers/semantics/test_starter_contracts.py @@ -1,4 +1,4 @@ -"""Public C starter contracts preserve pointer ambiguity for author edits.""" +"""Public C starter contracts expose pointer ambiguity as editable runtime rank.""" import subprocess import sys @@ -32,9 +32,9 @@ def test_c_starter_contract_preserves_every_documented_pointer_row(tmp_path: Pat contract = output.read_text(encoding="utf-8") assert "def by_value(\n value: Float64\n) -> Float64" in contract - assert contract.count("@native_call([Addr(Arg(0))])") == 2 - assert "def scalar_reference(\n value: Float64\n) -> None" in contract - assert "def const_scalar_reference(\n value: Float64\n) -> None" in contract + assert "@native_call" not in contract + assert "def scalar_reference(\n value: Float64[...]\n) -> None" in contract + assert "def const_scalar_reference(\n value: Float64[...]\n) -> None" in contract assert "def unsupported_multiple_reference(\n value: Addr[2](Float64)\n) -> None" in contract assert "def primitive_result() -> Float64" in contract assert "def unsupported_pointer_result() -> Addr(Float64)" in contract diff --git a/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py b/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py index 261684f84..c95929ba0 100644 --- a/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py +++ b/tests/c/primitive_scalars/end_to_end/test_direct_c_runtime.py @@ -24,7 +24,7 @@ def test_c_source_build_calls_renamed_user_symbol_without_a_fortran_adapter(tmp_ module = sole_native_module(result.import_module()) assert module.native_add(np.float64(1.5), np.float64(2.0)) == np.float64(3.5) - assert module.native_scale(np.float64(3.0)) == np.float64(6.0) + assert module.native_scale(np.array(3.0, dtype=np.float64)) == np.float64(6.0) assert all(path.suffix != ".f90" for path in result.generated_sources) binding = next(path for path in result.generated_sources if path.suffix == ".c") text = binding.read_text(encoding="utf-8") @@ -199,19 +199,27 @@ def test_c_source_directives_are_expanded_before_the_wrapper_reads_declarations( @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") -def test_default_c_pointer_scalar_documents_that_native_mutation_is_discarded(tmp_path: Path): - """The conservative ``T *`` default passes a call-local scalar address.""" - source = tmp_path / "discarded.c" +def test_default_c_pointer_writes_back_into_caller_owned_runtime_rank_storage(tmp_path: Path): + """The ``T *`` default passes caller-owned storage of whatever rank arrived.""" + source = tmp_path / "in_place.c" source.write_text("void twice(double *value) { *value *= 2.0; }\n", encoding="utf-8") - result = build_c_extension(source, output_dir=tmp_path / "build", output_name="c_discarded") + result = build_c_extension(source, output_dir=tmp_path / "build", output_name="c_in_place") module = sole_native_module(result.import_module()) - value = np.float64(3.0) + value = np.array(3.0, dtype=np.float64) assert module.twice(value) is None - assert value == np.float64(3.0) - assert "The update is not visible in Python." in module.twice.__doc__ - assert "update the supplied storage in place" not in module.twice.__doc__ + assert value == np.float64(3.0) * 2 + + values = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float64) + module.twice(values) + assert values[0, 0] == np.float64(2.0) + + assert "Rank: 0..15" in module.twice.__doc__ + assert "Layout: Any strides" in module.twice.__doc__ + assert "update the supplied storage in place" in module.twice.__doc__ + with pytest.raises(TypeError, match=r"numpy\.float64 for argument value"): + module.twice(np.float64(3.0)) @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") @@ -235,7 +243,7 @@ def test_c_typedef_declarations_resolve_to_their_exact_underlying_builtin(tmp_pa assert "long alias_offset(const long * value);" in binding assert "my_int" not in binding assert module.alias_step(np.int64(4)) == np.int64(5) - assert module.alias_offset(np.int64(4)) == np.int64(5) + assert module.alias_offset(np.array(4, dtype=np.int64)) == np.int64(5) @pytest.mark.skipif(shutil.which("cc") is None, reason="requires a C compiler") diff --git a/tests/c/primitive_scalars/end_to_end/test_direct_c_scalar_matrix.py b/tests/c/primitive_scalars/end_to_end/test_direct_c_scalar_matrix.py index e097229c0..7c0612554 100644 --- a/tests/c/primitive_scalars/end_to_end/test_direct_c_scalar_matrix.py +++ b/tests/c/primitive_scalars/end_to_end/test_direct_c_scalar_matrix.py @@ -83,6 +83,7 @@ def test_all_documented_c_arithmetic_spellings_return_exact_numpy_scalar_dtypes( assert module.no_result() is None with pytest.raises(TypeError, match=r"numpy\.uint8"): module.unsigned_char_identity(np.uint16(256)) - if np.dtype(np.int64).num != np.dtype(np.longlong).num: - with pytest.raises(TypeError, match=r"numpy\.int64"): - module.long_long_identity(np.longlong(1)) + # A scalar crosses by value, so either 64-bit spelling is accepted and cast + # to the exact native storage. Only an array buffer stays exact. + assert module.long_long_identity(np.longlong(1)) == np.int64(1) + assert module.long_long_identity(np.int64(1)) == np.int64(1) diff --git a/tests/fortran/README.md b/tests/fortran/README.md index 0214317e4..611befb16 100644 --- a/tests/fortran/README.md +++ b/tests/fortran/README.md @@ -64,6 +64,7 @@ test. | [Semantics Stage](../../docs/developer/packages/semantics.md) | `infrastructure/semantic_ir/` | `python3 -m pytest -q tests/fortran/infrastructure/semantic_ir` | | [`.pyi` Format](../../docs/user/reference/pyi-format.md) and [contract guides](../../docs/user/reference/pyi-contracts/index.md) | `infrastructure/semantic_pyi/` | `python3 -m pytest -q tests/fortran/infrastructure/semantic_pyi` | | [Building the Shared Library](../../docs/user/guide/building-shared-library.md) | `infrastructure/building/` | `python3 -m pytest -q tests/fortran/infrastructure/building` | +| [IPython and Jupyter Notebooks](../../docs/user/guide/notebooks.md) | `infrastructure/jupyter/` | `python3 -m pytest -q tests/fortran/infrastructure/jupyter` | | Completed ownership and wrapper-policy decisions | `infrastructure/policy/` | `python3 -m pytest -q tests/fortran/infrastructure/policy` | ## Infrastructure owners @@ -82,6 +83,7 @@ representation is supporting evidence, not the ownership rule. | `infrastructure/cli/` | Shared command-line parsing and output behavior | | `infrastructure/semantic_ir/` | Source and parser-model conversion into semantic IR | | `infrastructure/semantic_pyi/` | Semantic `.pyi` parsing, conversion, contracts, and loading | +| `infrastructure/jupyter/` | Optional cell-magic parsing, cache reuse, namespace publication, and compiled Fortran-cell integration | | `infrastructure/building/` | Shared native build modes, compiler integration, and runtime ABI behavior | | `infrastructure/policy/` | Internal ownership, policy completion, and completed wrapper-policy mechanics | | `infrastructure/codegen/` | Internal plan, planner, generator, binding, bridge, docstring, advisory review, and visitor mechanics | diff --git a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py index 89e34154a..502b60480 100644 --- a/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py +++ b/tests/fortran/infrastructure/building/end_to_end/test_source_build_modes.py @@ -287,6 +287,7 @@ def test_fortran_wrapper_out_names_importable_shared_library(tmp_path: Path): finally: sys.path.remove(str(tmp_path)) assert module.scale(np.float64(3.0), np.float64(2.5)) == np.float64(7.5) + assert module.scale.__module__ == "SCALE" def test_documented_readme_points_example_builds_and_imports(tmp_path: Path): @@ -320,6 +321,8 @@ def test_documented_readme_points_example_builds_and_imports(tmp_path: Path): try: geometry = importlib.import_module("geometry") points = geometry.points + assert points.__name__ == "geometry.points" + assert points.norm_squared.__module__ == "geometry.points" item = points.point(x=np.float64(3.0), y=np.float64(4.0)) points.move(item, np.float64(1.0), np.float64(-2.0)) assert item.x == np.float64(4.0) diff --git a/tests/fortran/infrastructure/jupyter/end_to_end/test_fortran_magic_runtime.py b/tests/fortran/infrastructure/jupyter/end_to_end/test_fortran_magic_runtime.py new file mode 100644 index 000000000..7c6c0e76b --- /dev/null +++ b/tests/fortran/infrastructure/jupyter/end_to_end/test_fortran_magic_runtime.py @@ -0,0 +1,278 @@ +"""Compiled Fortran-cell evidence for the IPython magic.""" + +from __future__ import annotations + +from pathlib import Path +import shutil +import sys + +from IPython.core.interactiveshell import InteractiveShell +from IPython.lib.pretty import pretty +import numpy as np +import pytest + +import prik.jupyter.magic as magic_module +from prik.jupyter import load_ipython_extension + + +pytestmark = pytest.mark.fortran_end_to_end + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_fortran_cell_compiles_once_and_publishes_its_declared_module(tmp_path: Path, monkeypatch): + build_calls = 0 + build_fortran_extension = magic_module.build_fortran_extension + + def counting_build(*args, **kwargs): + nonlocal build_calls + build_calls += 1 + return build_fortran_extension(*args, **kwargs) + + monkeypatch.setattr(magic_module, "build_fortran_extension", counting_build) + monkeypatch.setenv("PRIK_CACHE_DIR", str(tmp_path / "cache")) + monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) + shell = InteractiveShell() + load_ipython_extension(shell) + cell = """module maths +contains + real(8) function square(x) + real(8), intent(in) :: x + square = x*x + end function +end module +""" + + shell.run_cell_magic("fortran", "", cell) + first_namespace = shell.user_ns["maths"] + assert first_namespace.square(np.float64(4.0)) == np.float64(16.0) + assert first_namespace.__name__ == "maths" + assert first_namespace.square.__module__ == "maths" + assert pretty(first_namespace.square) == "" + + shell.run_cell_magic("fortran", "", cell) + assert build_calls == 1 + assert shell.user_ns["maths"] is first_namespace + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_published_names_carry_no_private_cache_module_identity(tmp_path: Path, monkeypatch): + """A cell's private cache module name must not reach anything the user sees. + + The extension is imported under a cache name derived from the cell digest. + Every published object is renamed to what the session actually binds, + including the generated heap type that carries module variables. + """ + monkeypatch.setenv("PRIK_CACHE_DIR", str(tmp_path / "cache")) + monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) + shell = InteractiveShell() + load_ipython_extension(shell) + + shell.run_cell_magic( + "fortran", + "", + """module cfg + real(8) :: gain = 2.0d0 +contains + real(8) function scaled(x) + real(8), intent(in) :: x + scaled = gain*x + end function +end module +""", + ) + namespace = shell.user_ns["cfg"] + + assert namespace.scaled(np.float64(3.0)) == np.float64(6.0) + assert namespace.gain == np.float64(2.0) + assert namespace.__name__ == "cfg" + assert namespace.scaled.__module__ == "cfg" + # The module-variable namespace is an instance of a generated heap type, + # whose own name embeds the private root until it is restated too. + assert type(namespace).__module__ == "cfg" + identities = ( + repr(namespace), + repr(type(namespace)), + pretty(namespace.scaled), + str(namespace.scaled.__module__), + str(type(namespace).__module__), + ) + # ``__prik_module_type`` is PRIK's own type marker; the cache module name + # is what must not appear. + assert not any("_prik_f_" in identity for identity in identities), identities + # The private name still owns the import registration. + assert any(name.startswith("_prik_f_") for name in sys.modules) + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_generated_fortran_contract_can_be_edited_then_compiled_once(tmp_path: Path, monkeypatch): + build_calls = 0 + build_pyi_extension = magic_module.build_pyi_extension + + def counting_build(*args, **kwargs): + nonlocal build_calls + build_calls += 1 + return build_pyi_extension(*args, **kwargs) + + monkeypatch.setattr(magic_module, "build_pyi_extension", counting_build) + monkeypatch.setenv("PRIK_CACHE_DIR", str(tmp_path / "cache")) + monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) + shell = InteractiveShell() + inserted: list[str] = [] + monkeypatch.setattr(shell, "set_next_input", lambda text, replace=False: inserted.append(text)) + load_ipython_extension(shell) + source = """module maths +contains + real(8) function square(x) + real(8), intent(in) :: x + square = x*x + end function +end module +""" + + shell.run_cell_magic("fortran", "--pyi", source) + + assert len(inserted) == 1 + magic_line, contract = inserted[0].split("\n", 1) + contract = contract.replace( + "from prik.contracts import Addr, Arg, Float64, native_call", + "from prik.contracts import Addr, Arg, Float64, bind, native_call", + ) + contract = contract.replace("@native_call", '@bind("square")\n@native_call') + contract = contract.replace("def square(", "def squared(") + line = magic_line.removeprefix("%%pyi").strip() + + shell.run_cell_magic("pyi", line, contract) + first_namespace = shell.user_ns["maths"] + assert first_namespace.squared(np.float64(4.0)) == np.float64(16.0) + assert first_namespace.squared.__module__ == "maths" + assert pretty(first_namespace.squared) == "" + assert not hasattr(first_namespace, "square") + + shell.run_cell_magic("pyi", line, contract) + assert build_calls == 1 + assert shell.user_ns["maths"] is first_namespace + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_handwritten_contract_cells_independently_expose_two_modules_from_one_file( + tmp_path: Path, + monkeypatch, +): + build_calls = 0 + build_pyi_extension = magic_module.build_pyi_extension + + def counting_build(*args, **kwargs): + nonlocal build_calls + build_calls += 1 + return build_pyi_extension(*args, **kwargs) + + monkeypatch.setattr(magic_module, "build_pyi_extension", counting_build) + monkeypatch.setenv("PRIK_CACHE_DIR", str(tmp_path / "cache")) + monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) + source = tmp_path / "geometry.f90" + source.write_text( + """module maths +contains + real(8) function square(x) + real(8), intent(in) :: x + square = x*x + end function +end module + +module maths2 +contains + real(8) function square(x) + real(8), intent(in) :: x + square = x*x + end function +end module +""", + encoding="utf-8", + ) + shell = InteractiveShell() + load_ipython_extension(shell) + shell.user_ns.pop("maths", None) + shell.user_ns.pop("maths2", None) + contract = """# prik: file={module}.pyi + +from prik.contracts import Addr, Arg, Float64, native_call + +@native_call([Addr(Arg(0))]) +def square(x: Float64) -> Float64: ... +""" + line = f"--native-fortran-sources {source}" + + shell.run_cell_magic("pyi", line, contract.format(module="maths")) + + first_namespace = shell.user_ns["maths"] + assert first_namespace.square(np.float64(3.0)) == np.float64(9.0) + assert pretty(first_namespace.square) == "" + assert "maths2" not in shell.user_ns + + shell.run_cell_magic("pyi", line, contract.format(module="maths2")) + + assert shell.user_ns["maths"] is first_namespace + assert shell.user_ns["maths2"].square(np.float64(4.0)) == np.float64(16.0) + assert pretty(shell.user_ns["maths2"].square) == "" + assert build_calls == 2 + + shell.run_cell_magic("pyi", line, contract.format(module="maths")) + assert build_calls == 2 + assert shell.user_ns["maths"] is first_namespace + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_handwritten_standalone_contract_builds_existing_fortran_source(tmp_path: Path, monkeypatch): + monkeypatch.setenv("PRIK_CACHE_DIR", str(tmp_path / "cache")) + monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) + source = tmp_path / "square.f90" + source.write_text( + """real(8) function square(x) + real(8), intent(in) :: x + square = x*x +end function +""", + encoding="utf-8", + ) + shell = InteractiveShell() + load_ipython_extension(shell) + shell.user_ns.pop("square", None) + contract = """from prik.contracts import Addr, Arg, Float64, native_call, standalone + +@standalone +@native_call([Addr(Arg(0))]) +def square(x: Float64) -> Float64: ... +""" + + shell.run_cell_magic("pyi", f"--native-fortran-sources {source}", contract) + + assert shell.user_ns["square"](np.float64(5.0)) == np.float64(25.0) + assert shell.user_ns["square"].__module__ is None + assert pretty(shell.user_ns["square"]) == "" + + +@pytest.mark.skipif(shutil.which("gfortran") is None, reason="requires gfortran") +def test_generated_standalone_contract_publishes_direct_function(tmp_path: Path, monkeypatch): + monkeypatch.setenv("PRIK_CACHE_DIR", str(tmp_path / "cache")) + monkeypatch.setenv("IPYTHONDIR", str(tmp_path / "ipython")) + shell = InteractiveShell() + inserted: list[str] = [] + monkeypatch.setattr(shell, "set_next_input", lambda text, replace=False: inserted.append(text)) + load_ipython_extension(shell) + source = """real(8) function square(x) + real(8), intent(in) :: x + square = x*x +end function +""" + + shell.run_cell_magic("fortran", "--pyi", source) + + magic_line, contract = inserted[0].split("\n", 1) + assert " file=" not in contract + assert "@standalone" in contract + shell.run_cell_magic("pyi", magic_line.removeprefix("%%pyi").strip(), contract) + + assert shell.user_ns["square"](np.float64(4.0)) == np.float64(16.0) + assert shell.user_ns["square"].__module__ is None + assert pretty(shell.user_ns["square"]) == "" + assert "cell" not in shell.user_ns diff --git a/tests/fortran/infrastructure/jupyter/test_fortran_magic.py b/tests/fortran/infrastructure/jupyter/test_fortran_magic.py new file mode 100644 index 000000000..f24cc9d66 --- /dev/null +++ b/tests/fortran/infrastructure/jupyter/test_fortran_magic.py @@ -0,0 +1,556 @@ +"""Notebook magic contracts shared by Fortran source cells.""" + +from __future__ import annotations + +import hashlib +from pathlib import Path +from types import ModuleType + +from IPython.core.error import UsageError +import pytest + +import prik.jupyter.contracts as contract_cells +import prik.jupyter.magic as magic_module +from prik.jupyter import load_ipython_extension +from prik.jupyter.magic import PrikMagics +from prik.pipeline.build import WrapperBuildResult + + +class _Shell: + def __init__(self) -> None: + self.user_ns: dict[str, object] = {} + self.next_inputs: list[tuple[str, bool]] = [] + + def push(self, values: dict[str, object]) -> None: + self.user_ns.update(values) + + def set_next_input(self, text: str, *, replace: bool = False) -> None: + self.next_inputs.append((text, replace)) + + +class _TerminalShell: + def __init__(self) -> None: + self.user_ns: dict[str, object] = {} + self.rl_next_input: str | None = None + + def push(self, values: dict[str, object]) -> None: + self.user_ns.update(values) + + def set_next_input(self, text: str, *, replace: bool = False) -> None: + assert replace is False + self.rl_next_input = text + + def take_next_input(self) -> str: + assert self.rl_next_input is not None + text = self.rl_next_input + self.rl_next_input = None + return text + + +def _mock_build_result( + source: Path, + kwargs: dict[str, object], + modules: dict[str, ModuleType], + *, + public_name: str, +) -> WrapperBuildResult: + output_dir = Path(kwargs["output_dir"]) + output_dir.mkdir(parents=True, exist_ok=True) + module_name = str(kwargs["output_name"]) + shared_library = output_dir / f"{module_name}.so" + shared_library.write_bytes(b"mock extension") + + extension = ModuleType(module_name) + namespace = ModuleType(f"{module_name}.{public_name}") + setattr(extension, public_name, namespace) + modules[module_name] = extension + return WrapperBuildResult( + sources=(source,), + module_name=module_name, + output_dir=output_dir, + shared_library=shared_library, + build_makefile=None, + compiled=True, + generated_sources=(), + generated_files=(), + ) + + +def test_fortran_magic_routes_options_publishes_declared_namespace_and_reuses_exact_cell( + tmp_path: Path, + monkeypatch, + capsys, +): + modules: dict[str, ModuleType] = {} + calls: list[tuple[Path, dict[str, object]]] = [] + + def build(source: Path, **kwargs) -> WrapperBuildResult: + calls.append((source, kwargs)) + return _mock_build_result(source, kwargs, modules, public_name="maths") + + monkeypatch.setattr(magic_module, "build_fortran_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + cell = "module maths\ncontains\nend module\n" + line = ( + "--compiler ifx --compiler-arg=-fpp " + '--native-compile-flags="-O3 -xHost" ' + "--wrapper-fortran-flags=-O2 --wrapper-c-flags=-O1" + ) + + magic.fortran(line, cell) + + first_namespace = shell.user_ns["maths"] + source, kwargs = calls[0] + digest = hashlib.sha256(f"fortran{cell}".encode()).hexdigest() + assert source == tmp_path / "cache" / digest / "cell.f90" + assert source.read_text(encoding="utf-8") == cell + assert kwargs["preprocessing"].compiler == "ifx" + assert kwargs["preprocessing"].compiler_args == ["-fpp"] + assert kwargs["native_fortran_flags"] == ("-O3", "-xHost") + assert kwargs["wrapper_fortran_flags"] == ("-O2",) + assert kwargs["wrapper_c_flags"] == ("-O1",) + assert kwargs["verbose"] is False + assert len(shell.user_ns) == 1 + + magic.fortran(f"{line} --verbose", cell) + + assert len(calls) == 1 + assert shell.user_ns["maths"] is first_namespace + assert f">> Reuse cached PRIK cell: {digest}" in capsys.readouterr().out + + magic.fortran(f"{line} --force", cell) + + assert len(calls) == 2 + assert shell.user_ns["maths"] is not first_namespace + assert calls[1][1]["output_name"] != calls[0][1]["output_name"] + + +def test_same_source_digest_rebuilds_when_compiler_configuration_changes(tmp_path: Path, monkeypatch): + modules: dict[str, ModuleType] = {} + calls: list[dict[str, object]] = [] + + def build(source: Path, **kwargs) -> WrapperBuildResult: + calls.append(kwargs) + return _mock_build_result(source, kwargs, modules, public_name="maths") + + monkeypatch.setattr(magic_module, "build_fortran_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") + cell = "module maths\nend module\n" + + magic.fortran("--compiler gfortran", cell) + magic.fortran("--compiler ifx", cell) + + digest = hashlib.sha256(f"fortran{cell}".encode()).hexdigest() + assert len(calls) == 2 + assert [path.name for path in (tmp_path / "cache").iterdir() if path.is_dir()] == [digest] + + +def test_generate_pyi_persists_source_and_inserts_module_and_standalone_contract_cells( + tmp_path: Path, + monkeypatch, +): + source = "module maths\nend module\n" + digest = hashlib.sha256(f"fortran{source}".encode()).hexdigest() + + def generate(path: Path, *, source_digest: str, options) -> contract_cells.GeneratedContracts: + assert path.read_text(encoding="utf-8") == source + assert source_digest == digest + assert options.compiler == "ifx" + return contract_cells.GeneratedContracts( + language="fortran", + source_digest=source_digest, + module_contracts={ + "maths.pyi": "def square() -> None: ...", + "stats.pyi": "def mean() -> None: ...", + }, + direct_contract=("from prik.contracts import standalone\n\n@standalone\ndef reset() -> None: ..."), + dependency_contracts={}, + ) + + monkeypatch.setattr(contract_cells, "generate_contracts_from_source", generate) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + + magic.fortran("--pyi --compiler ifx", source) + + assert (tmp_path / "cache" / digest / "cell.f90").read_text(encoding="utf-8") == source + assert len(shell.next_inputs) == 3 + maths, stats, standalone = (text for text, replace in shell.next_inputs if not replace) + assert maths.startswith("%%pyi --compiler ifx\n") + assert stats.startswith("%%pyi --compiler ifx\n") + assert f"# prik: file=maths.pyi source-sha256={digest}" in maths + assert f"# prik: file=stats.pyi source-sha256={digest}" in stats + assert f"# prik: source-sha256={digest}" in standalone + assert "file=__init__.pyi" not in standalone + assert "@standalone" in standalone + + +def test_terminal_ipython_presents_multiple_generated_contracts_sequentially( + tmp_path: Path, + monkeypatch, +): + source = "module maths\nend module\nmodule maths2\nend module\n" + modules: dict[str, ModuleType] = {} + public_names = iter(("maths", "maths2")) + + def generate(path: Path, *, source_digest: str, options) -> contract_cells.GeneratedContracts: + return contract_cells.GeneratedContracts( + language="fortran", + source_digest=source_digest, + module_contracts={ + "maths.pyi": "def square() -> None: ...", + "maths2.pyi": "def square() -> None: ...", + }, + direct_contract=None, + dependency_contracts={}, + ) + + def build(contract: Path, **kwargs) -> WrapperBuildResult: + return _mock_build_result(contract, kwargs, modules, public_name=next(public_names)) + + monkeypatch.setattr(contract_cells, "generate_contracts_from_source", generate) + monkeypatch.setattr(magic_module, "build_pyi_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + shell = _TerminalShell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + + magic.fortran("--pyi", source) + + maths_cell = shell.take_next_input() + assert "file=maths.pyi" in maths_cell + assert "file=maths2.pyi" not in maths_cell + magic_line, editable = maths_cell.split("\n", 1) + magic.pyi(magic_line.removeprefix("%%pyi").strip(), editable) + + maths2_cell = shell.take_next_input() + assert "file=maths2.pyi" in maths2_cell + magic_line, editable = maths2_cell.split("\n", 1) + magic.pyi(magic_line.removeprefix("%%pyi").strip(), editable) + + assert shell.rl_next_input is None + assert set(shell.user_ns) == {"maths", "maths2"} + + +def test_generated_module_contract_builds_against_cached_source_and_reuses_exact_edit( + tmp_path: Path, + monkeypatch, +): + modules: dict[str, ModuleType] = {} + calls: list[tuple[Path, dict[str, object]]] = [] + + def generate(path: Path, *, source_digest: str, options) -> contract_cells.GeneratedContracts: + return contract_cells.GeneratedContracts( + language="fortran", + source_digest=source_digest, + module_contracts={"maths.pyi": "def square() -> None: ..."}, + direct_contract=None, + dependency_contracts={}, + ) + + def build(contract: Path, **kwargs) -> WrapperBuildResult: + calls.append((contract, kwargs)) + return _mock_build_result(contract, kwargs, modules, public_name="maths") + + monkeypatch.setattr(contract_cells, "generate_contracts_from_source", generate) + monkeypatch.setattr(magic_module, "build_pyi_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + source = "module maths\nend module\n" + + magic.fortran( + '--pyi --compiler ifx --native-compile-flags="-O3 -xHost"', + source, + ) + inserted = shell.next_inputs[0][0] + magic_line, editable_cell = inserted.split("\n", 1) + line = magic_line.removeprefix("%%pyi").strip() + editable_cell = "# user note\n" + editable_cell.replace("def square()", "def square(value: int)") + + magic.pyi(line, editable_cell) + first_namespace = shell.user_ns["maths"] + contract, kwargs = calls[0] + assert contract.name == "__init__.pyi" + assert contract.read_text(encoding="utf-8") == "from . import maths\n" + assert "def square(value: int)" in (contract.parent / "maths.pyi").read_text(encoding="utf-8") + assert "# user note" in (contract.parent / "maths.pyi").read_text(encoding="utf-8") + source_path = tmp_path / "cache" / hashlib.sha256(f"fortran{source}".encode()).hexdigest() / "cell.f90" + assert kwargs["native_language"] == "fortran" + assert kwargs["input_compiler"] == "ifx" + assert kwargs["native_fortran_sources"] == (source_path,) + assert kwargs["native_fortran_flags"] == ("-O3", "-xHost") + + magic.pyi(line, editable_cell) + + assert len(calls) == 1 + assert shell.user_ns["maths"] is first_namespace + + magic.pyi(f"{line} --force", editable_cell) + + assert len(calls) == 2 + assert shell.user_ns["maths"] is not first_namespace + + +@pytest.mark.parametrize( + "changed_line", + ( + "--compiler gfortran --native-compile-flags=-O3", + "--compiler ifx --native-compile-flags=-O2", + ), + ids=("compiler", "flags"), +) +def test_generated_contract_rejects_changed_build_configuration( + tmp_path: Path, + monkeypatch, + changed_line: str, +): + def generate(path: Path, *, source_digest: str, options) -> contract_cells.GeneratedContracts: + return contract_cells.GeneratedContracts( + language="fortran", + source_digest=source_digest, + module_contracts={"maths.pyi": "def square() -> None: ..."}, + direct_contract=None, + dependency_contracts={}, + ) + + def unexpected_build(*args, **kwargs): + pytest.fail("changed generated-contract options must fail before compilation") + + monkeypatch.setattr(contract_cells, "generate_contracts_from_source", generate) + monkeypatch.setattr(magic_module, "build_pyi_extension", unexpected_build) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + source = "module maths\nend module\n" + magic.fortran("--pyi --compiler ifx --native-compile-flags=-O3", source) + _magic_line, editable_cell = shell.next_inputs[0][0].split("\n", 1) + + with pytest.raises(UsageError, match="compiler and build options"): + magic.pyi(changed_line, editable_cell) + + +def test_handwritten_module_contract_builds_existing_sources_and_tracks_their_contents( + tmp_path: Path, + monkeypatch, +): + modules: dict[str, ModuleType] = {} + calls: list[tuple[Path, dict[str, object]]] = [] + + def build(contract: Path, **kwargs) -> WrapperBuildResult: + calls.append((contract, kwargs)) + return _mock_build_result(contract, kwargs, modules, public_name="maths") + + monkeypatch.setattr(magic_module, "build_pyi_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + source = tmp_path / "geometry.f90" + helper = tmp_path / "helper.f90" + source.write_text("module maths\nend module\n", encoding="utf-8") + helper.write_text("subroutine helper()\nend subroutine\n", encoding="utf-8") + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + line = f'--native-fortran-sources {source} {helper} --compiler ifx --native-compile-flags="-O3 -xHost"' + cell = "# prik: file=maths.pyi\n\ndef square() -> None: ...\n" + + magic.pyi(line, cell) + + first_namespace = shell.user_ns["maths"] + contract, kwargs = calls[0] + assert contract.read_text(encoding="utf-8") == "from . import maths\n" + module_contract = (contract.parent / "maths.pyi").read_text(encoding="utf-8") + assert module_contract.endswith("def square() -> None: ...\n") + assert "# prik:" not in module_contract + assert kwargs["native_language"] == "fortran" + assert kwargs["input_compiler"] == "ifx" + assert kwargs["native_fortran_sources"] == (source, helper) + assert kwargs["native_fortran_flags"] == ("-O3", "-xHost") + + magic.pyi(line, cell) + + assert len(calls) == 1 + assert shell.user_ns["maths"] is first_namespace + + helper.write_text("subroutine helper_changed()\nend subroutine\n", encoding="utf-8") + magic.pyi(line, cell) + + assert len(calls) == 2 + assert shell.user_ns["maths"] is not first_namespace + + +def test_generated_standalone_contract_publishes_direct_declarations(tmp_path: Path, monkeypatch): + modules: dict[str, ModuleType] = {} + calls: list[Path] = [] + + def generate(path: Path, *, source_digest: str, options) -> contract_cells.GeneratedContracts: + return contract_cells.GeneratedContracts( + language="fortran", + source_digest=source_digest, + module_contracts={}, + direct_contract="@standalone\ndef square() -> None: ...", + dependency_contracts={}, + ) + + def build(contract: Path, **kwargs) -> WrapperBuildResult: + calls.append(contract) + output_dir = Path(kwargs["output_dir"]) + output_dir.mkdir(parents=True, exist_ok=True) + module_name = str(kwargs["output_name"]) + shared_library = output_dir / f"{module_name}.so" + shared_library.write_bytes(b"mock extension") + extension = ModuleType(module_name) + extension.square = lambda: 4 + modules[module_name] = extension + return WrapperBuildResult( + sources=(contract,), + module_name=module_name, + output_dir=output_dir, + shared_library=shared_library, + build_makefile=None, + compiled=True, + generated_sources=(), + generated_files=(), + ) + + monkeypatch.setattr(contract_cells, "generate_contracts_from_source", generate) + monkeypatch.setattr(magic_module, "build_pyi_extension", build) + monkeypatch.setattr(WrapperBuildResult, "import_module", lambda self: modules[self.module_name]) + shell = _Shell() + magic = PrikMagics(shell, cache_dir=tmp_path / "cache") + + magic.fortran("--pyi", "subroutine square()\nend subroutine\n") + inserted = shell.next_inputs[0][0] + magic_line, editable_cell = inserted.split("\n", 1) + assert magic_line == "%%pyi" + assert " file=" not in editable_cell + + magic.pyi(magic_line.removeprefix("%%pyi").strip(), editable_cell) + + assert calls[0].read_text(encoding="utf-8").endswith("@standalone\ndef square() -> None: ...\n") + assert shell.user_ns["square"]() == 4 + assert "cell" not in shell.user_ns + + +def test_multiple_generated_contracts_use_distinct_jupyter_payloads(): + writes: list[tuple[dict[str, object], bool]] = [] + + class _PayloadManager: + def write_payload(self, payload: dict[str, object], *, single: bool = True) -> None: + writes.append((payload, single)) + + class _PayloadShell: + payload_manager = _PayloadManager() + + def set_next_input(self, text: str, *, replace: bool = False) -> None: + self.payload_manager.write_payload( + {"source": "set_next_input", "text": text, "replace": replace}, + ) + + contract_cells.insert_editable_cells(_PayloadShell(), ["first", "second"]) + + assert writes == [ + ({"source": "set_next_input", "text": "first", "replace": False}, True), + ({"source": "set_next_input", "text": "second", "replace": False}, False), + ] + + +def test_editable_contract_requires_its_exact_cached_source(tmp_path: Path): + magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") + digest = "a" * 64 + cell = f"# prik: file=maths.pyi source-sha256={digest}\n\ndef square(): ...\n" + + with pytest.raises(UsageError, match="execute its %%fortran --pyi or %%c --pyi source cell again"): + magic.pyi("", cell) + + +def test_magic_reports_usage_without_terminating_ipython(tmp_path: Path, capsys): + magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") + + magic.fortran("--help", "") + assert "usage: %%fortran" in capsys.readouterr().out + + with pytest.raises(UsageError, match="non-empty"): + magic.fortran("", "\n") + with pytest.raises(UsageError, match="only generates editable cells"): + magic.fortran("--pyi --force", "source") + with pytest.raises(UsageError, match="generated source metadata or explicit"): + magic.pyi("", "def square(): ...\n") + with pytest.raises(UsageError, match="cannot mix"): + magic.pyi( + "--native-fortran-sources one.f90 --native-c-sources one.c", + "def square(): ...\n", + ) + source = tmp_path / "native.f90" + source.write_text("subroutine native()\nend subroutine\n", encoding="utf-8") + with pytest.raises(UsageError, match="cannot combine generated source-sha256 metadata"): + magic.pyi( + f"--native-fortran-sources {source}", + f"# prik: source-sha256={'a' * 64}\n\ndef native(): ...\n", + ) + with pytest.raises(UsageError, match="full lowercase source-sha256"): + magic.pyi( + "", + "# prik: file=maths.pyi source-sha256=short\ndef square(): ...\n", + ) + + +def test_editable_contract_metadata_errors_name_what_the_cell_got_wrong(tmp_path: Path): + """An edited contract cell must say which metadata a user broke. + + These are the guards an ordinary edit reaches: duplicating the reserved + line, or renaming the contract to something that is not a module path. + """ + magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") + digest = "a" * 64 + + with pytest.raises(UsageError, match="exactly one PRIK metadata line"): + magic.pyi("", f"# prik: source-sha256={digest}\n# prik: file=maths.pyi\n\ndef square(): ...\n") + for filename in ("../escape.pyi", "maths.txt", "not-an-identifier.pyi", "__init__.pyi"): + with pytest.raises(UsageError, match=r"Invalid editable \.pyi filename"): + magic.pyi("", f"# prik: file={filename} source-sha256={digest}\n\ndef square(): ...\n") + + +def test_dash_prefixed_flag_value_usage_names_the_equals_form(tmp_path: Path): + """A flag value argparse read as an option must say how to write it.""" + magic = PrikMagics(_Shell(), cache_dir=tmp_path / "cache") + + with pytest.raises(UsageError, match=r'--native-compile-flags="-O3 -march=native"'): + magic.fortran("--native-compile-flags -O3", "source") + # ``--compiler-arg`` carries exactly one argument, so it must not be told + # to pass a quoted group of several flags. + with pytest.raises(UsageError, match=r"--compiler-arg=-fopenmp"): + magic.fortran("--compiler-arg -fopenmp", "source") + assert "quoted group" not in _compiler_arg_usage_message(magic) + # An option whose value is never dash-prefixed keeps the plain message. + with pytest.raises(UsageError, match=r"^argument --compiler: expected one argument$"): + magic.fortran("--compiler", "source") + + +def _compiler_arg_usage_message(magic: PrikMagics) -> str: + with pytest.raises(UsageError) as raised: + magic.fortran("--compiler-arg -fopenmp", "source") + return str(raised.value) + + +def test_ipython_extension_hook_registers_the_magic_class(): + registered = [] + + class _RegistrationShell: + def register_magics(self, magic_class) -> None: + registered.append(magic_class) + + load_ipython_extension(_RegistrationShell()) + + assert registered == [PrikMagics] + + +def test_ipython_extension_refuses_to_replace_an_existing_cell_magic(): + class _ConflictingShell: + def find_cell_magic(self, name: str): + return (lambda: None) if name == "c" else None + + def register_magics(self, magic_class) -> None: + raise AssertionError("conflicting magics must be reported before registration") + + with pytest.raises(UsageError, match=r"already registered: %%c"): + load_ipython_extension(_ConflictingShell()) diff --git a/tests/fortran/infrastructure/printers/test_source_printers.py b/tests/fortran/infrastructure/printers/test_source_printers.py index ad9c12921..22e2bedd9 100644 --- a/tests/fortran/infrastructure/printers/test_source_printers.py +++ b/tests/fortran/infrastructure/printers/test_source_printers.py @@ -126,7 +126,7 @@ def test_c_source_printer_renders_function_local_cleanup_jumps(): def test_source_printers_reject_wrapper_plan_models(): plan = ModulePlan( owner_path="demo", - binding=BindingModulePlan("demo"), + binding=BindingModulePlan("demo", "demo"), entrypoint=NativeEntrypointModulePlan("demo"), bridge=BridgeModulePlan("demo"), namespaces=(NamespacePlan(owner_path="demo", python_path=()),), diff --git a/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py b/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py index f8c1e193b..e6ba0bc49 100644 --- a/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py +++ b/tests/fortran/infrastructure/semantic_pyi/semantics/test_calls_and_projections.py @@ -483,6 +483,26 @@ def scale(values: Float64[::], label: String[8]) -> None: ... ) +def test_total_size_projection_round_trips_with_default_and_typed_integer_storage(): + module = parse_pyi_text( + """from prik.contracts import Arg, Float64, Int32, native_call + +@native_call([Arg(0).size, Int32(Arg(0).size), Arg(0)]) +def scale(values: Float64[:]) -> None: ... +""", + module_name="total_size_projection", + ) + + projection = module.functions[0].projection + + assert [(item.value_kind, item.value, item.value_cast) for item in projection] == [ + ("size", {"kind": "arg", "position": 0}, None), + ("size", {"kind": "arg", "position": 0}, "Int32"), + ("", None, None), + ] + assert "@native_call([Arg(0).size, Int32(Arg(0).size), Arg(0)])" in emit_module(module) + + def test_typed_literal_keeps_its_constant_form_beside_typed_projections(): module = parse_pyi_text( """from prik.contracts import Arg, Float64, Int32, native_call @@ -522,7 +542,7 @@ def scale(value: Float64) -> None: ... def test_typed_scalar_constructor_rejects_a_visible_argument_reference(): with pytest.raises( ValueError, - match="Int32 accepts a literal value or a shape, stride, or length projection", + match="Int32 accepts a literal value or a size, shape, stride, or length projection", ): parse_pyi_text( """from prik.contracts import Arg, Int32, native_call