Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ Result: breaking

```bash
pip install tool-semantics
# optional marker extras (no provider SDKs): tool-semantics[openai] / [providers]
# or from source: pip install -e .
```

Expand Down
70 changes: 70 additions & 0 deletions docs/probes.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,76 @@ probe = Probe(
| `TOOL_SEMANTICS_MODEL` / `OPENAI_MODEL` | Model id (default `gpt-4o-mini`) |
| `TOOL_SEMANTICS_BASE_URL` / `OPENAI_BASE_URL` | OpenAI-compatible base URL |

## Packaging extras (SDK-free)

Core `pip install tool-semantics` never requires a provider SDK.
`OpenAICompatibleRunner` speaks HTTP with the standard library only.

Optional extras are **markers** (empty dependency sets) so teams can declare
intent without pulling SDKs:

```bash
pip install "tool-semantics[openai]"
# or
pip install "tool-semantics[providers]"
```

## HTTP provider profile matrix

| Profile | Base URL pattern | Auth | Notes |
| --- | --- | --- | --- |
| `openai` | `https://api.openai.com/v1` | Bearer | Official Chat Completions |
| `azure-openai` | `https://{resource}.openai.azure.com/openai/deployments/{deployment}` | `api-key` header + `api-version` query | Pass `resource=` / `deployment=` |
| `local` | `http://127.0.0.1:11434/v1` | Bearer (often ignored) | Ollama, vLLM, LM Studio, etc. |

```python
from tool_semantics.runner import (
RunnerConfig,
openai_compatible_from_profile,
)

runner = openai_compatible_from_profile(
"openai",
model="gpt-4o-mini",
api_key="…",
)
azure = openai_compatible_from_profile(
"azure-openai",
model="gpt-4o-mini", # deployment name
api_key="…",
resource="my-resource",
deployment="gpt-4o-mini",
)
local = openai_compatible_from_profile(
"local",
model="llama3.2",
api_key="ollama", # placeholder for local servers that expect a key
base_url="http://127.0.0.1:11434/v1",
)

cfg = RunnerConfig(
max_retries=2,
retry_backoff_seconds=0.05,
max_calls=20,
max_tokens=50_000,
max_cost_usd=0.50,
prompt_cost_per_1m=0.15,
completion_cost_per_1m=0.60,
max_output_tokens=1024,
seed=1,
)
```

Limits (`max_calls`, `max_tokens`, `max_cost_usd`) are enforced on the runner
and recorded on `ModelCompletion.metadata.run_config` together with cumulative
`total_tokens` / `total_cost_usd`.

### Retry / backoff

`OpenAICompatibleRunner.complete` attempts `max_retries + 1` times. Between
attempts it sleeps `retry_backoff_seconds` (fixed backoff; default `0.05`).
Unit tests monkeypatch `time.sleep` and never open a live network socket.

## Safety

- Do not embed secrets in probe intents or expected arguments.
Expand Down
4 changes: 4 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ Issues = "https://github.com/askmy-stack/tool-semantics/issues"
Changelog = "https://github.com/askmy-stack/tool-semantics/blob/main/CHANGELOG.md"

[project.optional-dependencies]
# Marker extras — OpenAI / Azure / local profiles use stdlib HTTP only.
# Installing [openai] documents intent; it does not pull a provider SDK.
openai = []
providers = []
dev = [
"pytest>=8,<10",
"pytest-cov>=5,<8",
Expand Down
Loading