diff --git a/skills/sdk-install/csharp.md b/skills/sdk-install/csharp.md index 203f5f9c..65e2e90f 100644 --- a/skills/sdk-install/csharp.md +++ b/skills/sdk-install/csharp.md @@ -1,155 +1,10 @@ # C# SDK Install -Reference guide for installing the Braintrust C# SDK. +**Read https://www.braintrust.dev/docs/sdks/csharp/install-and-instrument and follow it.** -- SDK repo: https://github.com/braintrustdata/braintrust-sdk-dotnet -- NuGet: https://www.nuget.org/packages/Braintrust.Sdk -- Requires .NET 8.0+ - -## Find the latest version of the SDK - -Look up the latest version from NuGet **without installing anything**. Do not guess -- use a read-only query so the environment stays unchanged until you write the version into the project file. - -```bash -dotnet package search Braintrust.Sdk --exact-match -``` - -Then install that version (the latest published release): - -### .NET CLI - -```bash -dotnet add package Braintrust.Sdk --version -``` - -### Or add to .csproj - -```xml - - - -``` - -## Initialize the SDK - -```csharp -using Braintrust.Sdk; -using Braintrust.Sdk.Config; - -var apiKey = Environment.GetEnvironmentVariable("BRAINTRUST_API_KEY"); -Braintrust? braintrust = null; -System.Diagnostics.ActivitySource? activitySource = null; - -if (!string.IsNullOrEmpty(apiKey)) -{ - // Set the project name in code (do NOT require an env var for project name). - var config = BraintrustConfig.Of( - ("BRAINTRUST_API_KEY", apiKey), - ("BRAINTRUST_DEFAULT_PROJECT_NAME", "my-project") - ); - - braintrust = Braintrust.Get(config); - activitySource = braintrust.GetActivitySource(); -} -``` - -`Braintrust.Get(config)` is the main entry point. The SDK requires an API key to be present, so initialize Braintrust conditionally and run the application normally when `BRAINTRUST_API_KEY` is missing. `GetActivitySource()` returns the `System.Diagnostics.ActivitySource` used to create spans. - -## Install instrumentation - -The C# SDK instruments LLM clients by wrapping them. Only instrument clients that are actually present in the project. - -### OpenAI (`OpenAI` NuGet package) - -```bash -dotnet add package OpenAI -``` - -Create an instrumented OpenAI client: +That page is the source of truth for installing and instrumenting the Braintrust C# SDK -- package installation, initialization, and the supported LLM client wrappers. Do not install or instrument from remembered snippets; they go stale. -```csharp -using Braintrust.Sdk; -using Braintrust.Sdk.Config; -using Braintrust.Sdk.Instrumentation.OpenAI; +Supporting references, if you need them: -var btApiKey = Environment.GetEnvironmentVariable("BRAINTRUST_API_KEY"); -var openAIApiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY"); - -if (!string.IsNullOrEmpty(btApiKey) && !string.IsNullOrEmpty(openAIApiKey)) -{ - var config = BraintrustConfig.Of( - ("BRAINTRUST_API_KEY", btApiKey), - ("BRAINTRUST_DEFAULT_PROJECT_NAME", "my-project") - ); - var braintrust = Braintrust.Get(config); - var activitySource = braintrust.GetActivitySource(); - var client = BraintrustOpenAI.WrapOpenAI(activitySource, openAIApiKey); - - // Optional: create a root activity so you can generate a permalink. - using var activity = activitySource.StartActivity("braintrust-openai-example"); - - var chatClient = client.GetChatClient("gpt-5-mini"); - var response = await chatClient.CompleteChatAsync( - new ChatMessage[] - { - new SystemChatMessage("You are a helpful assistant."), - new UserChatMessage("What is the capital of France?") - } - ); - - if (activity != null) - { - var projectUri = await braintrust.GetProjectUriAsync(); - var url = $"{projectUri}/logs?r={activity.TraceId}&s={activity.SpanId}"; - Console.WriteLine($"View your data in Braintrust: {url}"); - } -} -``` - -### Custom spans - -For business logic that isn't an LLM call, create spans manually with the `ActivitySource`: - -```csharp -using (var activity = activitySource.StartActivity("my-operation")) -{ - activity?.SetTag("some.attribute", "value"); - // LLM calls inside here are automatically nested under this span -} -``` - -## Run the application - -Try to figure out how to run the application from the project structure: - -- **dotnet run**: `dotnet run` or `dotnet run --project path/to/Project.csproj` -- **ASP.NET**: `dotnet run` (typically starts Kestrel) -- **Published app**: `dotnet path/to/app.dll` -- **Visual Studio / Rider**: run from IDE - -If you can't determine how to run the app, ask the user. - -## Generate a permalink (required) - -The installer must produce a permalink to the emitted trace/logs in its final output. - -In .NET, the most reliable permalink can be generated from the root Activity's TraceId/SpanId: - -```csharp -if (braintrust != null && activitySource != null) -{ - using var activity = activitySource.StartActivity("braintrust-install-verify"); - if (activity != null) - { - // Perform a real operation that triggers LLM spans / instrumentation here. - - var projectUri = await braintrust.GetProjectUriAsync(); - var url = $"{projectUri}/logs?r={activity.TraceId}&s={activity.SpanId}"; - Console.WriteLine($"View your data in Braintrust: {url}"); - } -} -``` - -The final assistant response must include the printed URL. - -If the SDK-generated URL is not available, construct the permalink manually using the URL format documented in `braintrust-url-formats.md` as described in the agent task (Step 5). +- SDK repo: https://github.com/braintrustdata/braintrust-sdk-dotnet +- NuGet: https://www.nuget.org/packages/Braintrust.Sdk diff --git a/skills/sdk-install/go.md b/skills/sdk-install/go.md index 19b0684b..f0d09ead 100644 --- a/skills/sdk-install/go.md +++ b/skills/sdk-install/go.md @@ -1,151 +1,10 @@ # Go SDK Install -Reference guide for installing the Braintrust Go SDK. +**Read https://www.braintrust.dev/docs/sdks/go/install-and-instrument and follow it.** -- SDK repo: https://github.com/braintrustdata/braintrust-sdk-go -- pkg.go.dev: https://pkg.go.dev/github.com/braintrustdata/braintrust-sdk-go -- Requires Go 1.22+ - -## Install the SDK - -Install the latest Braintrust SDK. Do not hard-pin the SDK version unless the user asks -- `go get` without a version suffix is fine and will record whatever version `go mod tidy` resolves. - -```bash -go get github.com/braintrustdata/braintrust-sdk-go -``` - -If you need to know what the latest version is: - -```bash -go list -m -versions github.com/braintrustdata/braintrust-sdk-go -``` - -**Note:** Orchestrion, the build-time instrumentation tool described below, **must** be pinned to an exact version. That requirement is separate from the SDK itself. - -## Instrument the application - -**You must read https://www.braintrust.dev/docs/instrument/trace-llm-calls before instrumenting anything.** That page is the source of truth for supported providers and setup, and may have changed since this guide was written. - -### Prefer automatic instrumentation (Orchestrion) - -**Automatic instrumentation via [Orchestrion](https://github.com/DataDog/orchestrion) is the recommended path and should be used whenever possible.** It injects tracing at compile time with no wrapper code in the application, so LLM client calls are traced automatically across your codebase and third-party code. - -Manual span/wrapper code should only be used as a **last resort** -- e.g. for bespoke business-logic spans, or when a provider isn't yet supported by the Orchestrion contrib packages. Don't reach for manual tracing before confirming Orchestrion can do the job. - -### Quick start - -Every Go project needs OpenTelemetry setup and a Braintrust client. - -```go -package main - -import ( - "log" - - "github.com/braintrustdata/braintrust-sdk-go" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/sdk/trace" -) - -func main() { - tp := trace.NewTracerProvider() - otel.SetTracerProvider(tp) - - _, err := braintrust.New(tp, braintrust.WithProject("my-project")) - if err != nil { - log.Fatal(err) - } -} -``` - -`braintrust.New` reads `BRAINTRUST_API_KEY` from the environment automatically. - -### Requirement: persist Orchestrion into the normal build/run path - -Auto-instrumentation requires the project to be built and run with Orchestrion. A one-off `orchestrion go build` during verification is **not enough** if the next developer, CI job, or deploy will go back to plain `go build` / `go run`. - -**1. Resolve and pin an exact Orchestrion version:** - -Orchestrion is a build-time dependency that modifies the Go toolchain, so it **must** be pinned to an exact version for reproducible builds -- this is different from the Braintrust SDK itself. - -```bash -go list -m -versions github.com/DataDog/orchestrion -go install github.com/DataDog/orchestrion@vX.Y.Z -``` - -Do not use `@latest`. Prefer the newest version that is compatible with the project's existing `go` / `toolchain` version. If Orchestrion would require bumping the project's Go version or toolchain, ask the user before making that change. - -**2. Create `orchestrion.tool.go` in the module root (the same directory as `go.mod`):** +That page is the source of truth for installing and instrumenting the Braintrust Go SDK -- supported providers, their `trace/contrib/` import paths, and build-time instrumentation setup. Do not install or instrument from remembered snippets; they go stale. -Prefer importing only the integrations the project actually uses. Use `trace/contrib/all` only if provider detection is genuinely unclear or the project uses many supported integrations. +Supporting references, if you need them: -To instrument all supported providers: - -```go -//go:build tools - -package main - -import ( - _ "github.com/DataDog/orchestrion" - _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/all" -) -``` - -Or import only the integrations the project actually uses: - -```go -//go:build tools - -package main - -import ( - _ "github.com/DataDog/orchestrion" - _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/anthropic" // anthropic-sdk-go - _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/genai" // Google GenAI - _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/github.com/sashabaranov/go-openai" // sashabaranov/go-openai - _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/langchaingo" // LangChainGo - _ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai" // openai-go -) -``` - -Then run `go mod tidy` so the exact Orchestrion and contrib versions are recorded in `go.mod` / `go.sum`. - -**3. Persist Orchestrion into the project's actual workflow:** - -Update the command the project already expects developers or CI to use: - -- `Makefile` / `justfile` / shell scripts: change `go build`, `go run`, and `go test` invocations to `orchestrion go ...` where appropriate. -- `Dockerfile`: change build steps to use Orchestrion. -- Bootstrap / CI / devcontainer setup: if the repo already has a checked-in way to install required tooling, add Orchestrion there too so future users do not hit `orchestrion: command not found`. -- Repo-local env/config files: if the project already uses a checked-in mechanism for env vars, set `GOFLAGS="-toolexec=orchestrion toolexec"` there. - -A shell-local `export GOFLAGS=...` in the current terminal does **not** satisfy this requirement by itself, because it will not help the next user or CI run. - -If you add `orchestrion.tool.go` but do **not** modify any persisted build/run path, treat the installation as incomplete. - -**4. Verify using the same persisted command:** - -After wiring Orchestrion into the normal workflow, run that exact command and confirm traces are emitted. Do not verify with a custom one-off command that the project will not use later. - -After this, LLM client calls are automatically traced with no application wrapper code. - -### Supported providers - -For the current list of supported providers and their `trace/contrib/` import paths, see https://www.braintrust.dev/docs/instrument/trace-llm-calls. - -## Run the application - -Prefer the project's existing build/run entrypoint, and make sure that entrypoint now goes through Orchestrion. - -Try to figure out how the project is normally run from the project structure: - -- **Makefile / justfile / scripts**: prefer `make run`, `just run`, or the existing repo script if present -- **go run**: if the project is normally run directly, update that path to `orchestrion go run .` or `orchestrion go run ./cmd/myapp` -- **Docker**: check for a `Dockerfile` or container build script - -If you can't determine how the app is supposed to be built or run in normal use, ask the user before proceeding. - -## Generate a permalink (required) - -Follow the permalink generation steps in the agent task (Step 5). Use the project name you configured in code above. +- SDK repo: https://github.com/braintrustdata/braintrust-sdk-go +- pkg.go.dev: https://pkg.go.dev/github.com/braintrustdata/braintrust-sdk-go diff --git a/skills/sdk-install/instrument-task.md b/skills/sdk-install/instrument-task.md index 60655891..37794400 100644 --- a/skills/sdk-install/instrument-task.md +++ b/skills/sdk-install/instrument-task.md @@ -7,7 +7,7 @@ - **Only add Braintrust code.** Do not refactor or modify unrelated code. - **One language, one service per install run.** If the repo has more than one candidate, ask the user which one to instrument before starting. Do not instrument multiple languages or services in the same run. - **If the language is unclear, ask the user.** Do not guess. See Step 2. -- **Install the latest Braintrust SDK.** Do not hard-pin the Braintrust SDK version unless the user asks for it -- use the package manager's normal install (which may produce an exact or a ranged version, whichever is idiomatic for that ecosystem). Build-time dependencies (e.g. Orchestrion for Go) must still be pinned to an exact version -- see the language-specific resource. +- **Install the latest Braintrust SDK.** Do not hard-pin the Braintrust SDK version unless the user asks for it -- use the package manager's normal install (which may produce an exact or a ranged version, whichever is idiomatic for that ecosystem). Build-time dependencies (e.g. Orchestrion for Go) must still be pinned to an exact version -- see the language's install docs. - **Set the project name in code.** Do NOT configure project name via env vars. - **App must run without Braintrust.** If `BRAINTRUST_API_KEY` is missing at runtime, do not crash. - **Abort install if API key is not set.** (Do not modify runtime behavior.) diff --git a/skills/sdk-install/java.md b/skills/sdk-install/java.md index 31694495..e3768b25 100644 --- a/skills/sdk-install/java.md +++ b/skills/sdk-install/java.md @@ -1,169 +1,10 @@ # Java SDK Install -Reference guide for installing the Braintrust Java SDK. +**Read https://www.braintrust.dev/docs/sdks/java/install-and-instrument and follow it.** -- SDK repo: https://github.com/braintrustdata/braintrust-sdk-java -- Maven Central: https://central.sonatype.com/artifact/dev.braintrust/braintrust-sdk-java/versions -- Requires Java 17+ - -## Find the latest version of the SDK - -Look up the latest version from Maven Central **without modifying the project**. Do not guess -- use a read-only query so dependencies stay unchanged until you write the version into the build file. - -```bash -curl -s 'https://search.maven.org/solrsearch/select?q=g:dev.braintrust+AND+a:braintrust-sdk-java&rows=1&wt=json' | python3 -c "import sys,json; print(json.load(sys.stdin)['response']['docs'][0]['latestVersion'])" -``` - -Then add the dependency with that version (the latest published release): - -### Gradle - -```groovy -dependencies { - implementation 'dev.braintrust:braintrust-sdk-java:' -} -``` - -### Maven - -```xml - - dev.braintrust - braintrust-sdk-java - - -``` - -### SBT - -```scala -libraryDependencies += "dev.braintrust" % "braintrust-sdk-java" % "" -``` - -### Generic fallback - -If the project uses a different build tool, the Maven coordinates are: - -- Group: `dev.braintrust` -- Artifact: `braintrust-sdk-java` - -## Initialize the SDK - -```java -import dev.braintrust.Braintrust; -import dev.braintrust.config.BraintrustConfig; -import io.opentelemetry.api.OpenTelemetry; - -var apiKey = System.getenv("BRAINTRUST_API_KEY"); -Braintrust braintrust = null; -OpenTelemetry openTelemetry = null; - -if (apiKey != null && !apiKey.isEmpty()) { - var config = BraintrustConfig.builder() - .apiKey(apiKey) - .defaultProjectName("my-project") - .build(); - braintrust = Braintrust.get(config); - openTelemetry = braintrust.openTelemetryCreate(); -} -``` - -`Braintrust.get()` is the main entry point. The SDK requires an API key to be present, so initialize Braintrust conditionally and run the application normally when `BRAINTRUST_API_KEY` is missing. - -## Install instrumentation - -The Java SDK instruments existing LLM clients by wrapping them. Find which clients the project already uses and wrap them as shown below. Only instrument frameworks that are actually present in the project. - -### OpenAI (`com.openai:openai-java`) - -Wrap the existing `OpenAIClient`: - -```java -import dev.braintrust.instrumentation.openai.BraintrustOpenAI; - -OpenAIClient openAIClient = BraintrustOpenAI.wrapOpenAI(openTelemetry, existingOpenAIClient); -``` +That page is the source of truth for installing and instrumenting the Braintrust Java SDK -- build-tool coordinates, initialization, and the supported LLM client wrappers. Do not install or instrument from remembered snippets; they go stale. -### Anthropic (`com.anthropic:anthropic-java`) +Supporting references, if you need them: -Wrap the existing `AnthropicClient`: - -```java -import dev.braintrust.instrumentation.anthropic.BraintrustAnthropic; - -AnthropicClient anthropicClient = BraintrustAnthropic.wrap(openTelemetry, existingAnthropicClient); -``` - -### Google GenAI / Gemini (`com.google.genai:google-genai`) - -Wrap the existing `Client.Builder`: - -```java -import dev.braintrust.instrumentation.genai.BraintrustGenAI; - -Client geminiClient = BraintrustGenAI.wrap(openTelemetry, existingClientBuilder); -``` - -### LangChain4j (`dev.langchain4j:langchain4j`) - -Wrap an existing `OpenAiChatModel.Builder`: - -```java -import dev.braintrust.instrumentation.langchain.BraintrustLangchain; - -ChatModel model = BraintrustLangchain.wrap(openTelemetry, existingOpenAiChatModelBuilder); -``` - -For LangChain4j AI Services, wrap the `AiServices` builder directly. This instruments LLM calls, tool calls, and concurrent tool execution: - -```java -import dev.braintrust.instrumentation.langchain.BraintrustLangchain; - -Assistant assistant = BraintrustLangchain.wrap( - openTelemetry, - AiServices.builder(Assistant.class) - .chatModel(existingChatModel) - .tools(new MyTools())); -``` - -### Spring AI - -For Spring Boot apps using Spring AI, only register Braintrust beans when `BRAINTRUST_API_KEY` is non-empty, and wrap the underlying LLM client only in that case. Example with Google GenAI: - -```java -@Bean -public Braintrust braintrust() { - return Braintrust.get(BraintrustConfig.fromEnvironment()); -} - -@Bean -public OpenTelemetry openTelemetry(Braintrust braintrust) { - return braintrust.openTelemetryCreate(); -} - -@Bean -public ChatModel chatModel(OpenTelemetry openTelemetry) { - Client genAIClient = BraintrustGenAI.wrap(openTelemetry, new Client.Builder()); - return GoogleGenAiChatModel.builder() - .genAiClient(genAIClient) - .defaultOptions(GoogleGenAiChatOptions.builder() - .model("gemini-2.0-flash-lite") - .build()) - .build(); -} -``` - -## Run the application - -Try to figure out how to run the application from the project structure: - -- **Gradle**: `./gradlew run`, `./gradlew bootRun` (Spring Boot), or a custom run task -- **Maven**: `mvn exec:java`, `mvn spring-boot:run` (Spring Boot) -- **SBT**: `sbt run` -- **Plain jar**: `java -jar ` - -If you can't determine how to run the app, ask the user. - -## Generate a permalink (required) - -Follow the permalink generation steps in the agent task (Step 5). Use the project name you configured in code above. +- SDK repo: https://github.com/braintrustdata/braintrust-sdk-java +- Maven Central: https://central.sonatype.com/artifact/dev.braintrust/braintrust-sdk-java/versions diff --git a/skills/sdk-install/python.md b/skills/sdk-install/python.md index 8bc70473..93116fa8 100644 --- a/skills/sdk-install/python.md +++ b/skills/sdk-install/python.md @@ -1,69 +1,10 @@ # Python SDK Install -Reference guide for installing the Braintrust Python SDK. +**Read https://www.braintrust.dev/docs/sdks/python/install-and-instrument and follow it.** -- SDK repo: https://github.com/braintrustdata/braintrust-sdk-python -- PyPI: https://pypi.org/project/braintrust/ -- Requires Python 3.9+ - -## Install the SDK - -Install the latest published version of `braintrust`. Do not hard-pin the version unless the user asks -- let the package manager record whatever it normally records. - -### pip - -```bash -pip install braintrust -``` - -### poetry - -```bash -poetry add braintrust -``` - -### uv - -```bash -uv add braintrust -``` - -## Instrument the application - -**You must read https://www.braintrust.dev/docs/instrument/trace-llm-calls before instrumenting anything.** That page is the source of truth for supported libraries, extras, and setup, and may have changed since this guide was written. - -### Prefer automatic instrumentation +That page is the source of truth for installing and instrumenting the Braintrust Python SDK -- supported libraries, extras, companion packages, and setup. Do not install or instrument from remembered snippets; they go stale. -**Automatic instrumentation (`auto_instrument()`) is the recommended path and should be used whenever possible.** It patches every supported library that is installed at startup with no call-site changes, so new code and third-party code are traced automatically. See the docs page above for the current list of covered libraries -- do not rely on a hard-coded list here, since coverage changes over time. +Supporting references, if you need them: -Manual `wrap_openai` / `wrap_anthropic` / `wrap_litellm` / etc. call-site wrappers should only be used as a **last resort** -- e.g. when instrumenting a library that `auto_instrument()` doesn't yet cover, or when you need per-client isolation. Don't reach for manual wrappers before confirming auto-instrumentation can't do the job. - -### Quick start - -```python -import braintrust - -braintrust.init_logger(project="my-project") -braintrust.auto_instrument() -``` - -`init_logger` is the main entry point for tracing and reads `BRAINTRUST_API_KEY` from the environment automatically. `auto_instrument()` must be called **before** creating any LLM clients. - -To selectively enable or disable integrations, or to see which libraries require extras (e.g. `braintrust[openai-agents]`, `braintrust[otel]`) or a companion package (e.g. `braintrust-langchain`), follow the docs page -- it lists the current extras, packages, and per-integration setup. - -## Run the application - -Try to figure out how to run the application from the project structure: - -- **Script**: `python main.py`, `python -m mypackage` -- **Poetry**: `poetry run python main.py` -- **uv**: `uv run python main.py` -- **Django**: `python manage.py runserver` -- **FastAPI**: `uvicorn app:app --reload` -- **Flask**: `flask run` - -If you can't determine how to run the app, ask the user. - -## Generate a permalink (required) - -Follow the permalink generation steps in the agent task (Step 5). Use the project name you configured in code above. +- SDK repo: https://github.com/braintrustdata/braintrust-sdk-python +- PyPI: https://pypi.org/project/braintrust/ diff --git a/skills/sdk-install/ruby.md b/skills/sdk-install/ruby.md index 2a335146..88e860ab 100644 --- a/skills/sdk-install/ruby.md +++ b/skills/sdk-install/ruby.md @@ -1,144 +1,10 @@ # Ruby SDK Install -Reference guide for installing the Braintrust Ruby SDK. +**Read https://www.braintrust.dev/docs/sdks/ruby/install-and-instrument and follow it.** -- SDK repo: https://github.com/braintrustdata/braintrust-sdk-ruby -- RubyGems: https://rubygems.org/gems/braintrust -- Requires Ruby 3.1+ - -## Install the SDK - -Install the latest published version of the `braintrust` gem. Do not hard-pin the version unless the user asks -- let Bundler record whatever it normally records. - -The SDK has three setup approaches. Choose the one that fits the project best. - -### Option A: Setup script (recommended for most apps) - -Add to the Gemfile with the `require` option. This auto-instruments all supported libraries at load time -- no additional code needed. - -```ruby -gem "braintrust", require: "braintrust/setup" -``` - -Then run: - -```bash -bundle install -``` - -Configure the project name **in code** by calling `Braintrust.init` during app boot. In Rails, add `config/initializers/braintrust.rb`: - -```ruby -Braintrust.init(default_project: "my-project") -``` - -For non-Rails apps, call `Braintrust.init(default_project: "my-project")` early in the boot sequence (e.g. `config.ru`, `boot.rb`, or the main entrypoint before any LLM clients are created). - -Do **not** set the project name via the `BRAINTRUST_DEFAULT_PROJECT` environment variable -- the project name must live in code. - -**Important**: The application must call `Bundler.require` for the auto-instrumentation to kick in (Rails does this by default). If not, add `require "braintrust/setup"` to an initializer file. - -### Option B: CLI command (no source code changes) - -Install the gem: - -```bash -gem install braintrust -``` - -Or, preferably, add it to the Gemfile so it is checked in: - -```ruby -gem "braintrust" -``` - -Then wrap the application's start command: - -```bash -braintrust exec -- ruby app.rb -braintrust exec -- bundle exec rails server -``` - -To limit which providers are instrumented: - -```bash -braintrust exec --only openai -- ruby app.rb -``` - -**Requirement: persist `braintrust exec` into the normal run path.** A one-off `braintrust exec -- ...` during verification is **not enough** if the next developer, CI job, or deploy will go back to a plain `ruby` / `bundle exec` / `rails server` command. Update whichever launch path the project actually uses: - -- **`Procfile` / `foreman`**: change `web: bundle exec rails server` to `web: braintrust exec -- bundle exec rails server`. -- **`Dockerfile` / container entrypoint**: update the `CMD` / `ENTRYPOINT` or checked-in start script. -- **Process managers / deploy config**: update systemd units, Kubernetes manifests, ECS task definitions, etc. -- **`Makefile` / scripts**: update `make run` / `bin/start` / etc. -- **Bootstrap / CI / devcontainer setup**: if the repo already has a checked-in way to install required tooling, make sure `braintrust` is installed there too so future users and CI do not hit `braintrust: command not found`. - -A shell-local one-off `braintrust exec -- ...` does **not** satisfy this requirement by itself. If you use Option B but do not modify any persisted launch path, treat the installation as incomplete. +That page is the source of truth for installing and instrumenting the Braintrust Ruby SDK -- the available setup approaches, auto-instrumented gems, and per-framework setup. Do not install or instrument from remembered snippets; they go stale. -### Option C: Braintrust.init (explicit control) +Supporting references, if you need them: -Add to the Gemfile: - -```ruby -gem "braintrust" -``` - -Then call `Braintrust.init` in your code: - -```ruby -require "braintrust" - -Braintrust.init(default_project: "my-project") -``` - -Options for `Braintrust.init`: - -| Option | Default | Description | -| ----------------- | ----------------------------------- | --------------------------------------------------------------------------- | -| `default_project` | `ENV['BRAINTRUST_DEFAULT_PROJECT']` | Default project for spans | -| `auto_instrument` | `true` | `true`, `false`, or Hash with `:only`/`:except` keys to filter integrations | -| `api_key` | `ENV['BRAINTRUST_API_KEY']` | API key | - -## Instrument the application - -**You must read https://www.braintrust.dev/docs/instrument/trace-llm-calls before instrumenting anything.** That page is the source of truth for supported providers and setup, and may have changed since this guide was written. - -### Prefer automatic instrumentation - -**Automatic instrumentation is the recommended path and should be used whenever possible.** All three setup approaches above (`braintrust/setup`, `braintrust exec`, `Braintrust.init`) auto-instrument every supported library that is installed -- no wrapping code needed. - -Manual span / wrapper code should only be used as a **last resort**, e.g. for custom business-logic spans or to cover a library that auto-instrumentation doesn't yet support. Don't reach for manual tracing before confirming auto-instrumentation can do the job. - -### Supported providers (auto-instrumented) - -For the current list of auto-instrumented gems and their integration names, see https://www.braintrust.dev/docs/instrument/trace-llm-calls. - -### Selectively enabling integrations - -```ruby -Braintrust.init(auto_instrument: { only: [:openai] }) -``` - -Or via environment variables: - -```bash -export BRAINTRUST_INSTRUMENT_ONLY=openai,anthropic -``` - -## Run the application - -Prefer the project's existing run entrypoint, and -- if you picked Option B -- make sure that entrypoint now goes through `braintrust exec`. - -Try to figure out how the project is normally run from the project structure: - -- **Procfile / foreman**: prefer `foreman start` or whatever the repo already uses, and update the `Procfile` entries (not an ad-hoc shell command) -- **Rails**: `bundle exec rails server` or `bin/rails server` -- if using Option B, update the persisted start script to wrap it in `braintrust exec --` -- **Rack/Sinatra**: `bundle exec rackup` or `ruby app.rb` -- update the persisted launch command, not a one-off invocation -- **Script**: `bundle exec ruby main.rb` or `ruby main.rb` -- **Docker / container**: update the `Dockerfile`'s `CMD` / `ENTRYPOINT` or checked-in start script - -If you can't determine how the app is supposed to be run in normal use, ask the user before proceeding. - -## Generate a permalink (required) - -Follow the permalink generation steps in the agent task (Step 5). Use the project name you configured in code above. +- SDK repo: https://github.com/braintrustdata/braintrust-sdk-ruby +- RubyGems: https://rubygems.org/gems/braintrust diff --git a/skills/sdk-install/typescript.md b/skills/sdk-install/typescript.md index d9bbd6bc..7c0c4117 100644 --- a/skills/sdk-install/typescript.md +++ b/skills/sdk-install/typescript.md @@ -1,150 +1,10 @@ # TypeScript SDK Install -Reference guide for installing the Braintrust TypeScript SDK. +**Read https://www.braintrust.dev/docs/sdks/typescript/install-and-instrument and follow it.** -- SDK repo: https://github.com/braintrustdata/braintrust-sdk-javascript -- npm: https://www.npmjs.com/package/braintrust -- Requires Node.js 18.19.0+ or 20.6.0+ (or Bun 1.0+, Deno with Node compat) - -## Install the SDK - -Install the latest published version of `braintrust`. Do not hard-pin the version unless the user asks -- let the package manager record whatever it normally records (a caret range or an exact version, whichever is idiomatic). - -Match the package manager the repo already uses. Check lockfiles to decide: - -- `pnpm-lock.yaml` → `pnpm` -- `yarn.lock` → `yarn` -- `bun.lock` or `bun.lockb` → `bun` -- `package-lock.json` (or none) → `npm` - -### npm - -```bash -npm install braintrust --no-audit --no-fund -``` - -### yarn - -```bash -yarn add braintrust -``` - -### pnpm - -```bash -pnpm add braintrust -``` - -### bun - -```bash -bun add braintrust -``` - -## Instrument the application - -**You must read https://www.braintrust.dev/docs/instrument/trace-llm-calls before instrumenting anything.** That page is the source of truth and may have changed since this guide was written. - -### Prefer automatic instrumentation - -**Automatic instrumentation is the recommended path and should be used whenever possible.** It patches supported LLM clients/frameworks (OpenAI, Anthropic, Vercel AI SDK, OpenAI Agents SDK, LangChain.js, etc.) at module load time with no call-site changes, so new code and third-party code are traced automatically. - -Automatic instrumentation is enabled one of two ways: - -- **Node.js, no bundler** → preload via `node --import` (see below). Node.js only -- `--import` does not work under Bun, Deno, or Cloudflare Workers. -- **Any runtime with a bundler** (Next.js, webpack, Vite, esbuild, etc.) → use the Braintrust bundler plugin. The bundler plugin is the preferred option whenever a bundler is in play and works regardless of runtime (Node, Bun, Deno, Cloudflare Workers, etc.). - -Manual `wrapOpenAI` / `wrapAnthropic` / `wrapAISDK` / etc. call-site wrappers should only be used when automatic instrumentation isn't available for your setup. The legitimate cases are: - -- Running on **Bun, Deno, or Cloudflare Workers without a bundler** -- there is no automatic path in that configuration, so manual wrappers are the correct choice. -- Instrumenting a client/framework that automatic instrumentation doesn't yet support. - -In every other case (Node.js, or any runtime with a bundler), prefer automatic instrumentation and don't reach for manual wrappers until you've confirmed neither `--import` nor a bundler plugin can be made to work. - -### Quick start - -Create a dedicated setup file (e.g. `instrumentation.ts`) that calls `initLogger`: - -```typescript -import { initLogger } from "braintrust"; +That page is the source of truth for installing and instrumenting the Braintrust TypeScript SDK -- supported clients and frameworks, automatic instrumentation (launch hook and bundler plugins), and per-runtime setup. Do not install or instrument from remembered snippets; they go stale. -initLogger({ - projectName: "my-project", - apiKey: process.env.BRAINTRUST_API_KEY, -}); -``` +Supporting references, if you need them: -`initLogger` is the main entry point for tracing. It reads `BRAINTRUST_API_KEY` from the environment automatically if `apiKey` is not provided. If `initLogger` is not called, instrumentation is a no-op. - -The exact contents of this file (which instrumentations to register, etc.) come from https://www.braintrust.dev/docs/instrument/trace-llm-calls -- follow it. - -### Setting up automatic instrumentation (recommended) - -Automatic instrumentation only works if the setup file is loaded **before** the rest of your application, so it can patch LLM client modules before user code imports them. The patch happens at startup, and no per-call code change is required. Pick whichever matches your setup: - -**Node.js without a bundler (`--import`)** - -`--import` is a Node.js-only flag. Do not use it under Bun, Deno, or Cloudflare Workers. -Call `initLogger()` once at startup, then run your application with the `--import` flag: - -```bash -node --import braintrust/hook.mjs ./dist/index.js -# or with tsx -npx tsx --import braintrust/hook.mjs ./src/index.ts -``` - -**Any runtime with a bundler (Next.js, webpack, Vite, esbuild, etc.)** - -Use the appropriate Braintrust bundler plugin / framework integration -- see https://www.braintrust.dev/docs/instrument/trace-llm-calls for the supported plugins and framework setup (e.g. Next.js `instrumentation.ts`, webpack/Vite/esbuild plugins). This is the preferred option whenever a bundler is in play and works under Node, Bun, Deno, and Cloudflare Workers alike. - -**Bun / Deno / Cloudflare Workers without a bundler → use manual wrappers** - -There is no automatic instrumentation path for these runtimes without a bundler. Use manual wrappers (`wrapOpenAI`, `wrapAnthropic`, `wrapAISDK`, etc.) at call sites instead -- see https://www.braintrust.dev/docs/instrument/trace-llm-calls for the available wrappers and how to apply them. - -If none of the above is configured, automatic instrumentation will silently do nothing. - -### Requirement: persist the launch hook into the normal run path - -Auto-instrumentation requires the application to be started with the hook on every run. A one-off `node --import braintrust/hook.mjs ...` or a shell-local `export NODE_OPTIONS=...` during verification is **not enough** if the next developer, CI job, or deploy will go back to plain `node`, `tsx`, or `npm start`. - -Persist the hook into whichever launch path the project actually uses: - -- **`package.json` scripts**: update `start`, `dev`, `serve`, etc. to include `--import braintrust/hook.mjs`, for example: - ```json - "start": "node --import braintrust/hook.mjs dist/index.js", - "dev": "tsx --import braintrust/hook.mjs src/index.ts" - ``` -- **`Dockerfile` / container entrypoint**: update the `CMD` / `ENTRYPOINT` or a checked-in start script so containers launch with the hook. -- **Process managers / deploy config**: update `Procfile`, systemd units, PM2 config, ECS task definitions, Kubernetes manifests, etc. that define the real start command. -- **Checked-in env/config**: if the project already uses a checked-in mechanism for env vars, set `NODE_OPTIONS="--import braintrust/hook.mjs"` there. Do **not** rely on a shell-local `export NODE_OPTIONS=...` -- it will not help the next user or CI run. -- **Bundler / framework config**: if a bundler plugin is used, register it in the project's real bundler/framework config file, not in an ad-hoc script. - -If you add `initLogger` but do **not** modify any persisted launch path, treat the installation as incomplete. - -Verify using the same persisted command the project will actually use (e.g. `npm start`, `npm run dev`, `docker run ...`), not a custom one-off invocation. - -## Run the application - -Prefer the project's existing launch entrypoint, and make sure that entrypoint now loads the Braintrust hook (or bundler plugin) automatically. - -Try to figure out how the project is normally run from the project structure: - -- **npm scripts**: prefer `npm start` / `npm run dev` / `pnpm dev` / `yarn start` / `bun run start` -- update the script in `package.json` so it includes `--import braintrust/hook.mjs`, for example: - ```json - "start": "node --import braintrust/hook.mjs dist/index.js", - "dev": "tsx --import braintrust/hook.mjs src/index.ts" - ``` -- **Next.js**: `npm run dev` or `npx next dev` -- wire the Braintrust bundler/framework integration in the project's real Next.js config (e.g. `instrumentation.ts`), not a one-off command -- **ts-node**: ts-node does not support `--import`; migrate to `tsx` instead (`npm install --save-dev tsx`) and update the persisted script -- **tsx**: update the persisted script to `tsx --import braintrust/hook.mjs src/index.ts` -- **Node with TypeScript**: update the persisted build + start scripts to `tsc && node --import braintrust/hook.mjs dist/index.js` -- **Bun**: `bun run