Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .changeset/pi-tool-current-span.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
"braintrust": patch
---

Keep Pi Coding Agent tool spans current during execution so nested spans attach to the tool span.
fix(pi-coding-agent): keep tool spans current during execution so nested spans attach to the tool span
9 changes: 9 additions & 0 deletions .changeset/span-export-hooks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"braintrust": minor
---

feat: add span export hooks

Support synchronous `onSpanExport` customizers for incremental instrumentation
span records. Customizers can add, modify, delete, or replace fields before export,
with callbacks applied once per record rather than once per transport retry.
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ pnpm run test # Run all workspace tests via turbo

Run from the repo root. **Always run `fix:formatting` before committing** — there is a pre-commit hook that will reject unformatted code.

Agents MUST run Prettier on every file they create or edit before handing work back, even when no commit is requested. Include Markdown, changelogs, config files, and generated files supported by Prettier—not just source code. From the repo root, run `pnpm exec prettier --write <edited-files>` followed by `pnpm exec prettier --check <edited-files>`. If further edits are made, repeat formatting and verification after the final edit. Do not rely on tests, typechecks, CI, or the pre-commit hook to catch formatting issues.

```bash
pnpm run formatting # Check formatting (prettier)
pnpm run lint # Run eslint checks
Expand Down
6 changes: 6 additions & 0 deletions js/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# braintrust

## Unreleased

### Minor Changes

- feat: Run configured `onSpanExport` customizers on incremental instrumentation span records, supporting field mutation, deletion, and replacement before export.

## 3.33.0

### Minor Changes
Expand Down
6 changes: 5 additions & 1 deletion js/src/exports.ts
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,10 @@ export {
braintrustFlueObserver,
braintrustFlueInstrumentation,
} from "./instrumentation";
export type { InstrumentationConfig } from "./instrumentation";
export type {
InstrumentationConfig,
SpanCustomizer,
SpanExportData,
} from "./instrumentation";

export { wrapElevenLabs } from "./wrappers/elevenlabs";
43 changes: 43 additions & 0 deletions js/src/instrumentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,49 @@ termination, and async context.
- Use narrow vendored provider interfaces shared by wrappers and plugins.
- Keep enable, disable, subscription, and patching behavior idempotent.

## Export Customizers

Configure `spanCustomizers` through the standalone instrumentation entrypoint
before importing the main SDK, which enables instrumentation during platform
initialization. Use a bootstrap module before any auto-instrumentation preload
that initializes the SDK. Static imports of the main SDK are hoisted; use a
dynamic import after configuration:

```ts
import { configureInstrumentation } from "braintrust/instrumentation";

configureInstrumentation({
spanCustomizers: [
{
onSpanExport(data) {
data.tags = ["reviewed"];
if ("output" in data) data.output = "[redacted]";
delete data.error;
return data;
},
},
],
});

const { initLogger } = await import("braintrust");
initLogger({ projectName: "my-project" });
// Import and use instrumented provider SDKs here.
```

`onSpanExport` receives each incremental record from an instrumentation-created
span after lazy values resolve, before attachment processing, merging, masking,
and JSON serialization. It can run before the span ends; fields may be absent.
Ordinary manually created spans, dataset rows, and feedback are not customized.

Callbacks run synchronously in registration order. Mutate and return the record,
or return a replacement for the next callback. Preserve identity and routing
fields and return JSON-serializable data. Exceptions are swallowed; remaining
customizers and export continue. Export retries reuse the transformed record
without invoking callbacks again. Configuration is shared across SDK bundles.

Customizers receive only the outgoing record, not a live span or provider
instrumentation context.

## Testing

Test at the narrowest useful layers:
Expand Down
20 changes: 20 additions & 0 deletions js/src/instrumentation/config.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
export type SpanExportData = Record<string, unknown>;

export interface SpanCustomizer {
/**
* Customize an outgoing span record after lazy values resolve, before JSON
* serialization. Records are incremental and may not contain every span field.
*
* Add, change, or delete fields, then return the record or a replacement.
* Preserve identity and routing fields, including id, span_id, root_span_id,
* and span_parents.
*/
onSpanExport?(data: SpanExportData): SpanExportData;
}

export interface InstrumentationIntegrationsConfig {
openai?: boolean;
anthropic?: boolean;
Expand Down Expand Up @@ -46,6 +60,12 @@ export interface InstrumentationConfig {
* Set to false to disable instrumentation for that SDK.
*/
integrations?: InstrumentationIntegrationsConfig;

/**
* Instrumentation-wide customizers, in callback execution order.
* Configure before instrumentation is enabled.
*/
spanCustomizers?: readonly SpanCustomizer[];
}

const envIntegrationAliases: Record<
Expand Down
1 change: 1 addition & 0 deletions js/src/instrumentation/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,4 @@ export {
// Configuration API
export { configureInstrumentation } from "./registry";
export type { InstrumentationConfig } from "./registry";
export type { SpanCustomizer, SpanExportData } from "./config";
4 changes: 4 additions & 0 deletions js/src/instrumentation/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
type InstrumentationConfig,
} from "./config";
import { GLOBAL_INSTRUMENTATION_HOOKS_PROTOCOL_VERSION } from "../global-instrumentation-hooks";
import { setSpanCustomizers } from "../span-customizer";

export type { InstrumentationConfig } from "./config";

Expand Down Expand Up @@ -62,6 +63,9 @@ class PluginRegistry {
return;
}
this.config = { ...this.config, ...config };
if ("spanCustomizers" in config) {
setSpanCustomizers(config.spanCustomizers);
}
}

/**
Expand Down
39 changes: 25 additions & 14 deletions js/src/logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ import {
mergeSpanOriginContext,
type SpanOriginEnvironment,
} from "./span-origin";
import { customizeSpanExport } from "./span-customizer";

// Manual type definition for inline attachments (not in generated_types)
const InlineAttachmentReferenceSchema = z.object({
Expand Down Expand Up @@ -8215,6 +8216,7 @@ export class SpanImpl implements Span {

private isMerge: boolean;
private loggedEndTime: number | undefined;
private readonly isInstrumented: boolean;
private propagatedEvent: StartSpanEventArgs | undefined;

// For internal use only.
Expand Down Expand Up @@ -8255,6 +8257,8 @@ export class SpanImpl implements Span {
const instrumentationName =
getSpanInstrumentationName(args) ??
INSTRUMENTATION_NAMES.BRAINTRUST_JS_LOGGER;
this.isInstrumented =
instrumentationName !== INSTRUMENTATION_NAMES.BRAINTRUST_JS_LOGGER;

const spanAttributes = args.spanAttributes ?? {};
const rawEvent = args.event ?? {};
Expand Down Expand Up @@ -8422,21 +8426,28 @@ export class SpanImpl implements Span {
);
}

const computeRecord = async () => ({
...partialRecord,
...Object.fromEntries(
await Promise.all(
Object.entries(lazyInternalData).map(async ([key, value]) => [
key,
await value.get(),
]),
const computeRecord = async () => {
const record = {
...partialRecord,
...Object.fromEntries(
await Promise.all(
Object.entries(lazyInternalData).map(async ([key, value]) => [
key,
await value.get(),
]),
),
),
),
...new SpanComponentsV3({
object_type: this.parentObjectType,
object_id: await this.parentObjectId.get(),
}).objectIdFields(),
});
...new SpanComponentsV3({
object_type: this.parentObjectType,
object_id: await this.parentObjectId.get(),
}).objectIdFields(),
};
// Customize inside the memoized lazy value, before attachment processing,
// merging, and masking. Retries reuse the already-customized record.
return this.isInstrumented
? (customizeSpanExport(record) as BackgroundLogEvent)
: record;
};
this._state.bgLogger().log([new LazyValue(computeRecord)]);
}

Expand Down
Loading
Loading