Skip to content

[Feat] [SDK-399] Add java agent for network telemetry events - #374

Open
buongarzoni wants to merge 38 commits into
masterfrom
feat/SDK-399/add-java-agent-for-network-telemetry-events
Open

[Feat] [SDK-399] Add java agent for network telemetry events#374
buongarzoni wants to merge 38 commits into
masterfrom
feat/SDK-399/add-java-agent-for-network-telemetry-events

Conversation

@buongarzoni

@buongarzoni buongarzoni commented May 26, 2026

Copy link
Copy Markdown
Collaborator

Description of the change

Add Java agent for automatic network telemetry capture

Auto-instruments all major HTTP clients via -javaagent: using ByteBuddy, capturing 4xx/5xx responses as Rollbar telemetry events with no changes at HTTP call sites.

Scope of "no code changes": request code is never touched — no wrappers, no interceptors, no per-call bookkeeping, and nothing to remember when a new HTTP call is added. Setup is a one-time wiring step: the agent JAR on the application classpath, and .telemetryEventTracker(RollbarAgent.getTelemetryTracker()) on the config builder.

That wiring is not automatic by design of the current SDK: ConfigBuilder.build() installs its default RollbarTelemetryEventTracker whenever telemetryEventTracker(...) was not called, and there is no global registry or ServiceLoader hook an agent could claim instead. Making the agent self-installing would require a change to rollbar-java and is tracked separately.

Caution

This module targets JVM-based applications only. Android is not supported — ART does not implement
the java.lang.instrument API required by Java agents. Android users should use the existing
rollbar-android module instead.

The acceptance criteria on the Shortcut story need the same narrowing — "zero application code changes" → "no changes at HTTP call sites; one-time tracker wiring at init".

What's included:

  • New rollbar-java-agent module — shadow JAR with ByteBuddy bundled and relocated
  • Instruments HttpURLConnection, java.net.http.HttpClient, Apache HC 4.x and 5.x
  • URL sanitization (strips credentials, query params, fragment before recording)
  • Deduplication via WeakHashMap to handle re-entrant getResponseCode() calls and dual-advice firing on HttpClient
  • Integration tests using WireMock 3.x for each instrumented client
  • README with installation and manual testing guide

Usage:

  -javaagent:/path/to/rollbar-java-agent.jar
  Rollbar.init(withAccessToken("...")
      .telemetryEventTracker(RollbarAgent.getTelemetryTracker())
      .build());

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Maintenance
  • New release

Related issues

Shortcut stories and GitHub issues (delete irrelevant)

Checklists

Development

  • Lint rules pass locally
  • The code changed/added as part of this pull request has been covered with tests
  • All tests related to the changed code pass in development

Code review

  • This pull request has a descriptive title and information useful to a reviewer. There may be a screenshot or screencast attached
  • "Ready for review" label attached to the PR and reviewers assigned
  • Issue from task tracker has a link to this pull request
  • Changes have been reviewed by at least one other engineer

@linear-code

linear-code Bot commented May 26, 2026

Copy link
Copy Markdown

SDK-399

@buongarzoni

Copy link
Copy Markdown
Collaborator Author

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization has reached its monthly code review spending cap.

An organization admin can view or raise the cap at claude.ai/admin-settings/claude-code. The cap resets at the start of the next billing period.

Once the cap resets or is raised, comment @claude review on this pull request to trigger a review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ad0b2acf09

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@buongarzoni

Copy link
Copy Markdown
Collaborator Author

@claude review

Comment thread rollbar-java-agent/src/main/java/com/rollbar/agent/AgentTelemetryStore.java Outdated
@brianr
brianr self-requested a review June 15, 2026 22:09
@buongarzoni buongarzoni added this to the v2.4.0 milestone Jun 15, 2026

@brianr brianr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posting review findings from the local review.

Comment thread rollbar-java-agent/build.gradle.kts Outdated
.type(ElementMatchers.named("java.net.HttpURLConnection"))
.transform((b, typeDescription, classLoader, module, protectionDomain) ->
b.visit(Advice.to(GetResponseCodeAdvice.class)
.on(ElementMatchers.named("getResponseCode")))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Capture HttpURLConnection requests that skip getResponseCode

For HttpURLConnection callers that trigger the request with getInputStream() or getErrorStream() and never call getResponseCode(), this is the only advised method, so a 4xx/5xx response (or the IOException thrown by getInputStream() on 4xx) is never recorded. This leaves a common HttpURLConnection usage path outside the promised automatic network-error capture.

Comment thread rollbar-java-agent/src/main/java/com/rollbar/agent/UrlSanitizer.java Outdated

@brianr brianr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on the current head. The inline comments cover four release blockers. Two existing review threads also still apply, so I have not duplicated them: the agent still records Rollbar's own SyncSender failures, and NetworkEventBridge.composeUrl() still treats :// inside a relative URI's query/path as an absolute URI. The former thread is marked resolved even though no suppression guard is present at this head.

// from the authority manually.
String authority = uri.getAuthority();
if (authority != null) {
int at = authority.indexOf('@');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve encoded credentials while locating the userinfo delimiter

URI.getAuthority() returns the decoded authority. For https://user:p%40ss@example.com/path, it yields user:p@ss@example.com; indexOf('@') then keeps ss@example.com, so part of the password is recorded as new userinfo. Please operate on getRawAuthority() / getRawUserInfo() (or otherwise locate the delimiter in the raw authority) before reconstructing the sanitized URL, and add a regression test with percent-encoded @ in userinfo.

Oracle documents the decoding behavior here: https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/net/URI.html#getAuthority()

Comment thread rollbar-java-agent/build.gradle.kts Outdated
}

dependencies {
implementation("net.bytebuddy:byte-buddy:1.14.18")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Support the advertised JVM range

Byte Buddy 1.14.18 supports class files only through Java 23 without experimental mode; Java 24 support starts at 1.15.4 and Java 25+ at 1.17.0. The README promises Java 11 or higher, but CI exercises only 11 and 17, so the agent can silently fail to transform HTTP classes on current Java 24/25/26 runtimes. Please upgrade Byte Buddy and add current-LTS/current-JDK coverage, or explicitly narrow the supported range.

Compatibility table: https://github.com/raphw/byte-buddy#java-version-compatibility

implementation("net.bytebuddy:byte-buddy:1.14.18")
implementation("net.bytebuddy:byte-buddy-agent:1.14.18")
api(project(":rollbar-api"))
implementation(project(":rollbar-java"))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Keep Rollbar SDK classes out of the shaded agent

shadowJar merges runtimeClasspath by default, so this implementation dependency and the rollbar-api api dependency are embedded unrelocated in the fat JAR (along with transitive dependencies such as SLF4J); only Byte Buddy is relocated below. The README also puts the agent alongside the application's Rollbar SDK, creating duplicate com.rollbar.* classes that can pin/override another SDK version or split class identity in containers. Please shade only agent-private dependencies (for example via a dedicated Byte Buddy configuration) and leave the Rollbar API/SDK as ordinary external dependencies.

Shadow's default dependency behavior: https://gradleup.com/shadow/configuration/dependencies/

Comment thread rollbar-java-agent/README.md Outdated
@@ -0,0 +1,202 @@
# Rollbar Java Agent

A zero-code-change Java instrumentation agent that automatically captures HTTP network errors (4xx and 5xx responses) as Rollbar telemetry events.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Align the implementation with the zero-code-change claim

The instrumentation removes changes at HTTP call sites, but installation still requires adding the agent as an application dependency and changing Rollbar.init(...) to install RollbarAgent.getTelemetryTracker() (steps 3–4). The behavior table also says that without this wiring events accumulate but are not sent. If SDK-399 requires genuinely zero application-code changes, the agent needs to integrate its tracker automatically; otherwise this claim should be narrowed to "no HTTP client call-site changes" and the stated problem/acceptance criteria updated.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the two nit-level findings from this run's inline comments, I also checked a few other candidate issues and ruled them out: instrumenting the Rollbar SDK's own SyncSender does not create a feedback loop, the repeated classloader/bridge-lookup boilerplate across the four HttpUrlConnectionInstrumentation advice classes is a style-only duplication (not a bug), the README's "zero-code-change" wording is accurate for the HTTP call-site instrumentation itself, and the HC4/HC5 hierarchy-walk name filter excluding only JDK prefixes is a performance micro-optimization rather than a correctness gap.

Extended reasoning...

This run's findings are both nits (a redirect-misattribution edge case in the JDK HttpClient advice, and an unnecessary-reflection quality nit in HttpUrlConnectionInstrumentation), and the PR has already absorbed a long sequence of substantive P1/P2 fixes from prior review rounds (recursive getResponseCode() guard, doExecute()-based Apache HC4/HC5 unification covering the HttpHost overloads, UrlSanitizer host-corruption and encoded-credential fixes, composeUrl scheme-position bounding, shading only Byte Buddy instead of the Rollbar SDK, Byte Buddy version bump for current JDKs, and preventing the thin jar from clobbering the shaded one). Given this module's complexity (bytecode instrumentation across four HTTP clients) and its security-sensitive URL-sanitization logic, I'm not approving outright, but wanted to record the additional items examined and ruled out this run so they aren't re-explored from scratch.

Comment on lines +129 to +138
if (response != null) {
int statusCode = (Integer) response.getClass().getMethod("statusCode").invoke(response);
if (statusCode >= 400) {
Object uri = request.getClass().getMethod("uri").invoke(request);
String method = (String) request.getClass().getMethod("method").invoke(request);
// response object is the dedup key — unique per send() call, shared between
// HttpClientFacade and HttpClientImpl so only one event is recorded
bridge.getMethod("recordNetworkEvent",
Object.class, String.class, String.class, String.class)
.invoke(null, response, method, uri.toString(), String.valueOf(statusCode));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The JDK HttpClient advice (SendAdvice.onExit sync path and NetworkEventBridge.createAsyncCallback async path) reads the URL/method off the original pre-redirect request argument instead of the response, so when the client is configured with a redirect policy other than the default NEVER (e.g. Redirect.NORMAL), a final-hop 4xx/5xx is recorded with the original request's host/method rather than the actual failing one — unlike the other three instrumented clients, whose instrumentation point already sees the final target. Fix by using response.uri() and response.request().method() instead of the request argument in both SendAdvice.onExit (JavaHttpClientInstrumentation.java:132-138) and the HttpResponse branch of createAsyncCallback (NetworkEventBridge.java).

Extended reasoning...

What's wrong: SendAdvice.onExit (JavaHttpClientInstrumentation.java:129-138) records telemetry using request.getClass().getMethod("uri").invoke(request) and request.getClass().getMethod("method").invoke(request), where request is @Advice.Argument(0) — the original HttpRequest object passed into HttpClient.send(...), not anything derived from the returned response. The async path (SendAsyncAdviceNetworkEventBridge.createAsyncCallback) has the identical pattern: the callback closes over the original request object captured at sendAsync() time and reads uri()/method() off of it in the response.statusCode() >= 400 branch.

Why this is wrong: java.net.http.HttpClient, when configured with a redirect policy other than the default Redirect.NEVER (e.g. HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NORMAL)), follows the entire redirect chain internally inside one send()/sendAsync() call. The javadoc for HttpResponse.uri() says the returned URI "may be different from the request URI if redirection occurred," and HttpResponse.request() returns the actual final HttpRequest (whose method can also change — a 303 converts POST to GET). So when the chain ends in a 4xx/5xx, the response correctly reflects the final hop, but the code reads the pre-redirect request's URI/method instead.

Why nothing else in the code catches this: there is no logic anywhere in SendAdvice/createAsyncCallback that inspects redirect history or consults response.request() — the response is used only for statusCode() and as the WeakHashMap dedup key, never for its own uri()/request().

Step-by-step proof:

  1. App code: HttpClient client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NORMAL).build();
  2. client.send(HttpRequest.newBuilder(URI.create("https://api.example.com/widgets")).build(), ...).
  3. api.example.com responds 301 → Location: https://cdn.example.com/widgets.
  4. The JDK client follows the redirect internally (still inside the single send() call) and issues the request to cdn.example.com.
  5. cdn.example.com responds 500.
  6. send() returns the final HttpResponse, whose uri() is https://cdn.example.com/widgets and whose request() is the request that was actually sent to cdn.example.com.
  7. SendAdvice.onExit fires with @Advice.Argument(0) request still bound to the original request object built in step 2 (URI https://api.example.com/widgets), and reads request.uri()/request.method() from it.
  8. Recorded telemetry: {method: <original>, url: "https://api.example.com/widgets", status_code: "500"} — attributing the 500 to api.example.com, when the failing dependency was actually cdn.example.com.

Impact: this defeats the purpose of the URL field, which exists specifically to let a developer identify which downstream host actually failed. A developer investigating the error would look at (and possibly page/alert on) the wrong service.

Fix: in SendAdvice.onExit, replace request.uri()/request.method() with response.uri() and response.request().method() (both available via reflection on the HttpResponse/HttpRequest interfaces, consistent with how the rest of the advice already does reflective lookups). The same substitution applies to the HttpResponse branch inside NetworkEventBridge.createAsyncCallback.

Severity: nit, not normal — it requires a non-default, opt-in redirect policy (the JDK's own default is Redirect.NEVER, under which a 3xx is simply not recorded at all) combined with a redirect chain that terminates in an error. The failure mode is misattributed/degraded telemetry, not a crash, exception, or data loss, and the misattributed URL is still a URL the app genuinely requested (just the wrong hop). This is consistent with how the similarly-scoped composeUrl nested-URL host-misattribution finding elsewhere in this PR was rated.

Comment on lines +171 to +178
if (statusCode >= 400) {
Object url = connection.getClass().getMethod("getURL").invoke(connection);
String urlStr = url != null ? url.toString() : "";
String method = (String) connection.getClass()
.getMethod("getRequestMethod").invoke(connection);
bridge.getMethod("recordNetworkEvent",
Object.class, String.class, String.class, String.class)
.invoke(null, connection, method, urlStr, String.valueOf(statusCode));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 GetResponseCodeAdvice.onExit() (and the equivalent code in GetInputStreamAdvice/GetErrorStreamAdvice) reflects into getURL()/getRequestMethod()/getResponseCode() via connection.getClass().getMethod(...).invoke(...), but these are public methods declared directly on the bootstrap-loaded java.net.HttpURLConnection/URLConnection, visible from every classloader — unlike the genuine NetworkEventBridge lookup in the same methods, which does need reflection to cross into the app classloader. Typing @Advice.This as HttpURLConnection and calling the methods directly removes six reflective lookups plus their checked-exception handling from a per-error-response path with identical behavior, since ByteBuddy inlines the advice regardless of the declared parameter type.

Extended reasoning...

What the finding is

GetResponseCodeAdvice.onExit (lines 171-178) reflectively invokes getURL() and getRequestMethod() on the HttpURLConnection instance via connection.getClass().getMethod(name).invoke(connection). The same pattern recurs for getResponseCode() in GetInputStreamAdvice.onExit (line 94) and GetErrorStreamAdvice.onExit (line 125).

All three methods — getResponseCode() and getRequestMethod() on java.net.HttpURLConnection, getURL() on its superclass java.net.URLConnection — are public and declared on bootstrap-loaded java.base classes. There is no classloader gap to bridge: every type this advice is inlined into (HttpURLConnection itself, or a concrete subclass like sun.net.www.protocol.http.HttpURLConnection) is a subtype of the bootstrap class HttpURLConnection, so a direct invokevirtual reference resolves from any classloader. This is fundamentally different from the NetworkEventBridge lookup a few lines below in the same methods, which genuinely must go through Thread.currentThread().getContextClassLoader().loadClass(...) because NetworkEventBridge lives in the application classloader and is invisible from a bootstrap-inlined advice body.

Why the change is safe

Typing @Advice.This as HttpURLConnection instead of Object is valid for every instrumented site: GetResponseCodeAdvice is inlined directly into java.net.HttpURLConnection.getResponseCode() (exact match), and GetInputStreamAdvice/GetErrorStreamAdvice target concrete subtypes, for which HttpURLConnection is always an assignable supertype. ByteBuddy inlines advice bytecode into the target method regardless of the advice parameter's declared type, so this is a purely mechanical substitution — connection.getResponseCode()/getURL()/getRequestMethod() called directly instead of via Method.invoke. getURL() and getRequestMethod() declare no checked exceptions, and getResponseCode()'s IOException is already caught by the surrounding catch (Throwable ignored), so no new exception handling is needed at the call site.

Step-by-step proof (GetResponseCodeAdvice)

  1. A 404 response triggers getResponseCode() to return, statusCode >= 400.
  2. Current code: connection.getClass().getMethod("getURL").invoke(connection) — a reflective lookup + invoke against a bootstrap class the calling code could reference directly.
  3. Replacement: connection.getURL() — same bytecode-inlined call site, same return value, no reflection, no NoSuchMethodException/IllegalAccessException/InvocationTargetException handling needed.
  4. Identical for getRequestMethod(), and for getResponseCode() in the two other advice classes.

Impact

Six reflective getMethod()+invoke() calls are removed from the response/error-handling path (fired on every 4xx/5xx HttpURLConnection response), improving clarity and per-response cost with no behavior change. It is also strictly safer under JPMS than reflecting on an internal JDK implementation class. This is a pure code-quality/efficiency cleanup — nothing about program correctness changes — so it does not block merge.

Comment on lines +40 to +49
private static void installInstrumentation(Instrumentation inst) {
// Override ByteBuddy's default which ignores all java.* and javax.* classes,
// so we can instrument JDK HTTP clients (HttpURLConnection, HttpClient).
// We still ignore ByteBuddy's own classes to avoid instrumentation loops.
AgentBuilder builder = new AgentBuilder.Default()
.ignore(ElementMatchers.nameStartsWith("net.bytebuddy.")
.or(ElementMatchers.nameStartsWith("com.rollbar.agent.shaded.")))
.with(new ErrorReportingListener())
.with(AgentBuilder.InitializationStrategy.NoOp.INSTANCE)
.with(AgentBuilder.TypeStrategy.Default.REDEFINE);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 installInstrumentation() (RollbarAgent.java:40-49) never sets .with(AgentBuilder.RedefinitionStrategy.RETRANSFORMATION), so ByteBuddy stays at the default RedefinitionStrategy.DISABLED and only transforms classes loaded after installOn(inst) runs — it never retransforms classes already loaded. This silently breaks the dynamic-attach path exposed via agentmain (and the manifest's Can-Redefine-Classes/Can-Retransform-Classes: true), since HTTP client classes are almost always already loaded in a running JVM by the time attach happens, so none of the four instrumentations install and zero telemetry is recorded with no error raised.

Extended reasoning...

What the bug is: RollbarAgent.installInstrumentation() builds its AgentBuilder (lines 40-49) with an ignore filter, the ErrorReportingListener, InitializationStrategy.NoOp, and TypeStrategy.Default.REDEFINE — but never calls .with(AgentBuilder.RedefinitionStrategy...). ByteBuddy's AgentBuilder.Default defaults RedefinitionStrategy to DISABLED. Under DISABLED, installOn(inst) registers the ClassFileTransformer with inst.addTransformer(transformer, /* canRetransform */ false) — the transformer only sees types as they are freshly loaded from that point forward. It never iterates over and retransforms classes the JVM already has loaded at the moment installOn runs. Note TypeStrategy.Default.REDEFINE (which is set) is a different, independent setting — it controls how a matched type's bytecode is rewritten, not whether already-loaded types get revisited — so it does not compensate for the missing RedefinitionStrategy.\n\nThe code path that triggers it: RollbarAgent exposes a public agentmain(String, Instrumentation) (lines 36-38) specifically for dynamic attach to an already-running JVM via VirtualMachine.loadAgent(...), and build.gradle.kts's manifest sets both Agent-Class and Can-Redefine-Classes/Can-Retransform-Classes: true — JVM-level permissions that exist for exactly this scenario. But in a real running application at the moment of attach, java.net.HttpURLConnection, java.net.http.HttpClient, and any already-used Apache HttpClient classes are, in virtually every case, already loaded. With RedefinitionStrategy left at DISABLED, none of the four installIfAvailable/install calls in installInstrumentation can retransform those already-loaded classes, so no advice is ever woven into them.\n\nWhy nothing else catches it: ErrorReportingListener.onError is the only failure-surfacing mechanism in this code, but it only fires when a transform attempt is made and fails to apply — here, no transform is even attempted on the already-loaded classes, so onError never fires. The agent's agentmain returns normally, giving every outward signal of successful attachment while silently receives zero events for the rest of the process's life.\n\nStep-by-step proof:\n1. A long-running application is already executing, having triggered classloading of sun.net.www.protocol.http.HttpURLConnection (or 's impl classes) well before any Rollbar tooling attaches.\n2. An operator (or tooling) dynamically attaches this agent via VirtualMachine.attach(pid).loadAgent(jarPath), invoking RollbarAgent.agentmain(args, inst).\n3. installInstrumentation(inst) builds the AgentBuilder and calls HttpUrlConnectionInstrumentation.install(builder, inst) (and the other three installIfAvailable calls), each ending in .installOn(inst).\n4. Because RedefinitionStrategy was never set, installOn calls inst.addTransformer(transformer, false)false meaning "do not retransform currently loaded classes."\n5. The already-loaded HttpURLConnection/HttpClient/Apache HC classes are never revisited by the JVM; no transform is attempted, so ErrorReportingListener.onError never fires.\n6. The application continues making HTTP calls through those already-loaded, un-instrumented classes. Every 4xx/5xx response goes completely unrecorded — AgentTelemetryStore stays empty for the process's entire remaining lifetime, with no log line or exception anywhere indicating the failure.\n\nHow to fix: add .with(AgentBuilder.RedefinitionStrategy.RETRANSFORMATION) to the builder in installInstrumentation(). The manifest already grants the required Can-Retransform-Classes: true JVM permission, so this is a pure one-line fix with no other changes needed; advice inlining here only rewrites method bodies, which retransformation supports without any schema/field changes.\n\nSeverity: the documented and tested path is -javaagent:/premain, where HTTP classes are loaded lazily after premain installs the transformer — this is proven by the passing WireMock integration test suite. Dynamic attach via agentmain is not documented in the README, even though the code (agentmain) and manifest (Agent-Class, Can-Redefine-Classes/Can-Retransform-Classes) both advertise it as a supported entry point. Since merging without this fix does not break the primary, documented feature, this should not block the PR — but it is a genuine, reproducible defect on an entry point the module explicitly exposes and grants JVM permissions for, and is worth a one-line fix (or removing the agentmain/Agent-Class surface if dynamic attach isn't actually meant to be supported yet).

Comment on lines +100 to +117
}

if (response != null && request != null) {
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode >= 400) {
// The host-based overloads carry the target separately from a request whose URI may be
// just a path, so rejoin the two rather than reading the request URI alone.
String base = target != null ? target.toURI() : null;
String requestUri = request.getRequestLine() != null
? request.getRequestLine().getUri() : null;
NetworkEventBridge.recordNetworkEvent(
response,
request.getRequestLine().getMethod(),
NetworkEventBridge.composeUrl(base, requestUri),
String.valueOf(statusCode)
);
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The doExecute(HttpHost, request, context) advice reads its recorded host/URI off the original @Advice.Argument(0)/(1) parameters, but Apache HttpClient (both HC4 and HC5) follows redirects internally via RedirectExec, which reassigns only its own local currentRequest/currentRoute and never mutates the objects doExecute is holding. So when a request to host A 3xx-redirects to host B and host B returns a 4xx/5xx, the recorded telemetry pairs the final status code with host A instead of the failing host B. This is a pre-existing design limitation, unrelated to any change introduced by this PR — it stems from choosing doExecute() as the single instrumentation point, which is otherwise the right choice for overload coverage.

Extended reasoning...

What the bug is. DoExecuteAdvice.onExit in ApacheHttpClient4Instrumentation.java (lines 100-117) binds @Advice.Argument(0) target and @Advice.Argument(1) request — the exact parameters doExecute(HttpHost, HttpRequest, HttpContext) was originally invoked with — and uses them, together with @Advice.Return response, to build the recorded URL via NetworkEventBridge.composeUrl(target.toURI(), request.getRequestLine().getUri()). ApacheHttpClient5Instrumentation.java's DoExecuteAdvice has the identical shape, reading request.getUri()/request.getMethod() off its own doExecute argument.

Why this is wrong. InternalHttpClient.doExecute() (verified via javap on httpclient-4.5.14) wraps the incoming request into a fresh HttpRequestWrapper and hands it to the exec chain (execChain.execute(route, wrapper, context)); it never reassigns its own target/request locals afterward. RedirectExec, the outermost element of that chain, is what actually follows a redirect: on each hop it reassigns only its own local currentRequest/currentRoute (via HttpRequestWrapper.wrap(redirect)), then returns the final response from its loop. It never touches the target/request objects that doExecute itself is holding. So for a redirect chain from host A to host B ending in a 4xx/5xx, doExecute() returns the true final (host B) response, but its own target/request parameters — the ones the advice reads — still describe host A. HC5's InternalHttpClient/RedirectExec follow the same wrap-and-reassign-local pattern, so ApacheHttpClient5Instrumentation has the same issue.

Reachability. This is on the default path: HttpClients.createDefault() enables DefaultRedirectStrategy, so HC4/HC5 follow redirects out of the box for GET/HEAD (and 307/308 for any method). Any followed redirect that ends in an error — cross-host, http→https, apex→www, a CDN hop — records the pre-redirect host/URL paired with the final status code.

Why nothing else in the code catches it. The advice never consults HttpClientContext (available as @Advice.Argument(2), and updated by RedirectExec to the final HTTP_TARGET_HOST/HTTP_REQUEST attributes), and ClassicHttpResponse/HttpResponse carry no back-reference to the request actually sent — so there is no way to recover the final target from the response alone, unlike java.net.http.HttpResponse, which does expose response.request().

Step-by-step proof.

  1. client.execute(new HttpGet("http://a.example.com/widgets")) where a.example.com 301-redirects to https://b.example.com/widgets.
  2. doExecute(target=a.example.com, request=GET /widgets, context) is invoked; target/request are bound as this method's own arguments.
  3. RedirectExec.execute() follows the 301 by reassigning its own local currentRequest/currentRoute to b.example.com, issues the redirected request, and gets back a 500.
  4. doExecute() returns that 500 HttpResponse — but its target/request parameters are unchanged, still a.example.com/GET /widgets.
  5. DoExecuteAdvice.onExit fires with target=a.example.com, request=GET /widgets, response=500, records {method: GET, url: "http://a.example.com/widgets", status_code: "500"}.
  6. The telemetry attributes the failure to a.example.com, when the request that actually failed went to b.example.com.

Impact and fix. This degrades (misattributes) the recorded URL/host for a redirected error — the status code is still correct, and no crash/data loss occurs, so it is a nit rather than a blocking issue. A fix would read the final target/request from HttpClientContext's HTTP_TARGET_HOST/HTTP_REQUEST attributes (updated by the exec chain on each redirect hop) instead of from doExecute's own arguments. This is distinct from the already-reported JavaHttpClientInstrumentation redirect finding (comment 2026-08-10T21:29:36Z): that finding's aside that HC4/HC5 "already see the final target" is incorrect, and its suggested fix (response.request()) does not apply here since Apache's ClassicHttpResponse has no such back-reference.

Comment on lines +160 to +168
if (thrown != null) {
Boolean recorded = (Boolean) bridge
.getMethod("markAsRecorded", Object.class).invoke(null, thrown);
if (recorded) {
String msg = thrown.getMessage() != null
? thrown.getMessage() : thrown.getClass().getName();
bridge.getMethod("recordError", String.class).invoke(null, msg);
}
return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit: the reflective markAsRecorded/recordError block is copy-pasted verbatim in three advice sites — HttpUrlConnectionInstrumentation.GetResponseCodeAdvice.onExit (lines 160-168), JavaHttpClientInstrumentation.SendAdvice.onExit, and SendAsyncAdvice.onExit — each doing two separate reflective Method lookups plus the same null-message fallback. A single NetworkEventBridge.recordThrowable(Throwable) helper would collapse each site to one reflective invoke, with no behavior change.

Extended reasoning...

What's duplicated: the same 5-line pattern — reflectively call markAsRecorded(thrown), and if it returns true, build a null-safe message (thrown.getMessage() != null ? thrown.getMessage() : thrown.getClass().getName()) and reflectively call recordError(message) — is copy-pasted verbatim across three advice classes: HttpUrlConnectionInstrumentation.GetResponseCodeAdvice.onExit (lines 160-168), JavaHttpClientInstrumentation.SendAdvice.onExit (~118-126), and JavaHttpClientInstrumentation.SendAsyncAdvice.onExit (~65-73). Each site performs two independent reflective getMethod()+invoke() calls (one for markAsRecorded, one for recordError) plus the identical message-fallback branch.\n\nThe same shape also shows up, non-reflectively, in the two Apache HC4/HC5 DoExecuteAdvice classes and in NetworkEventBridge.createAsyncCallback's thrown-exception branch — but the three reflective copies alone are enough to justify a shared helper.\n\nWhy this happened: each advice class is inlined into a different bootstrap/JDK class (HttpURLConnection, HttpClient's send/sendAsync) and has to cross the classloader gap to reach NetworkEventBridge, which lives in the application classloader. Because the crossing itself requires reflection, it was natural to write the whole mark+record sequence reflectively at each call site rather than factoring it into the bridge — but the branch and string logic don't need to be reflective at all; only the single entry-point call into does.\n\nThe fix: add one method to NetworkEventBridge — e.g. public static void recordThrowable(Throwable thrown) — that does the markAsRecorded check, builds the null-safe message, and calls recordError internally (all in the app classloader, no reflection needed for that part). Each of the three advice sites then reduces from five duplicated lines plus two reflective Method objects down to a single reflective invoke:\n\njava\nbridge.getMethod("recordThrowable", Throwable.class).invoke(null, thrown);\n\n\nStep-by-step proof this is safe and behavior-preserving:\n1. Today, GetResponseCodeAdvice.onExit does: look up markAsRecorded, invoke it with thrown → if true, look up recordError, build the message, invoke it.\n2. With the helper, NetworkEventBridge.recordThrowable(thrown) runs the identical two-step check but as plain Java inside the bridge class (which already has direct access to markAsRecorded/recordError since they're static methods on the same class).\n3. The advice site now looks up and invokes only recordThrowable, passing thrown through unchanged.\n4. Since recordThrowable's internal logic is byte-for-byte the same branch/message-building code that used to live at the call site, the recorded telemetry (message content, dedup behavior via markAsRecorded) is identical — only the amount of reflection and duplicated code changes.\n\nImpact: this is a pure code-quality/reuse cleanup — three (arguably five, counting the HC4/HC5 direct-call sites) copies of the same logic collapse to one, each reflective call site shrinks from ~8 lines and two Method lookups to one, and there's no behavior change. It doesn't block merging.

Comment on lines +124 to +141
}

/**
* Joins a base URI with a request URI, for clients that dispatch a target host separately from a
* request whose URI may be relative.
*
* <p>Apache HC's {@code doExecute(HttpHost, request, context)} receives the target host as its
* own argument, so a request issued through the host-based {@code execute(HttpHost, request)}
* overloads carries only a path (e.g. {@code /charge}). Rejoining the two is what keeps the host
* in the recorded URL. A request URI that is already absolute is returned untouched, and a null
* base (HC leaves the target null for a relative URI it could not resolve) degrades to the path
* alone.
*
* @param baseUri the target host as a URI (e.g. {@code https://api.example.com}), or null
* @param requestUri the request URI, absolute or relative, or null
* @return the joined URL — never null, so the caller always has something to sanitize
*/
public static String composeUrl(String baseUri, String requestUri) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 NetworkEventBridge.createAsyncCallback resolves Class.forName("java.net.http.HttpResponse") and Class.forName("java.net.http.HttpRequest") fresh inside the BiConsumer body, which fires on every completed async HTTP response (not only error ones, since resolving HttpResponse is needed just to read statusCode()).

Extended reasoning...

NetworkEventBridge.createAsyncCallback (lines 124-141) returns a BiConsumer that SendAsyncAdvice.onExit chains onto the future via whenComplete(...). That callback body runs once for every completed sendAsync() call. Inside it, Class.forName("java.net.http.HttpResponse") and Class.forName("java.net.http.HttpRequest") are re-resolved on every invocation, and — because statusCode() itself has to be invoked through httpResponseIface before the >= 400 gate is even checked — both lookups execute for every async response, including ordinary 2xx ones, not just failures.

Both types are fixed java.base interfaces that are always present and resolvable at agent-load time (the module's own build targets Java 11+, and java.net.http is an exported package per the comment already in this file). There is no case where the class identity of HttpResponse/HttpRequest could legitimately change between calls, so redoing the Class.forName lookup (a caller-sensitive stack walk plus a classloader loadClass call, even once the class is already loaded and initialized) on every response is unnecessary repeated work in what is effectively a hot path for any application making frequent async HTTP calls.

The fix is to resolve both Class<?> objects once and reuse them across invocations — either as fields resolved once per createAsyncCallback call (safest, since this method is only reached from the sendAsync advice path where java.net.http is guaranteed present) or as static final fields if NetworkEventBridge is confirmed to only ever be reached via that path. This is a pure efficiency cleanup: it does not change what gets recorded, when, or how — the resolved Class objects are identical either way — so it carries no behavioral risk.

Step-by-step proof:

  1. App code calls httpClient.sendAsync(request, bodyHandler) twice in a row for two different requests that both return 200 OK.
  2. SendAsyncAdvice.onExit chains a callback (from createAsyncCallback) onto each returned future via whenComplete.
  3. Both callbacks fire, once per completed future. Each independently executes Class.forName("java.net.http.HttpResponse") and Class.forName("java.net.http.HttpRequest") — two full lookups per response, four total for these two unrelated 2xx calls that never even reach the statusCode >= 400 branch.
  4. Hoisting the resolution out of the lambda body means the two Class.forName calls happen once (at createAsyncCallback invocation time, or once for the JVM lifetime if made static), and the returned callback just references the already-resolved Class objects — identical behavior, less repeated work per response.

Severity is nit: this is code-quality/efficiency cleanup only, with no correctness impact, and doesn't block merging.

Comment on lines +113 to +118
private static Object invokeVia(String apiTypeName, Object receiver, String methodName)
throws ReflectiveOperationException {
ClassLoader classLoader = receiver.getClass().getClassLoader();
Class<?> apiType = Class.forName(apiTypeName, false,
classLoader != null ? classLoader : ClassLoader.getSystemClassLoader());
return apiType.getMethod(methodName).invoke(receiver);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 ApacheHttpClient4Instrumentation.invokeVia (lines 113-118) and ApacheHttpClient5Instrumentation.invokeVia (lines 119-124) are byte-for-byte identical private reflection helpers, existing for the same reason in both files. Consider hoisting the shared logic into a common location (e.g. NetworkEventBridge) so a future fix doesn't need to be applied twice; this is a nit, not a blocker.

Extended reasoning...

ApacheHttpClient4Instrumentation.invokeVia (lines 113-118) and ApacheHttpClient5Instrumentation.invokeVia (lines 119-124) are byte-for-byte identical private static helpers. Both have the exact same signature — private static Object invokeVia(String apiTypeName, Object receiver, String methodName) throws ReflectiveOperationException — and the exact same three-statement body: resolve receiver.getClass().getClassLoader() (falling back to the system classloader if null), Class.forName(apiTypeName, false, classLoader), then apiType.getMethod(methodName).invoke(receiver).

Both copies exist for the identical structural reason: recordResponse in each instrumentation class needs to call methods on Apache HTTP objects (HttpResponse, StatusLine, HttpRequest/RequestLine in HC4; the hc5 equivalents in HC5) without the advice/support code naming those types directly at compile time in a way that could break classloading in embedding scenarios described elsewhere in this PR's review — so both classes fall back to reflecting through the receiver's own classloader.

Because the helper is fully generic — its own signature only mentions String/Object, never an org.apache.http/org.apache.hc type — nothing forces it to live inside the per-HTTP-client-version class. Both ApacheHttpClient4Instrumentation.recordResponse and ApacheHttpClient5Instrumentation.recordResponse already call NetworkEventBridge static methods directly (recordNetworkEvent, composeUrl), so NetworkEventBridge is the natural shared home; a public static method there (package-private wouldn't be reachable across the instrumentation package boundary) would let both instrumentation classes delegate instead of maintaining separate copies.

The risk of leaving this as-is is that a future change to the lookup strategy — for example, caching the Method object across calls instead of re-resolving it via getMethod() on every 4xx/5xx response, which another reviewer flagged as a possible efficiency improvement — would have to be remembered and applied in two files. Missing one during that kind of edit is an easy, quiet way to introduce version-specific behavior drift between the HC4 and HC5 instrumentations for no reason tied to any real difference between the two libraries.

Step-by-step proof of the duplication:

  1. Open ApacheHttpClient4Instrumentation.java:113-118 — see invokeVia(String, Object, String) resolving classloader, calling Class.forName, then getMethod().invoke().
  2. Open ApacheHttpClient5Instrumentation.java:119-124 — the method body is character-for-character the same aside from surrounding javadoc referencing "HC 5.x" vs "HC 4.x".
  3. Grep both files for org.apache inside invokeVia — neither method references a library type; the type name is always passed in as a caller-supplied String.
  4. Confirm both call sites (recordResponse in each file) already depend on NetworkEventBridge, showing that's the natural shared location without requiring a new file.

This is a pure reuse/DRY cleanup with no behavior change — extracting the method does not alter what gets recorded or how classloading is resolved, so it doesn't block merging. It's flagged as a nit for a maintainer to fold into a follow-up rather than something that needs to hold up this PR.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No bugs were found in this run. Given the scope and complexity of this change — a new bytecode-instrumentation agent with several JVM classloader edge cases — I'm deferring to a human reviewer rather than approving outright.

What was reviewed: the four HTTP client instrumentations (HttpURLConnection, java.net.http.HttpClient, Apache HC4/HC5) and their advice-to-bridge reflection paths, URL sanitization/credential-stripping, event dedup logic, and the Gradle shading/JDK-version setup. Two candidate issues were examined and ruled out: composeUrl's handling of protocol-relative request URIs, and the module-level (not per-Rollbar-instance) scope of the telemetry store.

Extended reasoning...

Overview

This PR adds a new rollbar-java-agent module: a -javaagent that uses ByteBuddy to instrument HttpURLConnection, java.net.http.HttpClient, and Apache HttpClient 4.x/5.x, recording 4xx/5xx responses and connection failures as Rollbar telemetry events. It includes URL sanitization, WeakHashMap-based dedup for re-entrant calls, a shaded build with only ByteBuddy relocated, and WireMock-backed integration tests for each client.

Security risks

The main security-relevant surface is UrlSanitizer, which strips userinfo/credentials, query params, and fragments before URLs are recorded as telemetry. Prior review rounds found and fixed real gaps here (encoded '@' in userinfo, unescaped-space fallback paths, underscore hostnames) and those fixes are present in the current code with dedicated regression tests. No injection, auth, or crypto surface is introduced elsewhere.

Level of scrutiny

This warrants more than a mechanical-change level of review: it's a brand-new module built on bytecode instrumentation and reflection across classloader boundaries, which is inherently subtle (as shown by the volume of substantive findings across this PR's review history — advice type resolution across classloaders, redefinition/ignore-matcher semantics, re-entrant call dedup, JDK/ByteBuddy version compatibility). It doesn't touch any existing production code path in rollbar-java itself, which lowers blast radius, but the agent module's own correctness is nontrivial.

Other factors

Comparing against the PR's history, every previously reported P1/blocking issue from both human review (brianr) and prior automated runs (JVM version support, shading the Rollbar SDK unrelocated, the zero-code-change claim, encoded-credential stripping, Apache advice NoClassDefFoundError risk, ignore-matcher dropping isSynthetic, connection-failure dedup) has a corresponding follow-up commit in the current code. The remaining open items visible in the timeline are nit-severity code-duplication suggestions, not correctness bugs. Test coverage is substantial (WireMock integration tests per client, dedicated URL sanitizer and composeUrl unit tests). Given the module is new and the domain is unusually easy to get subtly wrong, a human sign-off is still the safer default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants