Repository navigation
AsyncExceptionHandlerInterceptor lets a downstream grpc.RpcError through: the caller gets UNKNOWN with the other service's details #48
Description
Versions: grpc-server-kit 0.2.0, grpcio 1.84.0, Python 3.13.
A handler that calls another gRPC service and lets that call's error escape leaks the other service's status details to its own caller. AsyncExceptionHandlerInterceptor re-raises every grpc.RpcError untouched (aio/interceptors/exception_handler.py:145), and grpc.aio.AioRpcError, which an outbound call raises, is a grpc.RpcError. grpc.aio then answers the caller with UNKNOWN and the repr of the exception.
Reproduction
import asyncio
import logging
import grpc
from grpc_server_kit import GrpcApp, GrpcServerConfig
from grpc_server_kit.aio.interceptors import AsyncExceptionHandlerInterceptor
def generic(service, method, handler):
return lambda server: server.add_generic_rpc_handlers((
grpc.method_handlers_generic_handler(service, {method: grpc.unary_unary_rpc_method_handler(handler)}),
))
async def main():
async def ledger(request, context):
await context.abort(grpc.StatusCode.INTERNAL, "ledger dsn=postgres://ledger:s3cret@10.0.3.7/ledger")
downstream = GrpcApp(GrpcServerConfig(host="127.0.0.1", port=0))
downstream.register(generic("billing.Ledger", "Get", ledger))
async with downstream:
target = f"127.0.0.1:{downstream.bound_port}"
async def invoice(request, context):
async with grpc.aio.insecure_channel(target) as channel:
await channel.unary_unary("/billing.Ledger/Get")(b"", timeout=2)
return b"ok"
app = GrpcApp(
GrpcServerConfig(host="127.0.0.1", port=0),
interceptors=[AsyncExceptionHandlerInterceptor()],
)
app.register(generic("orders.Invoices", "GetInvoice", invoice))
async with app:
async with grpc.aio.insecure_channel(f"127.0.0.1:{app.bound_port}") as channel:
try:
await channel.unary_unary("/orders.Invoices/GetInvoice")(b"", timeout=3)
except grpc.aio.AioRpcError as error:
print("status: ", error.code().name)
print("details:", error.details())
logging.basicConfig(level=logging.CRITICAL)
asyncio.run(main())Actual
status: UNKNOWN
details: Unexpected <class 'grpc.aio._call.AioRpcError'>: <AioRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "ledger dsn=postgres://ledger:s3cret@10.0.3.7/ledger"
debug_error_string = "INTERNAL:ledger dsn=postgres://ledger:s3cret@10.0.3.7/ledger"
>
Expected
An AioRpcError from an outbound call is an unexpected exception for this server. It should go through the same path as any other exception: find_mapped_status (so INTERNAL with the default map), details from detail_factory, and a log record with the traceback. Only grpc.aio.AbortError, a status this RPC deliberately chose, should pass through untouched. The comment above the pass-through says "already aborted this RPC with its own status", which holds for AbortError but not for a client-side AioRpcError.
Possible fix
Remove the except grpc.RpcError: raise clause (keep the AbortError one) and add a test where the handler awaits a failing downstream RPC. If propagating the downstream code is wanted (UNAVAILABLE, DEADLINE_EXCEEDED), it can go through error_status_map with safe details rather than the raw repr.
servicewright's UnhandledErrorInterceptor has the same pass-through: bedrock-python/servicewright#86.
Activity
AlexeyShalaev commentedon Oct 16, 2026
Reproduced on master with the script above:
UNKNOWNplus the downstreamAioRpcErrorrepr in the details.The
except grpc.RpcError: raiseclause never catches anything this server decided.grpc.aio.AbortErroris not agrpc.RpcError(its MRO isAbortError -> BaseError -> Exception), and the onlyRpcErrorsubclass grpcio defines is the client-sideAioRpcError. So the clause only ever lets through failures of calls this server made, and grpc.aio answers those withUNKNOWNand the repr.Plan:
AbortErrorone. An outboundRpcErrorthen goes the same way as any other exception:find_mapped_status(INTERNALwith the default map, sinceRpcErrorisn't in it), details fromdetail_factory("Internal server error" by default),logger.exceptionwith the traceback.UNAVAILABLEcan putgrpc.RpcError(orgrpc.aio.AioRpcError) inerror_status_map. Documenting that, no new option.INTERNAL/ "Internal server error" and that the downstream details are not in the response. Both fail on master.docs/agents.md.Considered and not doing:
INVALID_ARGUMENTorUNAUTHENTICATEDmeans we sent a bad request or have bad credentials, and passing that on blames the caller.INTERNALis the honest default; services that know better translate in the handler or through the map.AsyncSentryInterceptor, which also skipsgrpc.RpcError. That's a separate question (it would start reporting every downstream outage as a new event), and the failure is already logged at ERROR with the traceback by the exception handler. Leaving Sentry as it is.Compatibility: the only response that changes is the one that was
UNKNOWNwith the downstream repr; it becomesINTERNALwith the safe detail. Nobody could have relied on the old status carrying information, since grpc.aio always answeredUNKNOWNfor these. Metrics, logging and tracing interceptors outside the handler now recordINTERNAL, which is what the client actually gets (before they recorded the downstream code while the client gotUNKNOWN). Shipping it as afix:.servicewright#86 is the same bug in
UnhandledErrorInterceptor; this fix keeps the two libraries in line (outbound failure -> maskedINTERNALand logged, deliberate abort -> passed through).AlexeyShalaev commentedon Oct 16, 2026
Fix is in #50, as planned. With it your script prints:
and the downstream status and details go to the log with the traceback. A deliberate
context.abort()still passes through unchanged. Nothing to change on your side; if you want downstream failures reported as something other thanINTERNAL, addgrpc.RpcError: <status>toerror_status_map(the interceptors guide has an example). Sentry is unchanged: it still doesn't capture outboundRpcErrors, the handler's ERROR log covers them.fix: map a failed outbound RPC instead of passing its status to the c…