fix: redact provider credentials from logs and API responses

[guofeng201507]

What changed and why

Provider API keys could end up in application logs — and in one path in the API
response — because the key was carried in the request URL and requests embeds
the URL inside its exception messages. The most common trigger is an ordinary
failure: an invalid/expired key (HTTP 403) or a network timeout, i.e. exactly
when someone is troubleshooting.

backend_api_python/app/services/llm.py — LLMService._call_google_gemini()

• before: url = f"{base_url}/models/{model}:generateContent?key={api_key}"
• after: the key is sent in the documented x-goog-api-key request header

backend_api_python/app/utils/redaction.py (new)

• redact_secrets() scrubs key=, api_key=, token=, secret=,
  password= and Bearer <token> from a string, keeping the host/path so
  failures stay debuggable.

backend_api_python/app/services/llm.py

• every exception-derived string now passes through redact_secrets() before
  being logged, re-raised, or written into the returned report field: the HTTP
  error, request error, invalid-data, aggregate error, alternative-provider
  fallback, and the _llm_post connection error paths.

backend_api_python/app/routes/settings.py — test_connection()

• the exception text was both logged and returned to the client as msg, so the
  Finnhub key (...?symbol=AAPL&token=...) could be displayed in the UI. Both
  places are now scrubbed.

How to test

Manual: configure the Google provider with a key, make the call fail (invalid
key → 403, unreachable GOOGLE_BASE_URL, or block
generativelanguage.googleapis.com), trigger any AI feature, then check the
backend log. Before: ...:generateContent?key=AIza...; after:
...?key=<redacted>.

Automated:

pytest tests/test_llm_litellm_provider.py
# 37 passed

Includes 3 new regression tests: the Gemini key is absent from the URL and
present in the header; the scrubber removes query-parameter secrets and bearer
tokens; harmless text passes through unchanged. Run on CPython 3.12 using the
project's own backend image. The 34 pre-existing tests are unchanged.

Backward compatibility

The Gemini change relies on the official x-goog-api-key header, which the
Gemini REST API documents. A Gemini-compatible proxy that only accepts ?key=
would need updating.

Notes

Filed as a normal fix rather than a private vulnerability report — happy to
re-route through the channel described in SECURITY.md if you would prefer that.

[likangjie110]

Applied this on current main and ran the suite — the Gemini key really is only in the query string (llm.py, url = f"{base_url}/models/{model}:generateContent?key={api_key}", with headers = {"Content-Type": "application/json"} and no auth header), and the scrubber behaves as documented. One gap means the "and API responses" half of the title is not closed yet.

Two bare raise statements still hand the caller the unredacted exception. In call_llm_api(), the HTTP and request exception handlers log the scrubbed text but then do:

if not use_fallback or current_model == models_to_try[-1]:
    raise            # re-raises the original exception, key included

With use_fallback=False — which is exactly how app/services/ai_decision.py, ai_authoring_intent.py and research_agent.py call it — the original HTTPError/RequestException propagates with its message intact. Routes echo that to clients: app/routes/ai_chat.py ends its handler with return jsonify({"code": 0, "msg": str(e), "data": None}), 500, so the key reaches the browser rather than only the log. I reproduced it with a provider error carrying ?key=... in its URL: the log line came out as ...?key=<redacted> while the raised message still contained the key.

The minimal fix is to re-raise the same type with a scrubbed message, keeping response/request attached so callers' except clauses and status handling are unaffected — I used a small helper (_scrubbed_request_error) at both sites.

Two smaller notes:

• The Google change is the right direction, and it matches what fix: repair LLM test-connection and make silent model/template fallbacks visible #281 does for its probe (credential in x-goog-api-key, never in the URL). Worth stating the proxy caveat in the PR body if you have not already: a Gemini-compatible proxy behind a custom GOOGLE_BASE_URL that only accepts ?key= would break.
• This adds the third scrubber in the repo: app/utils/redaction.py, plus _public_request_error_message() in app/data_providers/economic_calendar.py and _redact_sensitive_text() in app/utils/safe_exec.py. All three are pattern- or value-based and none is shared. Not a blocker for this PR, but consolidating them would prevent the next credential-leak fix from picking a fourth shape.

What I did with it: applied on main plus the re-raise fix, the two raise sites the patch did not cover (the LLMAPIError raised on the LiteLLM path and the OpenRouter compatibility message), a settings-endpoint call site, and tests — the scrubbed-message case fails against the unpatched code.