Callback Enhancement: Include intermediate responses

[Prajna1999]

Is your feature request related to a problem?
The current callback response only contains the final provider response, omitting important intermediate inputs and validation results. This limits visibility into the API's processing steps and effectiveness.

Describe the solution you'd like
Enhance the callback response to include:

• User’s input (post-input guardrail)
• Input to LLM (after input guardrail)
• Output from LLM
• response to user(after output guardrail)

Add a flag to enable detaled metadata

Original issue

Describe the current behavior
Consider a simple llm_call with a guardrail config.

POST api/v1/llm/call

{
  "query": {
    "input": "Amit Gupta phone number is 919611188278"
  },
  "config": {
    "blob": {
      "completion": {
        "provider": "openai",
        "type": "text",
        "params": { "model": "gpt-5.6-sol" }
      },
      "input_guardrails": [
        { "validator_config_id": "d9e3c944-5718-44d2-b1b6-b52bfba7ee33" } //PII_remover
      ],
      "output_guardrails": []
    }
  },
  "callback_url": "https://webhooksite.net/ffda71fe-ece9-4871-a47e-4949753a0748",
  "include_provider_raw_response": false
}

the final response sent to the user via the callback is

  "success": true,
  "data": {
    "response": {
      "provider_response_id": "resp_0860bf7f27cab00b006a9fcc7fd3b087d1a5c844f3d614176a",
      "conversation_id": null,
      "provider": "openai-native",
      "model": "gpt-5.6-sol",
      "output": {
        "type": "text",
        "content": {
          "format": "text",
          **"value": "Got it. What would you like me to do with that phone number?",**
          "language_code": null
        }
      }
    },
    "usage": {
      "input_tokens": 16,
      "output_tokens": 68,
      "total_tokens": 84,
      "reasoning_tokens": null
    },
    "provider_raw_response": null
  },
  "error": null,
  "errors": null,
  "metadata": {
    "warnings": []
  }
}

The response sent back to the caller only contains final provider response. No intermediate input/output is sent across to the api caller. e.g

What gets input to lllm call post stripping of PII information is
<PERSON> phone number is <PHONE_NUMBER>

In a similar manner, for chained validator setup, we store intermediate responses in database but do not send them with the response payload. Ditto for output guardrails. We need to add four fields in callback response

• User’s input (post-input guardrail)
• Input to LLM (after input guardrail)
• Output from LLM
• response to user(after output guardrail)