Conversation
Akatuoro
left a comment
Akatuoro
left a comment
There was a problem hiding this comment.
comments + remove the goallab-prediction-agent
| ```js | ||
| import cds from "@sap/cds" | ||
| import { test } from "vitest" | ||
| import { Judge, TrajectoryJudge, ConverstationJudge, matchToolCall } from "@cap-js/agents" |
There was a problem hiding this comment.
| import { Judge, TrajectoryJudge, ConverstationJudge, matchToolCall } from "@cap-js/agents" | |
| import { Judge, ConversationJudge, matchToolCall } from "@cap-js/agents/eval" |
Only Judge, no others
There was a problem hiding this comment.
I adjusted the structure and the initial sample just includes the Judge now but the other two are still mentioned as they are still relevant.
| }) | ||
| ``` | ||
|
|
||
| ## Eval run lifecycle |
There was a problem hiding this comment.
remove everything from here on, or drastically reduce it. Contains a lot of details which are likely to change, which simply leads to outdated docs
There was a problem hiding this comment.
I adjusted the structure and minimized it but fewer details are I believe not possible as else the docs loose their value.
|
|
||
| /** Install srv.chat(query, contextId?, opts?) on an @agent service. Called from cds.on("serving"). */ | ||
| export function registerChat(srv) { | ||
| srv.chat = async function chat(query, second, third) { |
There was a problem hiding this comment.
reduce options as discussed yesterday. Also move this to lib/eval, the current implementation has a lot of details on the result object that are wanted in an eval context, but do not belong in a general api: metrics, toolCalls, query, additional messages for evaluating the trajectory....
There was a problem hiding this comment.
Sorry for overlooking. I removed the options but I left it in chat.js as the other options are already behind a profile flag just for testing.
3 participants
Add evaluation helpers to easily test agents.
Integrated with MLFlow to view results.
Have you...