Issues

• Cost is one hardcoded rate per provider, applied to every model and both token directions

  bugenhancementhelp wanted
  AgentPostmortem/Evalgate#16 · Filter by author royalpinto007 opened on Aug 6, 2026

• Provider calls have no timeout and no retry, so a 429 fails the build

  bugenhancementhelp wanted
  AgentPostmortem/Evalgate#15 · Filter by author royalpinto007 opened on Aug 6, 2026

• One provider or scorer error discards the entire run

  bugenhancementhelp wanted
  AgentPostmortem/Evalgate#14 · Filter by author royalpinto007 opened on Aug 6, 2026

• Deleting a failing case makes the gate green: removed cases are never regressions

  bugenhancementhelp wanted
  AgentPostmortem/Evalgate#13 · Filter by author royalpinto007 opened on Aug 6, 2026