Automated monitoring of your agents behavior in production.

Traffic split metrics → DataFrame (evals vs errors) → push update

import agent_service as agents_api
import pandas as pd

client = agents_api.get_client("api_key")
agent_id = "support-agent"

# Current performance under the live traffic split (per arm: evals, errors, latency, …)
split_perf = client.get_traffic_split_performance(agent_id=agent_id)

df = pd.DataFrame(split_perf.rows)

# Custom analysis: compare eval signals vs errors across variants (slice, plot, regressions, …)
analysis = df.groupby("variant").agg({"eval_score": "mean", "error_count": "sum"})

# Push the workflow update you derived from that analysis
client.push_workflow_update(agent_id=agent_id, workflow=workflow_from_analysis(analysis))

Traffic split metrics → DataFrame (evals vs errors) → push update

import * as agentsApi from "agents-api";

(async () => {
  const client = agentsApi.getClient("api_key");
  const agentId = "support-agent";

  const splitPerf = await client.getTrafficSplitPerformance({ agentId });

  // Into a DataFrame / table: group by variant — eval vs error rates (your custom analysis)
  // df.groupby("variant").agg({ eval_score: "mean", error_count: "sum" })

  await client.pushWorkflowUpdate({
    agentId,
    workflow: workflowFromAnalysis(splitPerf),
  });
})();