Feedback is the beginning. Improvement needs evidence.
Vocion now surfaces learning assets alongside workforce activity. Here is how to use them without confusing more feedback or more evaluations with better performance.
More feedback can mean that people are engaged. It can also mean that the same defect keeps recurring. A larger evaluation dataset can improve coverage, or simply repeat cases you already had.
The useful question is not how much learning material an agent has accumulated. It is whether that material helps the team make a change and verify a better result.
Vocion's workforce measurement release puts learning evidence beside recorded activity and outcomes, so operators can examine that connection.
See what is available to learn from
The workforce view exposes evaluation datasets, distinct cases within each dataset, cases with reference answers, agent coverage and dataset freshness. It also shows feedback received through the feedback pipeline and its processing state, alongside approved and pending learning candidates.
These measures answer practical questions. Are there examples for this agent's workflow? Is feedback stuck in processing? Was an approved learning followed by a completed evaluation? Where is coverage missing?
They do not produce a universal “agent intelligence” score. Cases may overlap across datasets. Feedback coverage does not include every comment made anywhere in the product. A completed evaluation is not automatically a pass.
A reference answer is not automatically a gold standard
A trusted reference needs provenance: who reviewed it, against which rubric, on which source version, and for which task. Independent holdout cases also need to remain distinct from material used to tune the agent.
Until that evidence exists, a reference answer should be called a reference answer. Reviewed gold standards, holdout coverage and validated improvement remain unverified where the required provenance is absent.
The same care applies to a pass rate. Vocion only displays eligible recent results when the dataset version, completed run and recorded configuration meet its freshness rules. A newer failed run must not be disguised by showing an older successful result instead.
Make the weekly review a decision
Choose one recurring problem and examine representative outputs. Then decide what kind of intervention the evidence supports:
- Routing or use: the right tasks may never reach the agent. Check eligible demand and expected cadence before changing its prompt.
- Context: outputs may miss a necessary fact. Inspect source access and freshness before adding more documents.
- Playbooks: the same judgment may be corrected repeatedly. Make the rule explicit and test the boundary cases.
- Review: work may wait on a decision. Separate queue time from the human effort needed to review it.
- Evaluation: the team may lack examples that expose the failure. Add relevant cases and keep an independent check.
These are operating practices, not automatic diagnoses performed by the dashboard. Its investigation prompts help locate evidence; a person owns the intervention.
Close the loop on comparable work
Record the hypothesis, the change and the configuration version. Compare results on equivalent cases with the same rubric and grader. Check for regressions, then follow the outcome in production. Decide whether to keep, revise or revert the change.
A higher pass rate by itself does not establish causation. An approved learning by itself does not establish improvement. The discipline is to keep the claim no stronger than the evidence.
That is the purpose of measuring the learning stock: to make the next improvement more deliberate, and its result easier to assess.
Scope and next steps
The learning evidence contract documents the implemented measures and remaining provenance gaps. A full validated-improvement workflow remains further work. Start with the workforce dashboard release, then choose one workflow to investigate this week.