Find the term.
See how it fits.
Explore the concepts behind AI agents, one useful connection at a time.
Concept directory
Benchmark
A benchmark is a defined set of tasks and scoring conventions used to compare performance. An AI benchmark measures selected capabilities under its particular test conditions.
Evaluation
Evaluation is checking an AI system against defined tasks and success criteria. An evaluation may score an answer, inspect actions, or test whether the resulting environment matches the intended outcome.
Evaluation Harness
An evaluation harness is software that runs evaluation tasks against models or systems and collects results. It connects task definitions, model execution, and scoring into a repeatable testing process.
LLM-as-a-Judge
LLM-as-a-judge uses a language model to assess outputs against instructions or a rubric. It can produce scores, comparisons, or written judgments to support evaluation.
Trace
A trace records the operations involved in an execution so their sequence and relationships can be inspected. In distributed tracing, individual operations are represented as spans with timing and other recorded attributes.