benchmarking
Everything on Ground Truth tagged “benchmarking” — 3 items.
Inter-rater reliability: when two graders disagree Lesson
Inter-rater reliability measures how consistently graders apply an evaluation, and Cohen’s kappa adjusts agreement for label frequencies.
TraceDance turns real agent deployment traces into tests for risky next decisions News
TraceDance proposes extracting one-turn evaluations from real agent traces, finding a 26.7% mean pass rate across nine models on a selected safety stress set rather than a deployment-prevalence estimate.
Artificial Analysis Tool
A free public dashboard that independently benchmarks and compares AI models on a combined intelligence index alongside price and speed; the source of this week's finding that GLM-5.2 leads the open-weight class.