PERSON DIRECTORY
Olivia Watkins
Olivia Watkins appears in 1 indexed conversation across Latent Space. This directory brings every appearance, source, TL;DR, digest, and transcript into one searchable feed.
1 EPISODE1 SHOW
Language
The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals
SWE-bench Verified is effectively saturated and highly contaminated, making tiny gains unreliable despite OpenAI’s costly curation of roughly 500 real-world tasks.SWE-bench Pro offers harder, longer, more diverse work with less contamination evidence, shifting evaluation toward maintainability, expert judgment, and real-world augmentation rather than narrow test passing.