Can agents read dental X-rays?

Leaderboard

Mean score, 200 films. Error bars: 95% interval.

Post-training

fine-tuned + RL shuffled labels

Test · 387 films

fine-tuned0.12
+ RL0.18

Method

Task
Box each lesion, grade it PAI 3–5.
Score
½ lesion F1 + ½ graded F1, IoU ≥ 0.3
Agents
Codex (GPT-6 Astra), Claude Code (others), 50 turns
Films
First 200 of 387 test films, one try
Prompt
Task and answer format in the prompt; the answer is the agent's final reply.
Dentists
0.83: the source dentists annotated 59 films twice; the two annotations scored against each other (Do et al. 2024).