← work scientific ML · observed negative result

The point prediction worked. The confidence estimate did not.

A pretrained materials model ranked stable candidates efficiently on a static screening task, while the tested MC-dropout uncertainty signal failed the calibration test needed to justify uncertainty-aware acquisition.

Materials screening result showing roughly five times discovery acceleration from mean ranking and a negative correlation between MC-dropout uncertainty and absolute error.
Observed negative result. The mean prediction is useful on this proxy screen; the tested uncertainty score is not evidence of knowing when the model is wrong.
Problem

Active learning usually assumes the acquisition function can trade off predicted value and model uncertainty. That only makes sense if the uncertainty score carries information about error.

Result

Greedy mean ranking reached DAF 5.15 and low-weight UCB 5.17 at the tested 5% budget, while MC-dropout σ had Spearman −0.47 correlation with absolute error.

Independent evidence

The public benchmark runs five seeds over 18,928 perovskite structures and includes direct calibration diagnostics: σ↔error association, reliability, miscalibration area, and empirical coverage.

Boundary

This is a static labeled proxy benchmark, not a closed-loop DFT or laboratory discovery campaign. The result is specific to the tested pretrained model, readout-level MC-dropout configuration, dataset, and acquisition setup.

The dose response

RandomDAF ≈ 1.01
Greedy μDAF 5.15 ± 0.04
UCB λ=0.5DAF 5.17 ± 0.01
UCB λ=5DAF 0.91 — below random as the bad uncertainty signal receives too much authority

Verification path