Can a Machine Know When It Is Unsure?

CALIBRATION

Neural networks are very good at producing probabilities. This is not the same as being good at knowing how uncertain they are. A classifier can say 97% with impressive typography and be wrong far more than 3% of the time.

Calibration asks whether confidence matches frequency. Among a large set of predictions made at about 80% confidence, roughly 80% should be correct. A model can have excellent classification accuracy and poor calibration because accuracy asks whether the top answer was right while calibration asks whether the number attached to that answer deserved to be believed.

Guo and colleagues showed in 2017 that modern neural networks can be systematically miscalibrated even when their predictive accuracy is strong, and that a simple post-processing method called temperature scaling can often improve calibration without changing which class wins. The probability layer has its own failure mode. It is not automatically repaired by a better classifier.

ALEATORIC AND EPISTEMIC

Two different problems can both look like doubt

Aleatoric uncertainty comes from variability or ambiguity in the observation and system itself. Cloud obscures the surface. A class boundary cuts through one pixel. Rainfall varies sharply over short distances. Even a perfect model may not be able to recover information the measurement never contained.

Epistemic uncertainty comes from limited knowledge. The model has seen few examples of this landscape, sensor condition or event. Its parameters are poorly constrained here, or the deployment lies outside the part of the world represented in training. More representative evidence can, at least in principle, reduce this kind of uncertainty.

The distinction matters because the response should differ. If a scene is physically ambiguous, another copy of the same model may not help. A different sensor, finer observation or later acquisition might. If the model is outside its experience, a new label or a specialist model may be more useful. Uncertainty is valuable only if it tells the system what kind of ignorance it has.

Production systems often compress both into one confidence score because interfaces prefer one number. That makes integration easier and reasoning harder. Two 60% predictions can imply completely different next actions if one is uncertain because the scene is obscured and the other because the model has never seen anything like it.

UNCERTAINTY QUANTIFICATION

There are several ways to estimate doubt, and none is the doubt meter

Deep ensembles train several neural networks independently and use the variation among their predictions as information about model uncertainty. Lakshminarayanan and colleagues showed that this simple approach can produce strong uncertainty estimates without requiring a full Bayesian treatment of a neural network.

Bayesian approximations try to represent uncertainty over model parameters more directly. Monte Carlo dropout and related methods run multiple stochastic forward passes. Other systems learn explicit predictive distributions. The methods differ in computation and assumptions, but all are attempts to keep more information than one point estimate.

Conformal prediction takes a different route. Given a calibration set and an exchangeability assumption, it can wrap an existing model with prediction sets or intervals that achieve a target coverage rate in finite samples. A 90% conformal set is designed so that the true answer falls inside the set about 90% of the time under the validity conditions. The model does not become more correct. The output becomes more honest about the range of answers still plausible.

The conditions matter. Distribution shift can weaken ordinary calibration and conformal guarantees. Ensembles can agree because their members share the same biased training data. Bayesian-looking numbers can still be poorly specified. There is no method that inspects an arbitrary prediction and announces objective uncertainty independent of how the data were generated.

DISTRIBUTION SHIFT

Confidence is most fragile exactly where the world becomes unfamiliar

Calibration is usually measured on held-out data drawn from roughly the same process as training. Deployment is where the interesting geography begins. A model crosses into a different climate regime, a new sensor version, an unusual season, an extreme event or a landscape type it saw rarely. The confidence score can remain numerically precise while its empirical meaning changes.

This is especially important for foundation models. Pretraining broadens experience, but broad is not universal. Fine-tuning can also make calibration worse because a small local dataset changes the prediction head and representation in ways that improve task accuracy without preserving the original uncertainty behaviour.

Weather provides a useful operational reminder. Learned ensembles can have excellent average skill and still show region-specific or variable-specific reliability problems. ECMWF has documented cases where AIFS ENS was overdispersive for some upper-air variables and showed particular biases in precipitation, cloud and high-altitude surface temperature. An ensemble is a mechanism for expressing uncertainty, not proof that the expressed uncertainty is correct.

A planetary system therefore needs to monitor calibration continuously. Confidence should be checked by geography, season, event type, sensor condition and lead time rather than collapsed into one global score. Doubt, like almost everything else in this book, has a map.

ABSTENTION AND RETASKING

The useful response to uncertainty is sometimes not an answer

Uncertainty becomes operational when the system is allowed to do something with it. Selective prediction permits a classifier to abstain on low-confidence cases instead of forcing a label. A forecast can widen an interval. A fusion system can request another sensor. An autonomous spacecraft can spend its next observation on the place where the current evidence is least adequate.

This creates a trade-off between coverage and reliability. Abstaining more often can make the retained predictions more accurate, but a system that refuses every difficult case is not useful. The threshold should depend on consequence. A low-stakes map preview and an alert that closes a port should not require the same confidence.

The difficult part is preserving uncertainty across systems. If a flood model outputs a distribution and an integration layer converts it into a binary flag, the next model cannot reconstruct what was lost. The uncertainty has been laundered into certainty by a software interface rather than by evidence.

That makes uncertainty part of state, not decoration around a result. A planetary intelligence needs to know what it believes, how strongly it believes it, why the belief has that strength, and what observation would most efficiently change it. Only then can the loop from prediction back to observation work without confidence becoming self-fulfilling.

Knowing what to observe next begins with knowing which part of the current answer deserves the least trust.