Brier Score Explained for Football

What a Brier score measures, how to read one, and why it judges a football prediction model more honestly than counting how many results it called.

· Football Data Lab

Brier Score Explained

The Brier Score is one of the most useful tools in football analytics because it asks a simple but demanding question: were your probabilities good?

That question matters because modern football prediction is not really about saying “Team A will win”. It is about estimating uncertainty. A serious model should say something like: the home team has a 46% chance of winning, the draw is 28%, and the away team is 26%. Those probabilities can then be used in match prediction, betting analysis, tournament simulation, squad planning, or performance evaluation.

But probabilities need to be judged differently from ordinary predictions. If a model gives Liverpool a 70% chance of beating a relegation-threatened side and Liverpool draw, the model was not automatically “wrong”. A 70% event still fails three times in ten. Equally, if a model gives a team a 51% chance and that team wins, the forecast was not necessarily impressive.

The Brier Score helps solve this problem. It evaluates how close predicted probabilities are to actual outcomes, rewarding models that assign high probabilities to events that happen and penalising models that are confidently wrong. In football prediction modelling, it is widely used because it is intuitive, mathematically clean, and directly connected to calibration.

What Is the Brier Score?

The Brier Score measures the accuracy of probability forecasts. For a binary event, it is calculated as:

Brier Score = (predicted probability − actual outcome)²

The actual outcome is coded as either:

So if a model predicts a 70% chance of a home win, and the home team wins, the Brier Score for that event is:

(0.70 − 1)² = 0.09

If the home team does not win, the score is:

(0.70 − 0)² = 0.49

Lower is better. A perfect forecast receives a Brier Score of 0. A very poor confident forecast receives a high score.

This is why the Brier Score is especially useful in football. It does not merely ask whether the most likely outcome occurred. It asks whether the probability assigned to each outcome was sensible.

A model that says a team has a 55% chance of winning is making a much softer claim than a model that says the same team has an 85% chance. The Brier Score recognises that difference.

Why Football Forecasts Need Probability Evaluation

Football is a low-scoring sport with a high level of randomness. Even strong teams draw or lose matches they dominate. Red cards, finishing variance, goalkeeper performance, tactical matchups, weather, rotation, and game state can all disrupt expectations.

Because of this, evaluating football predictions using only win/loss accuracy is crude.

Suppose two models predict a home win:

The home team wins. If we use simple accuracy, both models get equal credit. But they did not make the same forecast. Model B was far more confident. If the home team had lost, Model B would deserve a much larger penalty.

The Brier Score captures this distinction. It evaluates the quality of the probability, not just the selected outcome.

This makes it particularly valuable for:

In football analytics, a model is not valuable because it occasionally predicts surprising results. It is valuable if, over many matches, its probabilities correspond well to reality.

Binary Brier Score in Football

The simplest use of the Brier Score is for binary markets or events. These include:

Consider an over 2.5 goals prediction.

A model estimates:

If the match finishes 2–1, the event happens. The Brier Score is:

(0.62 − 1)² = 0.1444

If the match finishes 1–1, the event does not happen. The Brier Score is:

(0.62 − 0)² = 0.3844

The penalty is larger when the model assigns a high probability to something that fails to occur.

Now compare that with a more cautious forecast of 0.52 for over 2.5 goals. If the match finishes under, the Brier Score is:

(0.52 − 0)² = 0.2704

That is still a penalty, but not as severe. The model was less confident, so it is punished less.

This is one reason the Brier Score discourages false certainty. In football prediction, being slightly uncertain is often more realistic than being aggressively confident.

Brier Score for 1X2 Football Predictions

Football match outcomes are usually not binary. The standard 1X2 market has three mutually exclusive outcomes:

To apply the Brier Score to this format, we compare the predicted probability for each outcome with what actually happened.

For example, suppose a model predicts:

OutcomePredicted probabilityActual result
Home win0.501
Draw0.280
Away win0.220

The home team wins. The multi-class Brier Score is calculated by summing the squared errors:

(0.50 − 1)² + (0.28 − 0)² + (0.22 − 0)²

That equals:

0.25 + 0.0784 + 0.0484 = 0.3768

Some analysts report this directly. Others divide by the number of classes, giving an average squared error per outcome. The important point is consistency: when comparing models, the same version of the Brier Score must be used.

Now consider a more confident forecast:

OutcomePredicted probabilityActual result
Home win0.751
Draw0.160
Away win0.090

The home team wins, so the score is:

(0.75 − 1)² + (0.16 − 0)² + (0.09 − 0)²

That equals:

0.0625 + 0.0256 + 0.0081 = 0.0962

This is much better. The model strongly backed the correct outcome.

But confidence cuts both ways. If the away team wins instead, the same 0.75 home-win forecast becomes heavily penalised:

(0.75 − 0)² + (0.16 − 0)² + (0.09 − 1)²

That equals:

0.5625 + 0.0256 + 0.8281 = 1.4162

The Brier Score therefore rewards justified confidence and punishes misplaced certainty.

What Makes a Good Brier Score?

There is no universal “good” Brier Score in football because the answer depends on the forecasting task.

A model predicting whether a heavy favourite avoids defeat is dealing with a different problem from a model predicting exact match outcomes. Binary markets, three-way outcomes, player props, and scoreline forecasts all have different difficulty levels and different score ranges.

A Brier Score is most useful when used comparatively:

The score should also be compared against simple baselines. For example, a model may be compared with:

A Brier Score alone is just a number. Its value comes from context.

Brier Score, Calibration and Sharpness

The Brier Score is powerful because it reflects two important qualities of a probability model: calibration and sharpness.

Calibration

A model is well calibrated if events predicted at a certain probability happen at roughly that frequency over time.

For example, if a football model assigns 60% win probability to a group of teams across many matches, those teams should win around 60% of the time. Not every match, of course, but across a large sample.

If teams given a 60% chance only win much less often, the model is overconfident. If they win much more often, the model is underconfident.

Calibration is central to football forecasting because probabilities are meant to describe reality, not just rank teams.

Sharpness

Sharpness refers to how far probabilities move away from the average. A model that always predicts 33%, 34%, 33% for home, draw, and away may be reasonably cautious, but it is not very informative.

A sharper model is willing to say that one team has a 65% chance when the evidence supports it. Sharpness is desirable only when calibration is maintained.

The Brier Score balances these two forces. A model can improve by becoming more confident when it is right, but it will be punished if that confidence is not supported by actual outcomes.

This is why Brier Score evaluation is more subtle than simple accuracy. A model that gets many favourites “right” may still be poorly calibrated if it consistently gives them probabilities that are too high.

Brier Score vs Log Loss

The Brier Score is often discussed alongside log loss. Both evaluate probability forecasts, but they behave differently.

The Brier Score uses squared error. It penalises wrong probabilities smoothly. Log loss uses logarithms and punishes extreme wrong forecasts much more severely.

For example, if a model gives an event a 1% probability and it happens, log loss applies a very large penalty. The Brier Score also penalises the model, but less explosively.

In football analytics, both metrics are useful.

The Brier Score is often easier to interpret because it is based on squared probability error. It is intuitive for calibration analysis and practical model comparison. Log loss is especially sensitive to extreme overconfidence, making it useful when avoiding catastrophic probability errors is a priority.

A good validation process should not rely on one metric alone. If a model improves on Brier Score but worsens on log loss, that may indicate it is performing better on average while becoming dangerously overconfident in a few cases. If it improves on both, the evidence for improvement is stronger.

How Football Data Lab Applies the Brier Score

Football Data Lab uses the Brier Score as part of its probability validation process, not as a decorative end-of-season number. The aim is to understand whether forecast probabilities behave correctly across the full modelling pipeline.

The process begins with pre-match probabilities. For a match, the model may produce a 1X2 distribution such as:

Those probabilities are generated before the match and stored with the model version, competition, date, team identifiers, and relevant contextual inputs. Once the final result is known, the realised outcome is converted into an outcome vector. If the home team wins, the actual vector is:

The Brier Score is then calculated across the three outcomes. In this simplified example:

(0.48 − 1)² + (0.27 − 0)² + (0.25 − 0)²

That produces a match-level probability error for the 1X2 forecast.

The same principle is applied to binary outputs such as both teams to score, over/under goal lines, and clean-sheet probabilities. This allows Football Data Lab to inspect whether the model is generally well calibrated or whether specific market types behave differently.

A key part of the methodology is segmentation. The aggregate Brier Score tells us whether one model version outperforms another overall, but it does not explain why. Therefore, forecasts are grouped into diagnostic bands and football-specific categories, such as:

This is where the Brier Score becomes genuinely useful. Suppose a model’s overall score improves, but the segmented validation shows worse performance on draws. That suggests the headline improvement may be coming from better favourite identification while draw probabilities have deteriorated. In football, that distinction matters because draw pricing is one of the hardest parts of 1X2 modelling.

Football Data Lab also uses Brier Score analysis alongside calibration curves. If teams assigned win probabilities between 50% and 55% win at roughly that rate over a sufficiently large sample, the model is behaving sensibly in that region. If the observed win rate is consistently lower, the model may be too aggressive. If it is higher, the model may be too conservative.

The Brier Score then acts as both a model comparison metric and a model debugging tool. It helps answer questions such as:

This matters because a football prediction model can appear better after a change simply because it becomes more confident. The Brier Score exposes whether that confidence is justified.

Using the Brier Score to Validate Probability Forecasts

The Brier Score is most valuable when used out of sample. That means the model should be evaluated on matches it did not train on.

In football, this is especially important because team strength changes over time. A model can look excellent on historical data if it has indirectly learned patterns that were only obvious after the fact. True validation requires chronological discipline: probabilities should be generated before matches are played and judged only once outcomes are known.

A robust validation workflow usually includes:

This prevents a common modelling trap: improving the past without improving the future.

For example, if a model is adjusted after observing that home favourites underperformed in a recent period, it may fit that period better without becoming more predictive. The Brier Score should be checked on later matches to see whether the adjustment genuinely improves probability quality.

Common Mistakes When Interpreting the Brier Score

Judging Too Few Matches

Football outcomes are noisy. A model can receive poor Brier Scores over a short run because several unlikely events happen close together. That does not automatically mean the model is bad.

A 20% away win will happen sometimes. If several occur in a small sample, the model may look worse even if its probabilities were reasonable. The Brier Score becomes more reliable over larger samples.

Confusing Accuracy With Probability Quality

A model can have high pick accuracy but poor Brier performance if it is overconfident. Always selecting favourites may produce many correct picks, but if the assigned probabilities are inflated, the model will be penalised.

In football prediction, the goal is not simply to pick winners. It is to estimate uncertainty accurately.

Ignoring the Draw

The draw is a defining feature of football modelling. Treating match prediction as if it were only home versus away removes a major source of uncertainty.

For 1X2 forecasting, the Brier Score should evaluate all three outcomes. A model that predicts home wins well but systematically misprices draws is incomplete.

Comparing Different Scoring Versions

Some analysts use the summed multi-class Brier Score. Others average across classes. Neither is inherently wrong, but they produce different scales.

Comparisons must use the same calculation method. Otherwise, the numbers are misleading.

Using Brier Score Without Calibration Analysis

The Brier Score gives a useful summary, but it does not reveal every weakness on its own. Two models can have similar average Brier Scores while behaving differently across probability bands.

One may be well calibrated but cautious. Another may be sharp but uneven. Calibration plots, reliability tables, and segmented analysis help reveal the difference.

Practical Example: Comparing Two Football Models

Imagine two models produce probabilities for the same match.

Model A:

Model B:

The home team wins.

Model A Brier Score:

(0.44 − 1)² + (0.29 − 0)² + (0.27 − 0)²

0.3136 + 0.0841 + 0.0729 = 0.4706

Model B Brier Score:

(0.60 − 1)² + (0.23 − 0)² + (0.17 − 0)²

0.1600 + 0.0529 + 0.0289 = 0.2418

Model B performs better for this match because it assigned a higher probability to the outcome that occurred.

But now imagine the match ends in a draw.

Model A:

(0.44 − 0)² + (0.29 − 1)² + (0.27 − 0)²

0.1936 + 0.5041 + 0.0729 = 0.7706

Model B:

(0.60 − 0)² + (0.23 − 1)² + (0.17 − 0)²

0.3600 + 0.5929 + 0.0289 = 0.9818

Model A is now better because it gave the draw more respect. This example shows why the Brier Score should be averaged across many matches. A single match tells us little; a large set of probability forecasts reveals whether the model’s confidence is justified.

FAQ

What does the Brier Score measure?

The Brier Score measures the accuracy of probability forecasts. It compares predicted probabilities with actual outcomes and penalises the squared difference. Lower scores indicate better probability forecasts.

Is a lower Brier Score always better?

Yes, when comparing the same type of forecast using the same calculation method. A lower Brier Score means predicted probabilities were closer to the outcomes that occurred.

Can the Brier Score be used for football match predictions?

Yes. It can be used for binary predictions such as over 2.5 goals, or multi-class predictions such as home win, draw, and away win. For 1X2 markets, the score is calculated across all three possible outcomes.

Does the Brier Score tell me which team will win?

No. The Brier Score evaluates the quality of probabilities after outcomes are known. It is a model evaluation metric, not a match selection method.

How is the Brier Score different from accuracy?

Accuracy only checks whether the selected outcome happened. The Brier Score evaluates the probability assigned to every outcome. This makes it much better suited to football prediction models, where uncertainty matters.

Why is calibration important for the Brier Score?

Calibration tells us whether predicted probabilities match real-world frequencies. If teams given a 60% chance win around 60% of the time over many matches, the model is well calibrated in that range. The Brier Score helps identify whether probabilities are behaving correctly.

Should Brier Score be used alone?

No. It is best used with other validation tools such as log loss, calibration curves, reliability tables, and segmented performance analysis. Together, these give a fuller picture of model quality.

Conclusion

The Brier Score is one of the clearest ways to evaluate football probability forecasts. It rewards models that assign high probabilities to outcomes that happen, penalises confident mistakes, and helps distinguish genuine forecasting skill from simple pick accuracy.

For football analytics, its real value lies in calibration. A good model should not merely identify likely winners; it should estimate uncertainty accurately across home wins, draws, away wins, goals markets, and team-level events.

Used properly, the Brier Score is more than a performance metric. It is a diagnostic tool for improving football prediction models, testing assumptions, and ensuring that probabilities mean what they claim to mean.

Want to see this in action? Check our live match predictions for daily updated probabilities on every fixture.

Related Articles