Brier Score
The Brier score measures the accuracy of probabilistic forecasts as the mean squared error between each predicted probability and the actual 0-or-1 outcome. Lower is better: 0 is a perfect forecast and higher scores indicate worse calibration.
See Brier Score in context
Build a watchlist, then use brier score alongside a signal’s entry, stop, target, and reasoning—not as a trade instruction.
Explained Simply
For a single yes/no forecast, the Brier score is (predicted probability − outcome)², where the outcome is 1 if the event happened and 0 if it did not. Averaging that across many forecasts gives one number between 0 and 1. It is the standard way to check whether a forecaster who says '70%' is actually right about 70% of the time — good calibration keeps the score low. Because it rewards both accuracy and honest probabilities, the Brier score is widely used to compare forecasting models and to detect over- or under-confidence.
How to Calculate a Brier Score
Take each forecast's predicted probability, subtract the realized outcome (1 for happened, 0 for not), and square the difference. If you forecast 0.80 and the event happens, the score for that forecast is (0.80 − 1)² = 0.04. If it does not happen, the score is (0.80 − 0)² = 0.64.
Average those per-forecast scores across all your predictions to get the overall Brier score. Confident forecasts are rewarded when right and penalized heavily when wrong.
Brier Score and Calibration
A low Brier score means predicted probabilities line up with reality over the long run — the essence of calibration. It exposes overconfidence: a model that says '95%' but is right only 70% of the time will score poorly.
This differs from simple accuracy, which only checks whether the more-likely side happened. Two models can share the same accuracy while one is far better calibrated, and the Brier score is what separates them.
Frequently Asked Questions
What is a good Brier score?
Lower is better, with 0 being perfect. Context matters: for evenly-matched coin-flip events, always predicting 50% yields a Brier score of 0.25, so a useful forecaster should score below that baseline.
How is a Brier score different from accuracy?
Accuracy only checks whether the predicted direction was right. The Brier score also rewards well-calibrated probabilities, penalizing a forecaster who is directionally correct but wildly over- or under-confident in the numbers.
Why does calibration matter for trading signals?
A confidence score is only useful if it means what it says. Tracking a Brier score shows whether a model that labels setups '70% confidence' is right about 70% of the time, or whether it is systematically over- or under-confident.
How Tradewink Uses Brier Score
Tradewink's signal confidence calibrator can compute a Brier score for signal confidence against realized outcomes, so that a stated 70% confidence can be checked against roughly a 70% hit rate over many signals. This is an internal accuracy and model-quality check for educational transparency, not a performance guarantee.
Save a signal preview for later
Get a concise AI signal example in your inbox, then build a watchlist when you are ready. No spam, unsubscribe anytime.
Related Terms
Learn More
How AI Trading Signals Work: From Data to Trade Idea
Ever wonder how AI generates trading signals? We break down the full pipeline: data ingestion, pattern recognition, scoring, filtering, and delivery.
AI Trading Signals Explained: What They Are and How to Use Them
AI trading signals use machine learning to identify potential trade opportunities across hundreds of stocks. Learn what they are, how they work, what the data means, and how to evaluate signal quality.
How Accurate Are Trading Signals? Evaluating Evidence
How accurate are trading signals? Learn to assess sample size, scoring rules, delivery delay, costs, and the limits of paper and live comparisons.
Keep Brier Score in context
Get practical learning notes and signal explanations that show where brier score fits in the broader setup.