Brier Score
The Brier score measures the accuracy of probabilistic forecasts as the mean squared error between each predicted probability and the actual 0-or-1 outcome. Lower is better: 0 is a perfect forecast and higher scores indicate worse calibration.
See Brier Score in context
Build a watchlist, then use brier score alongside a signal’s entry, stop, target, and reasoning—not as a trade instruction.
Starter watchlist: SPY, QQQ, AAPL.
Starter examples. Not recommendations; you can edit them before anything is saved.
Explained Simply
For a single yes/no forecast, the Brier score is (predicted probability − outcome)², where the outcome is 1 if the event happened and 0 if it did not. Averaging that across many forecasts gives one number between 0 and 1. It is the standard way to check whether a forecaster who says '70%' is actually right about 70% of the time — good calibration keeps the score low. Because it rewards both accuracy and honest probabilities, the Brier score is widely used to compare forecasting models and to detect over- or under-confidence.
How to Calculate a Brier Score
Take each forecast's predicted probability, subtract the realized outcome (1 for happened, 0 for not), and square the difference. If you forecast 0.80 and the event happens, the score for that forecast is (0.80 − 1)² = 0.04. If it does not happen, the score is (0.80 − 0)² = 0.64.
Average those per-forecast scores across all your predictions to get the overall Brier score. Confident forecasts are rewarded when right and penalized heavily when wrong.
Brier Score and Calibration
A low Brier score means predicted probabilities line up with reality over the long run — the essence of calibration. It exposes overconfidence: a model that says '95%' but is right only 70% of the time will score poorly.
This differs from simple accuracy, which only checks whether the more-likely side happened. Two models can share the same accuracy while one is far better calibrated, and the Brier score is what separates them.
Frequently Asked Questions
What is a good Brier score?
Lower is better, with 0 being perfect. Context matters: for evenly-matched coin-flip events, always predicting 50% yields a Brier score of 0.25, so a useful forecaster should score below that baseline.
How is a Brier score different from accuracy?
Accuracy only checks whether the predicted direction was right. The Brier score also rewards well-calibrated probabilities, penalizing a forecaster who is directionally correct but wildly over- or under-confident in the numbers.
Why does calibration matter for trading signals?
A confidence score is only useful if it means what it says. Tracking a Brier score shows whether a model that labels setups '70% confidence' is right about 70% of the time, or whether it is systematically over- or under-confident.
How Tradewink Uses Brier Score
Tradewink's confidence calibrator can compute a Brier score by comparing signal confidence scores with realized outcomes over many signals. Tradewink shows those scores as NN/100 and they are not a probability of profit; the Brier score is an internal model-quality check, not a performance guarantee.
Save a signal preview for later
Get a concise AI signal example in your inbox, then build a watchlist when you are ready. No spam, unsubscribe anytime.
Related Terms
Learn More
How AI Trading Signals Work: Check the Evidence
Trace an AI signal from inputs to delivery. Copy an evidence card for timestamps, confidence and paper review, with a fictional worked example.
AI Trading Signals Explained: What They Are and How to Use Them
AI trading signals use machine learning to scan hundreds of stocks. Learn how AI stock signals are generated, how to read them, and how to judge quality.
How Accurate Are Trading Signals? Evaluating Evidence
How accurate are trading signals? Learn to assess sample size, scoring rules, delivery delay, costs, and the limits of paper and live comparisons.
Keep Brier Score in context
Get practical learning notes and signal explanations that show where brier score fits in the broader setup.
Comments
Comments are reader opinions, not investment advice. A moderator reviews your first comment before it appears, and moderators can remove any comment later.
Loading comments…