Tom here again. Another subject I’ve been investigating in the comments for the past few weeks is ranking of each week’s picks by the model’s “confidence” in a win or loss.

TT gave us the official Beatpaths model’s picks for Week 12, but these picks treat each prediction as equally valid. I’m interested in figuring out if we can tell which picks the model says are “sure things” and which picks it’s less sure about. So I thought that instead of using straight Beatpath ranks to pick a winner, we could compare the BeatPower numbers of each matchup. Teams with a big difference in their BeatPowers would be a high confidence pick, whereas teams with little difference in their BeatPowers would be a low confidence pick.

So, below I’ve ranked Week 12′s picks by “confidence” — the higher the confidence score, the more confident the model is that the pick will turn out to be correct.

Matchup (Winner-Loser)	“Confidence” (out of 100)	BeatPower comparison (predicted winner – predicted loser)
Pittsburgh-Cincinnati	83.9	90.0-6.2
Tampa Bay-Detroit	80.0	80.0-0.0
Dallas-San Francisco	71.8	82.7-10.9
Washington-Seattle	67.9	75.0-7.1
Chicago-St. Louis	67.3	67.3-0.0
Minnesota-Jacksonville	42.3	52.3-10.0
Indianapolis-San Diego	27.1	63.5-36.4
New England-Miami	15.0	40.0-25.0
Buffalo-Kansas City	13.9	23.9-10.0
Arizona-NY Giants	12.3	89.6-77.3
Carolina-Atlanta	8.9	98.0-89.1
Houston-Cleveland	8.1	43.8-35.7
Denver-Oakland	5.4	15.9-10.5
Green Bay-New Orleans	5.3	72.0-66.7
Tennessee-NY Jets	2.2	100.0-97.8
Baltimore-Philadelphia	-9.2	55.8-65.0

If the BeatPower rankings are generally correct, we should see incorrect picks clustered toward the bottom of this chart. There are five games that have a “confidence” of over 50, and we should see far fewer incorrect picks in the top half of this chart.

There is one odd result produced by the BeatPower “confidence” ranking when compared with the normal Beatpaths picks: the Baltimore-Philadelphia game. The regular Beatpaths method predicts a Baltimore win, whereas the BeatPower ranking favors Philadelphia (leading to the negative “confidence” score because of the disagreement between the two methods). It’s unclear at this point whether one method is more accurate than the other when they disagree, but either way it’s a low confidence game (with a confidence score of only 9.2), so it should be an uncertain pick in either model.

We already know that the Pittsburgh game was correctly picked, but I’ll check back after Monday night to assess whether or not this method of ranking the model’s confidence in the picks it makes is valid. If it is valid, we should see the high confidence games have few missed picks, and low confidence picks with a greater number of misses.