Analyzing the Ranking System – Week 5

Hey everyone, Tom here. Last week I introduced three different ways of looking at the Beatpaths rankings as a whole system:

  • Stability of the rankings;
  • Using BeatPower to assess the system’s confidence in picking the winner of a given matchup; and,
  • Looking backwards to see how well each team’s rank ‘predicts’ its past record.


Below are two graphs, one of which is the traditional graph that I’ve been using as a shorthand way of tracking how much movement there is in the team rankings week-by-week. The other is a new graph, made at the suggestion of commenter Thurhame, that displays the statistical variance of team rankings week-by-week.

The traditional graph simply adds up the number of ranks each team moved between each week. This way we can see the raw total of slots teams moved. More dramatic changes in the rankings means a spike in the graph. As you can see, the ranks are settling down so far this season, with less and less dramatic movement. The two biggest movers last week were Atlanta, jumping up from #16 to #6, and the NY Jets, dropping from #5 to #15.

The new graph shows the statistical variance of rank changes each week. If I’ve understood it correctly, it’s actually a pretty simple operation. We take the change in rank of each individual team, and square each rank change. Then we take the average of all the squared rank changes. This is the “mean of the squares.” Then we go back and take all the changes in rank of each individual team again, only this time we average them all and then square them. This is the “square of the mean.” To get the variance, we subtract the square of the mean from the mean of the squares. (If you want to take a shortcut, you can just have Excel calculate the variance.)

What does variance tell us? Like standard deviation, variance is a statistical measure of the dispersion of a set of numbers. If most of our numbers are close to each other, and only a few are large outliers, then the variance will be low. If most of our numbers are wildly disparate, then our variance will be high.

Hopefully I’ve gotten all this right. But if I haven’t, I’m sure someone who has taken stats more recently that I have will correct me. 😉

The good news so far is that this season’s rankings are settling down rather quickly. The more data that is fed in, the more accurate the rankings (or so we hope). Even with our first beatloop forming this past week (when Miami beat the NY Jets), the removal of the loop data did not cause a major shake-up in the rankings. We will see if the ranking stability continues to hold, or if major upsets in coming weeks cause spikes in these graphs.

Pick Confidence for Week 6

Last week the picks did pretty well. As you can see, the incorrect picks clustered toward the lower confidence games, with only one ‘upset’ in the high confidence picks. Picks for this coming weekend, based on the BeatPower scores from the Week 5 rankings, are below:

(out of 100)
BeatPower comparison
(predicted winner – predicted loser)
New England-Tennessee 81.8 81.8-0.0
Philadelphia-Oakland 78.3 83.3-5.0
Cincinnati-Houston 75.9 94.7-18.8
NY Jets-Buffalo 67.1 68.8-7.1
Denver-San Diego 61.5 100.0-38.5
Green Bay-Detroit 53.3 83.3-30.0
Pittsburgh-Cleveland 38.1 66.7-28.6
Carolina-Tampa Bay 33.3 33.3-0.0
Minnesota-Baltimore 30.0 100.0-70.0
Jacksonville-St. Louis 28.6 28.6-0.0
Washington-Kansas City 13.3 13.3-0.0
Arizona-Seattle 12.7 55.6-42.9
Atlanta-Chicago 12.2 90.0-77.8
New Orleans-NY Giants 0.0 100.0-100.0

The most interesting matchup is the zero-confidence matchup at the bottom between New Orleans and the NY Giants. While the official rankings based on the tiebreakers picks the Saints, the BeatPower numbers give no edge to either team.

As with last week, I’ll return to update this chart with correct and incorrect picks as the results roll in.

Predicting the Past

Because of our first beatloop forming last week (ATL->MIA->NYJ->NE->ATL), we can now start looking at the ‘flukeyness’ of teams. Flattening out cyclical relationships into a ranked list will always produce distortions, just like the surface of the earth’s globe requires distortions in order to make a flat map. These distortions arise from teams that are ranked above teams that they lost to, or below teams that they beat. By examining which teams introduce the most distortions, we might get a better idea of which teams are inconsistent, or which games were an ‘any given sunday’ or ‘fluke’ victory.

Here is the list of the teams involved in the beatloop, Beatpaths’ record of retroactively picking each team’s past games, and the games that Beatpaths’ would pick incorrectly given the team’s current rank:

  • Atlanta (4-1): loss to lower ranked New England
  • New England (3-2): win over higher ranked Atlanta, loss to lower ranked NY Jets
  • NY Jets (3-2): win over higher ranked New England, loss to lower ranked Miami
  • Miami (4-1): win over higher ranked NY Jets

All the other teams, because they are not involved in beatloops, have perfect 5-0 or 4-0 retroactive picks, given their Beatpath ranking.

So it appears that New England and the NY Jets may be inconsistent in a way that makes it hard to rank them correctly (more so than Atlanta or Miami at the moment). We will see as the season wears on whether this beatloop is broken, or whether the formation of new beatloops make more teams appear inconsistent in their level of play.

2 Responses to Analyzing the Ranking System – Week 5

  1. ThunderThumbs says:

    I love the retroactive records… partly because it underscores my thinking on the beatloop approach. These retroactive records could be improved by simply picking a game in the beatloop to ignore, and keeping the resultant beatpath in the graph… but, I can’t help but think that would lead to lower accuracy in the picks for the rest of the season. In other words, it makes intuitive sense to me that New England should be 3-2 against the current rankings, rather than 4-1 or 5-0, because it does seem they’re being a little inconsistent this season.

  2. […] at pick confidence last week didn’t go very well. Usually, the low confidence picks have about twice the incorrect results […]

Leave a Reply

Your email address will not be published. Required fields are marked *