First, hello to everyone. My name is Tom, and like Kenneth, I’ve also been asked by TT to help out with contributions.

For several weeks I’ve been examining the “stability” of the rankings produced by the beatpaths method. In theory, as more data is fed into the beatpaths system, we should have a better and better idea where a team lies relative to the field. Still, teams can jump around a great deal due to certain graph dynamics: for instance, if they manage an upset victory, or if the teams they’ve previously beaten start to fall in rankings themselves.

One way to measure whether or not the rankings are actually stabilizing over time is simply to add up the number of slots that each team has moved up or down from week to week. This includes both upward and downward movement. The process is simple: for each team, look at its rank this week and take the absolute difference with its rank from last week. This week Carolina is ranked #1, whereas it was ranked #2 last week, for a total change of only 1 rank. In contrast, the Redskins dropped from #6 to #14, for a total change of 8 ranks. Summing the rank change of all 32 teams yields the overall “stability” score for the week.

If the beatpaths method does produce more and more accurate assessments of relative team strength as the season wears on and as more data is fed in, then we should see an overall decline in stability scores, indicating that the changes in rank are getting smaller as time wears on, with fewer big leaps in the evaluation of team strength.

Here are the scores from Week 3 through Week 11:
3 – 202
4 – 152
5 – 124
6 – 128
7 – 90
8 – 108
9 – 138
10 – 43
11 – 59

As you can see, there is a visibly negative trend, meaning that the rankings are more and more stable over time. As more data is fed into the beatpaths method each week, the assessment of each team’s strength relative to the field gets better. To be sure, teams still move, but there are fewer big leaps.

Let’s take a look at the big movers from each week. I’m going to arbitrarily decide that any team moving 10 or more ranks from one week to the next we’ll label a “big mover.” Following is a list of the big movers from each week:
3 – Redskins, Saints, Cardinals, Ravens, Patriots, 49ers, Texans, Browns, Bears
4 – Chargers, Jets, Bears, Eagles, Ravens
5 – Saints, Chargers, Jaguars, Colts
6 – Jaguars, NY Giants, Bears
7 – Broncos, NY Jets, Chargers
8 – Eagles, Bills
9 – NY Jets, Eagles, Jaguars, Broncos
10 – None
11 – None

We can observe two things: 1) that the number of big movers has declined over time, so not only are teams in general changing rank less, but major corrections to the assessment of teams is rarer; and, 2) certain team names reappear several times on that list. Often this is due either to a fluke win or loss that temporarily shoots the team to a drastically different rank, which is subsequently corrected by the beatpaths method the following week when new data comes in, or due to genuinely inconsistent play by teams that can pull off quality wins and then play down to otherwise poor opponents.

I think most of us use this method to follow our favorite teams’ individual performance. However, looking at the overall characteristics of the system itself can provide some valuable insights and confirmation that the system does improve over time as more data comes in. I’ll be doing more of these “stability” analyses in subsequent weeks, after new rankings are posted.