Even though the aggregate number of rank changes this week is less than in Week 6, the variance of rank changes in Week 10 is much higher than in Week 6. This means that in Week 10 there are more teams moving very few spaces and a significant number of teams moving 10+ spaces. In Week 6 all teams shifted ranks, without as many outliers either moving 0-1 spaces or 10+ spaces.

Pick Confidence – Week 11

With the new, flatter graph, the system’s confidence in making picks is lower this week than in previous weeks. Oddly, last week’s picks suffered the most in the mid-range, rather than in the low confidence range. We’ll see how this week’s flatter, more ambiguous graph performs.

The match-up that stands out most to me is how closely the system calls the Dallas-Washington division rivalry game now that Washington has beaten Denver. As usual, I’ll return to fill in the results as they roll in.

Stability & Variance

Two big movers this week, Philadelphia and Arizona, each moving over ten ranks. The overall ranking changes have fallen again, demonstrating more overall stability in the rankings after the shakeup in Week 6.

One point of comparison with Week 8 is Week 5. Approximately the same overall number of rank changes (86 vs. 84), but a higher variance this week. The difference in variance means that back in Week 5 lots of teams moved an average amount, whereas in Week 8 most teams had a small change in rank while a few teams had major shifts.

Pick Confidence

Last week the pick confidence didn’t do well. The top confidence picks were less reliable than the low confidence picks. We’ll see if the shuffle in the ranks gives us greater accuracy in making picks.

As usual, I’ll return to fill in the results as they roll in.

Stability & Variance

The Week 7 rankings shifted around a good deal last week, but not quite as much as the week before. Given the number of upsets, the amount of change to the ranks is not that bad.

An interesting point of comparison is between Week 7 and Week 4. The raw number of ranking changes was just fewer in Week 7 than in Week 4 (113 vs. 112), but the variance in Week 7 was much higher (7.75 vs. 9.31). What this means is that in Week 4, most teams were moving several ranks, and there were only a very few big moves. In Week 7, most teams were not moving much, but a minority of teams had big moves—teams like Cincinnati’s rise from 17 to 6 and San Francisco’s fall from 8 to 20.

Pick Confidence

TT has already given this week’s official Beatpaths picks. Below are the unofficial rankings of how much ‘confidence’ the system has in making each of the picks:

As usual, I’ll return to fill in the results of the picks as they come in.

Stability

As you might expect, the rankings have undergone a significant shuffle this week. Three matchups looked like pretty significant upsets:

• Philadelphia’s loss to Oakland;
• Cincinnati’s loss to Houston; and,
• NY Jets’ loss to Buffalo.

The overall effect on the system of team rankings was substantial. For the last two weeks, we’ve only had two teams move 10+ spots. This week we had four: Houston, Cincinnati, San Diego, and Green Bay. All four of those big shifts, as Kenneth explained, were based on the results of the Houston-Cincinnati game.

The two graphs below show the raw number of rank changes between Week 5 and Week 6 (“Stability”), and the statistical measure of the dispersion of rank changes (“Variance”). Both show a big spike this week.

Pick Confidence for Week 7

Looking at pick confidence last week didn’t go very well. Usually, the low confidence picks have about twice the incorrect results than the high confidence picks. But last week, the outcome was reversed, with Philadelphia, Cincinnati, and the NY Jets all falling to their low-ranked opponents. Certainly some or all of these can be explained away as ‘flukes,’ but a dismissive approach to problems with one’s own model doesn’t lead to a better model.

Fortunately, over the past few weeks TT has been discussing and now implementing a new tiebreaker model. Part of the implementation has been to get rid of the old BeatPower numbers and replace them with a new set of numbers—EdgePower. TT has backtested this scheme as a tiebreaking method and found that it is more accurate when applied to the last few seasons worth of data. Hopefully, using the EdgePower numbers will give us slightly more sensitive comparisons between teams when trying to determine the confidence that the Beatpaths system has in a given matchup’s outcome.

So without further ado, here are this week’s confidence rankings of the Week 7 picks:

One aspect of the new system that is immediately apparent is the much larger number of low confidence games. The contrast between team strengths, using EdgePower numbers, is much less stark. My intuition is that over the course of the season, as the graph gets more and more connected, the EdgePower of the teams at the top and bottom will begin to diverge more clearly from the middling or inconsistent teams.

Only two games are given a high confidence pick by the system: New England over Tampa, and Indianapolis over St. Louis. The two bottom games stand out to me as well. Just like last week, we have a zero-confidence game between Buffalo and Carolina. And the very close call the system is making between the NY Jets and Oakland is a bit shocking at first, but I suppose since Russell had a not-terrible day against the Eagles D, whereas Sanchez did his best Delhomme imitation last Sunday, I can see how such a close pick can be rationalized.

I like having a new and updated EdgePower system, but I am sensitive to the fact that it’s much less ‘confident’ about making picks now. Making more qualified predictions is not necessarily making better predictions. However, we’ll see how it goes over the course of the season, and get a sense for what exactly constitutes a high confidence score using the new numbers. And, after all, BeatPaths was not built to be predictive so much as it was built to be descriptive. So confidence rankings for picks remains, as always, just a fun side-project.

I’ll be back on game-day to fill in the correct and incorrect results of the picks.

Who’s Flukey Now?

Last week I was able to start looking at the retroactive records of several teams, once we got our first beatloop. Because Beatpaths simply removed any games involved in a beatloop from the system, it introduces some ambiguity into the rankings. So the idea here is to look and see which past games don’t make sense given each team’s current ranking. Maybe this will give us some insight into inconsistent teams or fluke games.

Here are the teams involved in beatloops this week. The record beside each team is not its win-loss record, but the Beatpaths record at ‘picking’ each team’s past matchups, and the results that are inconsistent with their current rank.

• Cincinnati (3-3): wins over Green Bay, Baltimore, and Pittsburgh.
• NY Jets (3-3): wins over Houston, New England; loss to Miami.
• San Diego (3-2): losses to Baltimore, and Pittsburgh.
• Green Bay (3-2): win over Chicago, loss to Cincinnati.
• Houston (4-2): losses to NY Jets, and Jacksonville.
• Baltimore (4-2): win over San Diego; loss to Cincinnati.
• Pittsburgh (4-2): win over San Diego; loss to Cincinnati.
• Jacksonville (4-2): win over Houston, loss to Seattle.
• Chicago (4-1): loss to Green Bay.
• New England (5-1): loss to NY Jets.
• Miami (5-1): win over the NY Jets.
• Seattle (5-1): win over Jacksonville
• Buffalo (5-1): win over NY Jets

For all the other teams not listed here, Beatpaths has a perfect 6-0 or 5-0 record in ‘predicting’ their past record, given their current rank.

As you can see, the two most difficult teams to place are Cincinnati and the NY Jets. You can get an intuitive sense for their inconsistency by looking back at the Week 6 rankings, and seeing the range of possible ranks Cincinnati and the NY Jets could have and remain consistent with the graph. They could be top teams, they could be terrible teams, or they could be both and we might just never know which will show up on game day.

• Stability of the rankings;
• Using BeatPower to assess the system’s confidence in picking the winner of a given matchup; and,
• Looking backwards to see how well each team’s rank ‘predicts’ its past record.

Stability

Below are two graphs, one of which is the traditional graph that I’ve been using as a shorthand way of tracking how much movement there is in the team rankings week-by-week. The other is a new graph, made at the suggestion of commenter Thurhame, that displays the statistical variance of team rankings week-by-week.

The traditional graph simply adds up the number of ranks each team moved between each week. This way we can see the raw total of slots teams moved. More dramatic changes in the rankings means a spike in the graph. As you can see, the ranks are settling down so far this season, with less and less dramatic movement. The two biggest movers last week were Atlanta, jumping up from #16 to #6, and the NY Jets, dropping from #5 to #15.

The new graph shows the statistical variance of rank changes each week. If I’ve understood it correctly, it’s actually a pretty simple operation. We take the change in rank of each individual team, and square each rank change. Then we take the average of all the squared rank changes. This is the “mean of the squares.” Then we go back and take all the changes in rank of each individual team again, only this time we average them all and then square them. This is the “square of the mean.” To get the variance, we subtract the square of the mean from the mean of the squares. (If you want to take a shortcut, you can just have Excel calculate the variance.)

What does variance tell us? Like standard deviation, variance is a statistical measure of the dispersion of a set of numbers. If most of our numbers are close to each other, and only a few are large outliers, then the variance will be low. If most of our numbers are wildly disparate, then our variance will be high.

Hopefully I’ve gotten all this right. But if I haven’t, I’m sure someone who has taken stats more recently that I have will correct me.

The good news so far is that this season’s rankings are settling down rather quickly. The more data that is fed in, the more accurate the rankings (or so we hope). Even with our first beatloop forming this past week (when Miami beat the NY Jets), the removal of the loop data did not cause a major shake-up in the rankings. We will see if the ranking stability continues to hold, or if major upsets in coming weeks cause spikes in these graphs.

Pick Confidence for Week 6

Last week the picks did pretty well. As you can see, the incorrect picks clustered toward the lower confidence games, with only one ‘upset’ in the high confidence picks. Picks for this coming weekend, based on the BeatPower scores from the Week 5 rankings, are below:

The most interesting matchup is the zero-confidence matchup at the bottom between New Orleans and the NY Giants. While the official rankings based on the tiebreakers picks the Saints, the BeatPower numbers give no edge to either team.

As with last week, I’ll return to update this chart with correct and incorrect picks as the results roll in.

Predicting the Past

Because of our first beatloop forming last week (ATL->MIA->NYJ->NE->ATL), we can now start looking at the ‘flukeyness’ of teams. Flattening out cyclical relationships into a ranked list will always produce distortions, just like the surface of the earth’s globe requires distortions in order to make a flat map. These distortions arise from teams that are ranked above teams that they lost to, or below teams that they beat. By examining which teams introduce the most distortions, we might get a better idea of which teams are inconsistent, or which games were an ‘any given sunday’ or ‘fluke’ victory.

Here is the list of the teams involved in the beatloop, Beatpaths’ record of retroactively picking each team’s past games, and the games that Beatpaths’ would pick incorrectly given the team’s current rank:

• Atlanta (4-1): loss to lower ranked New England
• New England (3-2): win over higher ranked Atlanta, loss to lower ranked NY Jets
• NY Jets (3-2): win over higher ranked New England, loss to lower ranked Miami
• Miami (4-1): win over higher ranked NY Jets

All the other teams, because they are not involved in beatloops, have perfect 5-0 or 4-0 retroactive picks, given their Beatpath ranking.

So it appears that New England and the NY Jets may be inconsistent in a way that makes it hard to rank them correctly (more so than Atlanta or Miami at the moment). We will see as the season wears on whether this beatloop is broken, or whether the formation of new beatloops make more teams appear inconsistent in their level of play.

• Did the Beatpaths system arrive at more accurate rankings as the season progressed? And,
• Does the BeatPower score tell us anything about the confidence of the system in picking the winner of a given matchup?

At the end of the 2008 regular season, I also did a retrospective look at how well the rankings reflect a team’s overall record.

The point is to think about Beatpaths as a whole system, instead of just looking at any one team. This is the idea of the Beatpaths system in its essence: to look at each team relative to all other teams.

Stability in the Ranks

Below are two graphs: the final graph of ranking shifts from last season, and the first graph of this season. The purpose of these graphs is to look at how many teams shift in the rankings each week, and how dramatically they shift. The method is simple: add up the difference between each team’s rank from this week and the previous week. The higher the number, the less ‘stable’ the rankings are overall. If the Beatpaths system is working, the rankings should become more and more stable as the season progresses, because we will have more information about each team relative to all the others, and can more accurately place them in context.

While this year and last both started out with a high degree of instability in the rankings (typical of the early season), this year’s instability has dropped much more sharply. This may be due to the new tie-breaking criterion being used by TT this year. It also may be due to the lack of beatloops so far this season, because the creation or breaking of beatloops tends to create significant shifts throughout the rankings. We’ll see if this level of stability in the rankings will hold or not.

Making Picks with Confidence

Beyond the looking at the stability of the rankings as a whole, I also looked into which matchups the BeatPower scores of each team predicted with the most confidence. You’ll notice when looking at the weekly rankings that they include a BeatPower score on the right-hand side. The method here is simple: compare the matchups for Week 5 by weighing the rival teams’ BeatPower scores against one another. The greater the disparity, the greater the confidence the system has in picking the winner. If the BeatPower scores are close, the system has less confidence in picking a clear winner. Last year I found as a general rule that the top half of the ‘Confidence’ chart (high confidence picks) would have about half as many incorrect picks as the bottom half of the chart (low confidence picks).

Here are the matchups for Week 5 and the ‘confidence’ that the system has in them:

Note the negative confidence pick at the bottom. We ran into these a few times last year. These occur because the Beatpath Rankings and BeatPower scores occasionally diverge (i.e. a team with a lower BeatPower score will nevertheless be ranked higher than another team with a higher BeatPower score). Since these uniformly happen with low confidence games, it doesn’t seem to make a difference that the score is negative: the Beatpaths system isn’t terribly confident about picking a winner one way or the other. My advice is not to bet on those games

Also note: that the ‘confidence’ scores are simply something I’ve been playing around with out of curiosity. TT always gives the official Beatpaths picks, and this year he’s comparing Beatpaths, Isaacson-Tarbell, and a hybrid of those two methods (along with his personal picks, as always).

Looking Back

A final way of gaining insight into the Beatpaths system and how well it’s functioning is to look to the recent past. Specifically, how well does the current ranking of each team retroactively predict its win-loss record? Teams that are consistently good or consistently bad will usually be well represented by their rank relative to other teams. However, teams that are flukey, win against good opponents but find ways to lose to bad opponents, will be difficult to rank no matter what system you’re using. Looking at each team’s retroactive pick record is a good way to identify which teams these are and assess how well the Beatpaths system is handling them.

At this point in the season, without any beatloops, the Beatpath Rankings have a 4-0 (or 3-0 for the bye week teams) retroactive pick records. While the tie-breakers between certain teams may be disputable, the rankings at the moment do not have any team ranked higher than a team that defeated it. Only when beatloops are formed will ambiguities enter into the rankings, which Beatpaths attempts to resolve by simply removing those paths that form a loop. When ambiguous data is removed the rankings may no longer correspond with who-beat-who, and we can begin to examine the retroactive pick record for each team in detail.

That’s it from me this week. I’ll update this post on game day to fill in the pick confidence table with actual game results.

Beatpaths has a 3-7 pick record in the post-season. One notable difference between the regular season picks and the post-season picks is that very few of the post-season picks are high confidence. Of the ten games played so far, only two had a confidence rating over 50 (Pittsburgh-San Diego was called correctly, Indianapolis-San Diego was an upset).

The Super Bowl is no different from the rest of the post-season: a low confidence pick, one subject to greater rates of error than high confidence picks. However, Beatpath’s pick (a Steeler’s victory) conforms broadly with the conventional wisdom about this game: Pittsburgh is expected to do better on the strength of having the best defense in the league.

Both of these picks conform well to the conventional wisdom.

One curious note: Beatpaths has a 1-6 record of predicting the results of Philadelphia games since correctly picking their loss to Baltimore in Week 12. I’ve been used to Beatpaths picking against Philly and being happy to see it proven wrong. Now that Beatpaths is finally picking them to win, I’m suddenly anxious about their winning streak coming to an end in the desert.

In comparison, Beatpaths is 3-4 picking Arizona results over the same period, 5-1 picking Pittsburgh, and 4-3 picking Baltimore.

Like last week, the BeatPower comparisons of the matchups produces picks that diverge from the conventional wisdom. The system is most confident about Pittsburgh beating San Diego, whereas most spreads seem to think that Carolina has the largest expected margin of victory over Arizona. BeatPower gives Tennessee the edge here, while a number of other assessments seem to give Baltimore the edge—although in either case, no one seems to be predicting a blow-out by either team. Finally, I think BeatPower captures the NY Giants-Philadelphia matchup well, with only a very slim preference for the NY Giants. Given their previous season split, injuries, and intense, long-standing division rivalry, there are few analysts taking a strong stance either way on this game.

The home teams are all favored this week. A rare week of harmony with the conventional wisdom.

Baltimore at TENNESSEE: No change anywhere if Tennessee wins. If Baltimore wins, it creates a season split, while Tennessee has an alternate beatpath – no change to the graph or rankings in this case, either. What this really means is that if Baltimore wins, it would very much definitely be an upset.

Arizona at CAROLINA: No change in the graph if Carolina wins, although Carolina would be helped in the rankings a bit. If Arizona wins, they’d shed the beatloss to Carolina, which would subtly hurt Carolina in the rankings.

Philadelphia at NY GIANTS: The Giants are favored, but if they win, crazy things happen – Philadelphia loses their beatwins over Cleveland and Minnesota, and Arizona. Also, ARI->DAL->CIN->WAS disappears. Since ARI->DAL disappears, the Giants actually come out of it looking weaker than the AFC teams, and fall in the rankings, although Philadelphia would fall by more. If Philadelphia wins, it’s cleaner – all the resultant beatloops are already redundant, and so there’s no change to the graph or rankings.

San Diego at PITTSBURGH: No change anywhere if Pittsburgh wins. If San Diego wins, it’s just another season split that is invisible due to alternate beatpaths.

Overall, a continuation of a very weird season – usually, nonfavored teams have an opportunity climb up in the graph throughout the playoffs. There doesn’t seem to be much likelihood of a sane result in the beatpaths graph this season – best we can hope for is for San Diego to defeat Pittsburgh, lose to Tennessee (which would loop away the Jets), and then have Tennessee win the Super Bowl.