I’ve managed to configure the backtesting script a bit more correctly now, and I’ve also included a first stab of the tiebreaker method that uses paths in and paths out, which is what Moose uses for his non-weighted rankings. Below is the grid for the vanilla variant.

And here are all the tiebreaker variants for single-year data sets using beatflukes graphs:

So, conclusions? First it’s odd and a bit discouraging that the accuracy for a season is pretty much indistinguishable from accuracy based off of multiple seasons of games all mixed together. As for the accuracy levels, I found one resource saying that Isaacson-Tarbell was 158-97-1 in 2008 – in the regular season. That’s 61.9%. None of the variants beat that, but the actual beatpath picks (a subset of all the games) do beat that. But, who’s to say that Isaacson-Tarbell wouldn’t pick that same subset even more accurately? At any rate, further investigation is required to see if some combination of beatpaths picks, better-record, and home team could beat Isaacson-Tarbell.

So, there’s some food for thought. Here’s how to read it – the records in parentheses are the actual beatpaths records, meaning, the games that have actual beatpath relationships. All the other methods in the table are tiebreaker methods; meaning ways to rank teams that don’t have beatpath relationships. All are based off of the same beatpath graph, and so all start from the records in the left column. We wouldn’t expect those methods to have a higher win percentage than the beatpath win percentage. These are various tiebreaker methods that I have tried out in the past. I haven’t tried out Weighted or Moose’s other ranking system – I’d expect Weighted to not perform very well, and the other ranking system to perform very well. This is also the vanilla method of finding beatpath relationships – not beatflukes or iterative.

I am curious what the historical win percentage of Isaacson-Tarbell is, at least over the same five-year period. This page indiciates Isaacson-Tarbell’s win percentage long-term is 62.29%, which would indicate that perhaps the best strategy to follow is 1) pick the team with a beatpath relationship to another team, 2) If there isn’t one, pick the team with the winning record, 3) if they’re tied, pick either the home team or the one selected by the beatpaths tiebreaker.

Hopefully in the future I’ll be able to look at how the beatpaths percentage (64.13%) compares to beatflukes, iterative, or whatever other kind of beatloop resolution scheme we come up with.

Isaacson-Tarbell is “picking for idiots”. And it’s pretty bulletproof – it seems most pick experts can’t outperform it. The picking rules are: Pick the team with the better win-loss record. If they’re tied, pick the home team.

Isaacson-Tarbell was 9-7 for week one. I am thinking of using a variant of Isaacson-Tarbell, perhaps called Isaacson-Tarbell-Beatpaths (ITB) for this season. This is – pick the team with the better win-loss record. If they’re tied, pick the team that is ranked higher according to beatpaths. Last season, beatpaths outperformed the home-team-wins approach, so I’m hoping I’ll be able to beat Isaacson-Tarbell this season.

For week 1 (where no one had a better win-loss record), I relied on beatpaths rankings, so we’re consistent with this approach. Beatpaths had an 11-5 record, or two better than Isaacson-Tarbell.

We start out with all the wins and losses for the season so far. The first design choice I made was to ignore points, and any other stats other than wins and losses. This is the conceit of the site and the idea. We focus only on wins, losses, and who beat who – and then, usually around Week 4, we start seeing beatloops, like A->B->C->A.

It may be that the initial beatloops are very long, or very short.

The next design choice that I made long ago was to remove the smallest beatloops first. I did this for both philosophical and expedient reasons. First, the smallest beatloop is a season split – where two teams beat each other. What this says to me is that we are unable to determine which team is better by looking at only those two game outcomes, so we throw them out. And so after all team splits are removed, I remove remaining 3-team beatloops, and then the remaining 4-team beatloops, etc.

This is arguable, as you may have A->B->C->A, and A->B->D->E->A, and C->A might be spectacularly flukey. Removing the smallest first would yield B->D->E->A. Removing the flukey game and the longer beatloop would retain B->C. If C->A is flukey, I think you can make a stronger case that B->C than B–>A. But even then, the question is how do you determine that C->A is flukey?

The above paragraph is a good example of the pretzels we enjoy tying our brains up in during the NFL season.

The expedient reason was that removing smaller beatloops first tends to straighten out the longer beatloops. If you don’t remove the smaller ones first, things get so circular with so many different routes, that any other scheme to prioritize loop removal becomes about even more ambiguous tradeoffs.

The other main tradeoff with this approach is that one game can be in several shared beatloops, thereby eliminating several other game outcomes. This has been the source of much of our commentary in previous seasons – whether we see these games as flukey games or key games. I have never been convinced that a game being in more shared beatloops means that is is flukey. But we have developed a few theories (check the comments here for instance) on how to identify actual links (game outcomes) to remove, thereby de-emphasizing loop removal in favor of breaking them into segments that we keep.

At any rate, all this discussion is to get down to one data structure, a directed acyclic graph (DAG).

From there, it’s about finding a power ranking. The general approach is to examine all teams that don’t have a beatloss, apply a tie-breaker, pick one, remove it from the graph, and then examine all remaining teams that don’t have a beatloss (which may include new teams if its only beatloss was the team that got removed in the previous step). We’ve also discussed many tiebreaker methods, from referring to the previous week’s rankings, to other more complex methods.

My principles for the system have always been to keep things simple enough that the method can be explained to a sports layperson. There are articles every year on how Podunk University should be ranked ahead of University of Miami, because of a chain that is long beatpath, so the concept itself is fun and interesting. So far the approach has been to not try and identify single games as flukey and remove them, because what seems like a fluke could instead be a massive clue or harbinger. Instead, the approach has been to instead merely remove ambiguities, and rely on the more clarified parts of the season to attempt to fill in the blanks.

The final part of the system has been to have fun using the system in various ways to come up with weekly picks. I’d love the commenters to come up with their own picking systems that use beatpaths either in full or in part. I’m thinking of a variant of the Isaacson-Tarbell predictor that relies on beatpaths just a tad.

One statistic I’m curious about – the historical predictive pick record for matchups that actually have beatpaths, versus ones that don’t.

I’m surprised at how clear this looks. This is based off of my brainstorming post last week and the ensuing discussion. What I did for this graph is I simply excluded all matches this season that were intra-conference or inter-conference seeding matches. Every team had two games excluded; a home game and an away game.

The goal here has always been to create a pecking order of the teams, based off of wins and losses and nothing else. This means a DAG – a directed acyclic graph. Acyclic means no beatloops.

Beatloops exist and there’s no way around that. The goal is to remove them. The question was what a beatloop means. Does it mean the teams in a beatloop are tied? The thought here was to reject that – a beatloop doesn’t mean that the teams are tied; it just means that the relationship between the teams is ambiguous.

This frees us up a bit – all we have to do is remove ambiguity and just rely on the rest of the graph to sort itself out into a rough pecking order.

We want to retain clear win/loss relationships and reject ambiguous data. Which means rejecting the most “clearly ambiguous” data. To me, this has always meant, the smallest set of data. I’ve had it linked in my head that the smallest set of ambiguous data by definition meant the most tightly linked set of ambiguity. It’s clear that in the NFL, this is not necessarily true. I think it’s true that a four-team beatloop can be more tightly ambiguous than a three-team beatloop. But more on that later.

But I think the general principle is to remove the smallest amount of ambiguity, in order to retain other clear relationships.

So one approach was just always to resolve the smaller beatloops first. At first, this makes sense – two-team-beatloops (home-and-home season series splits) mean that it’s hard to say which team is better than the other just based off of the wins between the two teams. That’s intuitive. That’s a clear example of “tight ambiguity”. It’s an easy decision to remove season splits. Those links in there can lead to an absolute ton of beatloops, and it was gratifying to see how much simpler the graph got after removing even only the season splits.

Next was again the question of what is the best way to fairly remove the smallest set of ambiguous data? My approach here was – when there’s a question, punt – as a way to avoid as much subjectivity as possible. (Leaving aside the semantic discussion of how even choosing to create this website was an exercise in subjectivity. )

Punting meant – rather than trying to determine how to split apart the collection of three-team beatloops into several families, to just remove all of them at once, no matter how much they overlapped.

This has been the base system from the beginning. I like it because it’s simple enough that I believe anyone can grasp the approach without forgetting the details. Map all the victories. Remove the season splits. Remove the beatloops, smallest first. Done.

So what are the subjective choices I drew here? First, there was the desire to actually end up with a DAG (direct acyclic graph). We can even challenge that one and just choose to determine rankings based off of a graph with loops in it, but I want to avoid that direction for now. It’s not visual enough and doesn’t strike me as intuitive to a casual graph viewer.

So the first subjective choice was to remove smallest loops first. We already have one open challenge there – Boga sketched out a quick algorithm that I’d like to explore (if I can just find the comment again). That would be to work on resolving all loops at once, I think (or it might have just been how to rank teams before the loops are taken out).

And I’m also pretty convinced at this point that resolving smallest loops first is more sound only if the teams all play each other, as they do in the NBA and MLB. In the NFL, a four-team beatloop can exist and it doesn’t necessarily mean it shouldn’t be removed at the same time as some other three-team beatloops, and this is specifically because of the rarity of intra-conference games, which is not a team’s fault.

The other subjective choice was to just remove all n-sized beatloops at once, no matter how much they overlapped. The thought here was to avoid the subjectivity of choosing to remove 3-team beatloops before removing other 3-team beatloops. But in a sense I am already introducing a similar subjectivity by removing 3-team beatloops before removing 4-team beatloops.

So while one direction is resolving all beatloops at once, the other direction is to choose to split out (e.g.) 3-team beatloops into multiple families, and resolve some of them first, and then the others.

There are some easy ways to chip away at this that would be noncontroversial. The problem is not a team being in several beatloops, it’s of a single segment (TeamA=>TeamB) being in several beatloops. So you could first remove every beatloop where none of its single segments are in other beatloops. If I were to do this and then resolve the remaining beatloops with current rules, I’m pretty sure I’d end up with the exact same result. But it could introduce some greater clarity in terms of examining each step of the process. There is room to explore in terms of choosing how to remove some beatloops with overlapping segments, before removing other ones.

Iterative is an example of an approach that seeks to remove segments, not loops – it identifies actual single-match outcomes that should be ignored in the initial data set, as opposed to entire loops. Doktarr, please correct me if I’m wrong on this. We’ve had one other similar approach of ignoring actual segments; the beatfluke approach, which obliterates beatloop segments that are directly contradicted by the resultant DAG, thereby restoring the rest of the beatloop to the graph. There is one other obvious segment-removal approach we haven’t explored yet, and that is the one that merely seeks to find the fewest number of matches in a season that can be excluded in order to create a DAG. (For ties, for instance, five ways to remove only seven game outcomes, we’d isolate down by removing the game outcomes that appear in all scenarios, and then rank the rest of the candidates through some set of objective-as-possible standards.) Doktarr has been very patient with my reticence to get back into segment-removal.

So I’ve got a couple of questions here.

1) Doktarr, is it possible to restate iterative logically, in a way that doesn’t use ratios or fractional numbers? Is it as simple as just removing NYJ->MIA first because that link appears “most” (5) in all the 3-team beatloops? I think someone actually did restate it this way once, but I might be confusing it with Boga and MOOSE’s discussion.

2) Also, is it possible to restate iterative in terms of full beatloops being removed?

3) Finally, more thoughts from anyone appreciated on how to judge removing beatloops if multiple sizes of beatloops are considered at once. There’s obviously a problem with this, in that by the time every team has at least one win and one loss, the entire league is one big beatloop. The only thing that we can REALLY say at this point is that Detroit is definitely worse than every team they have directly played. But every other team can draw a circuitous beatpath to any team that has beaten them.

If you’re interested in representing a slice of the content – whether it’s a posting once a week or something more irregular, drop me a note and make a proposal. I can set up a mailing list of us to do some coordination.

For those who don’t know, the backend system is written in perl – I may be interested in allowing others to collaborate on the programming if there’s enough interest.

If you’re interested, contact me through this webform.

Graph Method: Methods that have to do with determining what teams have beatpaths and beatwins to other teams. Examples are the current Standard graph method on this site, the Iterative graph method, and the Beatflukes graph method.

Power Method: Methods that take the Graph methods and then determine sequential rankings from them. Examples are the current strength-of-beatpower power method on this site, the Weighted power method, and MOOSE’s (what’s the name?) Total Paths power method.

I might have already misstated something, because I think I’m still unclear what power method the Iterative graph method uses.

Last week’s power rankings are 10-5 9-6 so far this week.

The only two teams with beatpaths to every other team in their division: Denver and Arizona.

#31 beat #1, with some awesome direct snap misdirection.

And it’s time to start thinking of tiebreaker and rank strategies.

There are two stages to determining rankings. First is the beatloop resolution strategy. That is pretty stable, although doktarr and moose have written about possible ways to enhance it. The general principle of beatloops is not to imply that the teams in a beatloop are tied – it’s more just that it is the smallest set of data that can be seen as ambiguous/confusing, and thus should be removed. That way we rely on the rest of the graph to imply rankings. I think that trying to divine too much data from a beatloop just introduces too many judgment calls into a graph. We always remove smallest beatloops first, starting with splits, and then recalculate.

We’ve tried some methods to bust beatloops here in the past. One that I was fond of was called the beatfluke method, defined as: If Team A’s loss to Team B was beatlooped away, and Team A also has an entirely different remaining alternate beatpath to Team B, then it contradicts Team A’s loss, and thus the A-beats-C-beats-B part of the beatloop can be restored to the graph.

I found this made the graph more vertical, and also slightly more accurate, but I didn’t like how it would lead to more dramatic shifts in the power rankings each week. It made the graphs vary more from week to week. Perhaps if it were combined with a more stabilizing tiebreaker, it could be used again.

The other two approaches of busting beatloops were doktarr’s “iterative” method, and Moose’s score method. The “iterative” method breaks shared beatloops at their shared link, like if one game is responsible for the existence of several beatloops. It is another effort to try to identify one link of a beatloop (a game outcome) as flukey. I do have trouble justifying that one intuitively, though – I feel like I need another reason to believe that link actually is flukey, other than it just being part of several beatloops. The other is a weighted system having to do with score differentials. I believe this ended up accurate and perhaps superior, although I’m trying to keep the main system here free of extra data like points (as opposed to just wins and losses).

After that, there’s how to determine rankings from the resultant beatpath graph. So far this season, I’ve been breaking ties based off of the rankings of the previous week. But the usual tiebreaker for later in the season is to compare the strength of the teams’ direct beatwins – for instance, if every team in a tied set has at least three beatwins, it averages the strength of the top three beatwins of each of those teams, and picks the top team. Finally, I think Moose came up with a tiebreaker having to do with counting all the links in a resultant beatpath graph. This is somewhat similar to what I used in the first and second year here, which counted number of teams above and below each team, but it yields more information in that it counts every link of every possible path, thereby giving extra weight to stronger paths. I probably have this explanation wrong but Moose will correct me in the comments. This is a good candidate to apply as a tiebreaker to the official rankings this season.

A few years back, I was reading one of those websites that purports to power-rank sports teams based off of every single in-game stat in the book. And, being a Broncos homer, I was peeved that the Broncos were ranked so low.

My gut-based thought – completely lacking in any evidence and data – was that wins aren’t always linked to stats. Sometimes it’s heart! And guts! Or the intelligence of a team’s scheme – sometimes they just play the matchup game better. But really, I was just thinking about how it all comes down to wins and losses, and damn the rest of the data.

The advanced ranking schemes pay attention to a lot of data – all the data except for wins and losses. So my perverse idea was to do the inverse. Focus only on wins and losses.

Turns out there are other schemes that also only pay attention to wins and losses – they use big words and probability and math and statistical theory.

I decided to use graphs. Here’s how the idea came about.

I started graphing the NFL season based off of one simple rule – if a team beats another, you draw a line from the winner to the loser. The End.

Soon, you start seeing long sequences of wins and losses – The Broncos beat the Steelers, who beat the Patriots, who beat the Giants — oh wait, they didn’t beat the Giants. Anyway, those sequences of wins become what I call a beatpath.

Now, early in the season, you see sportswriters try in vain to structure their own power rankings by always having teams ranked ahead of other teams they’ve beaten.

But soon, you notice that contradictions start appearing. Every year, there’s some silly article over in college football land about how Captain Doohickey Tech is better than Ohio State, since they beat Team A, who beat Team B, who beat Team C…. who beat Team X, who upset Ohio State back in week 2. When you know just as well that Ohio State has demolished someone who has in turn demolished the Raiders, I mean, Captain Doohickey Tech.

That contradiction is called a beatloop.

And so we, that is the Royal We, not to be confused with Eddie Royal, worked out a system to resolve these beatloops and come up with automatic power rankings. And surprisingly, it’s ended up weirdly accurate. Well, sometimes just weird. But a hell of a lot of fun.

Every week I’ll post graphs that are a snapshot of the NFL season so far. And the automatic power rankings that come from the graph. And we’ll be continuing to discuss ways to tweak the algorithm – commenters often come up with variants to the algorithm that might even be more accurate. And we’ll have arguments about whether the beatpath algorithm is actually predictive (when it’s doing well), or merely descriptive of the season so far (when there are lots of upsets). And there will probably be some subtle Broncos homerism littered throughout, until someone calls me out on it.

Welcome to the site (redesign coming soon!), and enjoy your stay!