I’ve managed to put a prediction checker into the beatpaths algorithm – checking a few of the algorithm variants for how well they’ve predicted the outcomes of games. I should first start with a caveat – beatpaths is not really intended to be used as a predictive tool. It merely attempts to accurately represent the performance of the teams so far, not predict how future games will turn out. But, under the theory that good teams will continue to play well, it’s interesting to look at how well this actually predicts results. Keep in mind that this system doesn’t even attempt to take into account things like lines or home field advantage.
I always use the same approach to calculate beatpaths and beatloops – that part is stable. But I’ve used a few different tiebreaker algorithms to determine which team to put next in the rankings when multiple teams aren’t blocked by beatlosses. Here are the results for a few of these algorithms:
- BeatPower: This was the rc-candidate algorithm from a couple of weeks back. It calculates a beatwin-beatloss-beatloop record for each team, counting all beatloop teams (including the team itself). Record so far: 97-63.
- Modified BeatPower: This one was a bit easier to explain, in that it only includes the beatloop teams that aren’t otherwise in a team’s beatpaths (whether beatwins or beatlosses). Record so far: 98-62.
- Beatloop-adjusted BeatPower: Last week’s power rankings used this algorithm, as did the college rankings. The rankings appear to be more realistic for college, but I don’t really like this algorithm as much because the algorithm choices feel more arbitrary. Record so far: 94-66.
- Last Week: This is my fallback method, where I don’t really tiebreak at all based on beatpath stats, and simply look at the rankings for the previous week. Record so far: 97-63
This has pretty much shown me that I should ditch the algorithm I used last week, at least for the NFL. If I can’t beat the Last Week algorithm, there’s no sense in rolling it out.
Overall, I think the performance pretty good. King Kaufman has compiled a list of win/loss picks from various experts. So far, we’re better than Ron Jaworski, which isn’t saying much. But it isn’t bad, especially for not looking at lines or home field advantage at all.
I think it can be better though. Now, due to the beatpaths, there’s a real maximum performance to the predictions that is less than 100%. If we only pick teams according to the actual beatpaths, we’d be picking less games. So far this season, the Beatpaths picks are 36-22, about 62%, only slightly better than the 61% for Modified Beatpower. Theoretical max through last week is correctly picking all the rest of the games, for a record of 138-22, or 86.25%. That would be pretty much impossible since it would involve some really crazy and (seemingly) nonsensical shifts in the rankings from week to week – but I do think there’s room to do better.
Maybe not enough to best King’s toddler son, but still.
Update: I did find one that performs at 102-58, but it’s a little bit crazy and hard to explain, so I might not use it.