Pac-12 ESPN FPI Prediction Accuracy So Far This Season

Over half of the games for the Pac-12 this season have now been played (51 out of 90), even though some teams have not yet played half of their games. I’ve been tracking ESPN’s FPI projections throughout the season and thought that this would be a good time to take a look to see how their projections have fared so far. I’ll also take a look at their projections for the rest of the season.

For ESPN’s FPI projections, I’ll look at them in two ways. First, I’ll look at their projections before the season started (preseason predictions). And because they update those projections each week, I’ll look at how well they’ve adjusted those projections (weekly predictions).

Preseason Predictions

Here is how ESPN’s FPI did with their preseason win projections for the games played so far.

That is 43 correct games and 8 wrong, or an accuracy of 84%. (You can’t just use the numbers from the table above or else you’ll double-count the games between Pac-12 teams.)

The one team that stands out here is obviously Oregon State. But when you look at their data closely, their accuracy doesn’t look as bad as it seems. Here is a look at all of the games that they got wrong.

• Arizona at San Diego State. ESPN’s preseason FPI projected that Arizona had 32.9% win probability.
• Eastern Michigan at Arizona State. ESPN’s FPI had ASU with a 90.5% win probability.
• Boise State at Oregon State. Oregon State had a 49.8% win probability.
• Oregon State at Fresno State. In this game they gave Oregon State a 48.4% win probability.
• Oregon State at Stanford. Oregon State had a 43.5% win probability in the preseason.
• Utah at UCLA. UCLA had a 41.6% win probability.
• Michigan State at Washington. UW had a 39.3% win probability before the season started.
• Washington State at Wisconsin. WSU had just a 11.2% win probability.

Half of their misses were when the team that won had a 40-50% chance of winning. That should be expected to happen occasionally (almost half of the time)-that’s why it is 40-50% rather than lower (like 0%).

To answer the question of whether those misses were within the expected range or not (too many misses), I looked at the win projections for the games and whether they fell into the expected win percentages. Below I’ve compiled all 51 games and grouped them by the projected win percentages for the favored team.

As you can see, so far they have been fairly consistent with their projected ranges. In one case they were in the expected range (90-100%), and in 2 cases they were at one end of the range (50-60% and 80-90%). The case where they were off the most was when they had the fewest games. The one where they were within the middle of the range was the one which had the most games.

Weekly Projections

Each week ESPN updates their projections based on the games that had been played. Theoretically, this should provide more accurate results since they will be relying more on this season than the previous season. Let’s see how they did.

These numbers are different than above because they changed who the favorite was based on previous games. For example, Oregon State had been the underdog to Fresno State in the preseason (48.4%), but after they beat Boise State, ESPN changed them to be the favorite (52.2%).

As you can see, even after adjusting, they missed on 3 of UW’s games (Michigan State, UCLA, and Arizona State).

Surprisingly, their accuracy was identical: 43 correct and 8 wrong or 84%. They flipped the favorite from the preseason on 2 games where they ended up being correct (Oregon State at Fresno State and Oregon State at Stanford) and 2 where they ended up being wrong (UW at UCLA and UW at Arizona State).

I will note that I included the games from the first week in these numbers-so projections which were not based on any games played this year. However, if you look at just the games since the the first week, the accuracy is about the same: 83.7% instead of 84.3%.

Here’s a look at the same breakdown as above for the win projection ranges.

As you can see, they did better than expected in every range except the 90-100% range, but that was off by only by a small amount.

And if you are wondering, WSU had an 11.2% win projection in the game against Wisconsin in the preseason and that moved to 7.1% the week of that game. That is the reason that the one wrong projection in the 80-90% range is no longer there and why there are two wrong projections in the 90-100% range.

Pac-12 Games

An argument that I’ve heard is that the out-of-conference games, which take place early in the season, are mostly one-sided, so ESPN’s accuracy is skewed because of all of those easy games. But let’s look at just the Pac-12 conference games-most of which have taken place over the last 3 weeks.

There have been 18 Pac-12 games so far. Looking at ESPN’s preseason predictions, they got 16 of the 18 correct; 89%. The 2 that they missed were Oregon State over Stanford and UCLA over Utah. Looking at their weekly predictions, they got 15 of the 18 correct; 83%. Neither value is significantly different than for all of the games, but that will be something to watch as the season progresses.

Future Win Projections

Here is a table which shows each Pac-12 team and the number of games where they are favored and ESPN’s projected wins for each team.

Without going into all of the details for all of the teams, here is a quick summary of the current future win projections for each Pac-12 team.

• Arizona has a less than 12% chance of winning each of their next 4 games. They had been favored over WSU in the preseason, but now are underdogs (36.4%). They were underdogs against ASU to start the season, then moved to be the favorite after week 3; now they are slight underdogs again (44.5%).
• Since the preseason, Arizona State moved from favorite to underdog against Stanford (45.1%), WSU (29.4%), and Oregon State (45.2%). The only team, besides Arizona, that ASU is favored over is Colorado (80.9%).
• California was an underdog to Colorado in the preseason, but now is the favorite (82.2%). They are also the favorite against Stanford (61.3%).
• Colorado does not have above a 20% win projection against any remaining team. Their best chances are home games against California (17.8%) and Arizona State (19.1%).
• Oregon is favored in all of their remaining games. After week 1, they had moved to be the underdog against Utah, but now are a slight favorite (51%).
• Oregon State is an underdog to UW (30.5%) and Oregon (32%), but favored in their other games. They had been the underdog against Arizona State in the preseason, but now are a slight favorite (54.8%).
• Stanford was an underdog against Arizona State in the preseason, but now is a slight favorite (54.9%); that is the only game where they are currently the favorite. They were favored against WSU in the preseason, but now are a slight underdog (46.4%).
• UCLA had been favored over USC in the preseason, but now is the underdog (38.1%). They are favored in all of their other games except this weekend against Oregon (28.4%).
• In addition to the UCLA game, USC is now favored over Notre Dame (68.4%). Those are the two differences from the preseason.
• Utah is favored against all of their remaining opponents except Oregon (49%).
• Washington’s win projections for their remaining games are virtually unchanged since the preseason. It improved by 16.6 points against Colorado, but there is a less than 8 point difference in their remaining games (all improved, except for Oregon). They are favored in all of their games except against Oregon (29.5%).
• ESPN’s preseason projections had WSU as the underdog in all of their remaining games. But now they are favored in 3 of the 6: Stanford (53.6%), Arizona State (70.6%), and Arizona (63.6%).

This Weekend’s Game Projections

There are 5 games this weekend. Here are the projections for these games including the win projections from the preseason and their current projections.

Except for the California-Colorado game, there is not much difference in these games between the preseason and the current projections.

Conclusions

I’ve heard some Husky fans say that ESPN’s FPI relies too much on the previous season. While that may be the case, so far it has not affected their accuracy. Yes, they missed some games (like WSU-Wisconsin and UW-Michigan State), but they’ve been correct much more often than they’ve missed-even on the conference games.

There are 38 games remaining in the regular season, including 3 more out-of-conference games (Stanford-Notre Dame, USC-Notre Dame, and Stanford-BYU). Currently there are differences in which team is favored now compared to the preseason in 7 of them: WSU at Arizona (now WSU), Arizona State at Stanford (now Stanford), Arizona State at WSU (now WSU), Oregon State at Arizona State (now Oregon State), California at Colorado (now California), WSU at Stanford (now WSU), and USC at UCLA (now USC). We’ll have to see if their preseason projections or their weekly updates end up being more accurate.

Obviously no team should take any game for granted. Even teams that had less than a 10% win projection have won, and there are 8 Pac-12 games where that is currently the case (most involving Colorado). We can expect that there will be more wins by the projected underdog. Injuries and bad bounces that effect the outcome of games happen all the time. But hopefully they only miss on the UW-Oregon game!