Revisiting Preseason Model Projections

The college basketball season is over. So is the NFL Draft. So is spring football. We’ve got the college world series still ahead of us but otherwise we’re definitely now in the offseason doldrums. Which means it seems like a perfect time to look over some of the information from my college basketball modeling system (and yes, you are still allowed to continue reading without associating yourself with being a nerd).

Back when the Huskies made the switch from Lorenzo Romar to Mike Hopkins I designed this system to try to figure out whether a coach is under or over achieving based on the makeup of their team. The model includes players since the 2012 season who play in one of the football power conferences plus the Big East. I’ve refined the system over time and at this point it includes the following base components: the recruiting rankings of a team’s players, their grade level, and their performance in the most recent season.

I’m sure the first question that many of you have is how well does a player’s recruiting ranking/experience track with their actual performance?

Glad you asked. I’ve divided a player’s recruiting ranking into 9 tiers which have a fairly evident impact on production. The numbers for the boundaries may seem a little too round but even if you shifted things by a couple of spots it doesn’t change much. The borders are at the following spots: 1-5, 6-10, 11-25, 26-50, 51-100, 101-200, 201-300, 301-400, 400+ or unranked. And here’s how it looks when you graph it out using offensive and defensive possession efficiency data from Synergy Sports.

The number above each bar are the number of players in that group. As you can see, the older a player gets the less steep the slope as you move down the recruiting rankings. Almost everyone who is a top-5 recruit coming out of high school heads to the NBA after their first season and the same is true for those in the 6-10 group after their second years. By the time you get to their senior years though just about everyone left at a P6 school who was a top-100 recruit entering college ends up averaging out to be about the same. And that’s at about the same level as a 6-10 ranked freshman.

If you want some context around the production numbers that are in that graph, Quade Green led Washington this past season at 173 which is a little bit worse than you would expect from a tier 4 senior (he was ranked 26th so was also on the highest end of tier 4). Jaylen Nowell had 411 and 429 points in his 2 seasons which meant that he was better as a true freshman than even the average top-5 freshman. The only other Huskies in the last decade to eclipse the 400 point mark were: Andrew Andrews, Isaiah Stewart, Noah Dickerson, and Terrence Ross (Markelle Fultz was at 350 because of some missed playing time and below average defensive numbers).

The model then takes the performance of each player and checks it against the average for a player of their experience/talent level and adjusts it accordingly. Ultimately the output is in the form of KenPom’s Adjusted Efficiency Margin (AEM) which is the best way we have so far to put a number on how good a team is compared to their competition.

If a team finishes much better than the model would predict based on the balance of playing time and the player’s talent/experience/production then it suggests that the coach did a good job of getting the team to over perform. If they finish worse than the model predicts then it suggests the coach did a poor job. Whether the diagnosis for the subpar result is team chemistry, positional imbalance, or a poor scheme it all ultimately falls back on the shoulders of coaching.

2021 Predictions Revisited

Before the season starts of course we don’t know exactly how many minutes everyone is going to play. In order to try to hedge against injuries and highlight the importance of depth I pick the 5 presumed starters and assume they’ll each play 70% of the team’s minutes while 5 backups play 30% each. This provides extra uncertainty for teams that run a 7-man rotation with no injuries but evens out for situations like when Quade Green was ruled academically ineligible and suddenly the rotation is forced to open up.

Let’s start out with a look at what was projected for the Pac-12.

Obviously there ended up being a few glaring issues with this projection. I still think Oregon being projected first was the right call even if they finished 4th in AEM (16th rather than USC at 6th). Starters Will Richardson and N’Faly Dante were both projected starters and combined to play 60% of the team’s minutes rather than 140% (out of 500). Even with the injuries they still finished higher than this projection in terms of absolute AEM.

But most people are probably going to be fixated on those bottom 3 teams. Washington State I had as by far the worst roster in the conference coming into the season and they instead finished basically identical to where I had the UW/Cal/Zona trio. Noah Williams took a leap as a sophomore and Efe Abogidi ended up being drastically better than his recruiting ranking as a true freshman.

In the middle of February Oregon State was about who I thought they would be as they were 123rd in the country and sported a 6-8 conference record. From that point on though they finished 10-3 including a 6-game winning streak in the Pac-12 and NCAA tournaments with all of the wins coming over teams ranked in the top-35 nationally. That spectacular run catapulted them to where UCLA sits on the graph above.

Finally USC went from 10th to 1st in these projections. There are a few reasons for that. The Trojans brought in 5 transfers from traditional one-bid conferences who had success at that level. Those players were all either tier 8 or tier 9 players in the rankings coming out of high school. My model currently only uses past production for players who played at other power conference programs so that it’s a like to like comparison. That’s something I plan on investigating over the summer to determine if it makes the model more accurate to either just include that production despite the lower competition or to include it but with an adjustment. Part of that was also that Andy Enfield was the 2nd lowest rated coach in my system before this season and this is the only time in his tenure that the Trojans have significantly overperformed.

Finally, let’s talk about what happened to Washington in the projections. Washington’s starting 5 of Green, Stevenson, Bey, Wright, and Roberts included a pair of seniors who were tier 4/5, a junior who was tier 5, and a sophomore and junior who were tier 7. There was also raw talent coming off the bench as Brooks, Battle, and Bajema were all top-150 recruits coming out of high school and there was experience since none of them were true freshmen. If you were solely looking at recruiting rankings and experience then the Huskies pretty clearly had a top half of the conference roster coming into last year.

It’s safe to say though that only Green, Bey, and Tsohonis played at an above average level for the majority of the season. Erik Stevenson had his moments and and there were a few single game explosions from guys like RaeQuan Battle, Hameir Wright, and Riley Sorn but otherwise there’s a reason the Huskies finished with just 5 wins.

Using the same Synergy net points totals the Huskies finished within the Pac-12 out of 124 players that played at least 10% of their team’s minutes: 20th (Green), 21st (Tsohonis), 34th (Stevenson), 49th (Bey), 64th (Wright), 76th (Sorn), 80th (Bajema), 88th (Battle), 90th (Roberts), 98th (Brooks), and 99th (Pryor). Coming into the season I had Green 7th, Bey 17th, and Wright 28th. Tsohonis (70th in preseason) was essentially the only Husky to finish better than could have been expected given the preseason information. When out of your 11-man roster you only have 1 player overperform, have about 4 guys finish where expected, and then another 6 play much worse then this is what ends up happening.

Now let’s take a quick look at the rest of the conferences:

ATLANTIC COAST CONFERENCE

I didn’t quite get the ordering right but I correctly called that it would be a down year for the conference as a whole compared to the rest of the power conferences. Duke missed the NCAA tournament but still finished 4th in AEM at 17.56 compared to my projected 18.06. Virginia took a bit more of a leap which should always be expected with Tony Bennett while Florida State vaulted from middle of the conference in my projections to the #1 spot by virtue of most of their cast being slightly better than I thought and RaiQuan Gray improving his FG% by 10 and hi FT% by 6 in his junior year.

Looking at the bottom of my projections I had Boston College and Wake Forest as the clear bottom 2 in the conference and that’s exactly how it turned out as they combined to go 10-32 (5-26). Louisville overperformed where I had them in part because their best player was a grad transfer from Radford and as I mentioned earlier in the article, it’s hard for my system to predict which small school transfers will actually pan out.

BIG 10 CONFERENCE

On the one hand the top-8 teams in my projections for the B1G all made the NCAA tournament so that’s worth something. However, Michigan State did not end up as the clear #1 and the next 3 squads plus Michigan from further below all ended up getting #2 seeds or better. The Wolverines under Juwan Howard had the #42 ranked recruit come in as a true freshman and put up a season that was comparable to Isaiah Stewart which wasn’t exactly a likely scenario.

Meanwhile the Spartans had 7 players who previously had been between 39th and 98th in the recruiting rankings that failed to be anything more than at best an average contributor. That’s over half of the roster that seemed extremely logical candidates to take a leap and none of them did. Nebraska and Northwestern were my clear bottom 2 teams and they did in fact finish as the worst schools in AEM.

BIG 12 CONFERENCE

It was evident that Baylor was a cut above the rest in the Big 12 coming into the year but no one told my projection model apparently. Multiple Baylor players last year overperformed what the model would’ve expected by enough that it essentially expected regression when in reality they just got even better this season. The Bears won the national title in dominant fashion despite having 0 top-50 recruits on their team so it’s not a surprise that a model with a foundation built around recruiting rankings didn’t fully capture their greatness.

The Texas roster meanwhile had 9 players ranked in the top-60 coming out of high school. And it’s not like they were bad they just didn’t quite live up to expectations and then played one of the worst NCAA tournament games I’ve ever seen. I had West Virginia, Texas Tech, and Kansas as teams 2, 4, and 5 and it turns out they did finish in AEM in exactly that order since Baylor and Texas flipped spots. Oklahoma was the biggest overperformer in the conference for me while Iowa State who I thought would be middle of the pack ended up finishing 0-18 in conference which rightly got head coach Steve Proehm fired.

BIG EAST CONFERENCE

There’s a big caveat for the Big East as I had this as a 2 horse race between Villanova and Creighton. It ended up that UConn finished almost tied with Creighton but I didn’t have the Huskies added into the model yet in their first year back in the Big East. I would’ve had them 3rd but pretty much in the middle between Villanova and Seton Hall.

The only other major flaw in the projection was that Georgetown and DePaul should’ve essentially been flipped. The Hoyas lost basically all their top players to transfer the previous year but managed to rebuild on the fly thanks to some internal talent taking big leaps plus some grad transfer pieces overperforming. However even after winning the Big East tournament they still just finished 5th in the conference in AEM.

SOUTHEASTERN CONFERENCE

Similar to the B1G my model correctly picked out who the top-7 teams were going to be but the order got a little jumbled. Alabama and Arkansas both took massive leaps in their 2nd year under their new head coach to create a new tier despite my projections having them 5th and 6th. Among the others near the top I had the order as Florida, LSU, Tennessee, Kentucky when due to the Gators poor showings it ended up LSU, Tennessee, Florida, Kentucky with Missouri jumping up to slot in just ahead of John Calipari’s bunch.

Kermit Davis continues to be a wizard as he took what looked like perhaps the worst roster in all of power conference basketball based on playing time and turned it into an NIT team. And that’s with their best freshman being a complete 0 on offense.

OTHER

My model generally only looks at the Power 6 conferences but by popular demand after the season I went back to add in Gonzaga and see where they fall. This past season my model showed the Bulldogs as having an AEM of 26.6 which would’ve made them my #1 team in the country by a fairly wide margin. They still finished nearly 10 above that with a final total of 36.48 to lead the country even though they completely fell apart in the national title game.

The only non-Gonzaga team from outside a power conference that was in the preseason AP Top 25 this past year was Houston which makes it reasonable to do a comparison for final performance. I took the 25 highest ranked teams in my projections and put them into a Top 25. I then compared both that and the preseason AP Poll to the final KenPom rankings and looked to see how far away each one was from their final placement. The results?

About the same. The average team in my projections was off by 16.1 points compared to 15.4 for the AP Poll. Both were off by a median of 11 spots while the standard deviation from my system was a little bit closer. The big outlier from my system was NC State who I had as essentially 19th in my rankings when they ended up 71st. Meanwhile the AP Poll had Arizona State at 18th overall who finished 86th.

Next week I’ll be back to talk about how this past season’s results changes (or doesn’t) the feelings of some coaches including UW’s Mike Hopkins.