Talent, Experience, and the Pac-12

The firing of Lorenzo Romar has led to a number of conversations about the relative worth of talent and coaching when creating a winning college basketball team. The primary reason behind the change was that there was an increasing gulf between the level of talent coming into the program and the level of coaching and player development happening. In many people’s opinion, great player after great player (Tony Wroten, Terrence Ross, Dejounte Murray, Marquese Chriss, Markelle Fultz) were entering the UW program and leaving without actually contributing towards wins. While having NBA alumni isn’t a bad thing, unless you’re John Calipari it shouldn’t define your program more than the actual results on the court.

However, there was also a feeling from many that 2017-2018 was setting up to be a landmark year due to the mix of returning experience and exciting new talent. While many teams of the last 6 years had star power, some misses in recruiting led to a severe lack of depth and veteran leadership that could potentially have turned things around. The comment section has been upset about the argument that may or may not have been bandied about by our fearless (and I do mean fearless, I wouldn’t have ventured into those comment sections willingly) leader that elite level talent often trumps shortcomings in coaching. What then is the relationship between recruiting talent and on court success?

Methodology

First, we have to define what we mean by those two things. We’ll start with on court success. Many people view wins as the be all end all for determining performance. After all, the point of playing is to win games. That can be a little misleading though. There’s a big difference between a 20 win team that goes 12-0 in the non-conference playing Coppin St. and UC Riverside and 8-10 in Pac-12 play and one that goes 6-6 playing in Maui and the Battle for Atlantis and then 14-4 in conference.

College basketball also has the problem of postseason play. If a team goes 24-8 with a close loss in the 1st round of the NCAA tournament, is that worse than going 25-12 and losing in the finals of the NIT? Most people would rather the NCAA berth than the NIT run despite having less wins. As a compromise, I’m going to use kenpom.com’s final adjusted efficiency margins to judge the success on the season. For those unfamiliar with the system, they measure how many points per 100 possessions a team is better than an average opponent. A score of 30 is usually a top-5 squad, 20 is a top-20 team, 10 is about a top-75 team, and 0 is is around a top-175 team.

Determining the true talent level of a team is difficult. There are countless examples of 2-star players becoming all-conference talents (Andrew Andrews) and 5-star players who struggle mightily their entire careers (Abdul Gaddy). Nonetheless, recruiting rankings are the best clue that we have. In order to try to estimate a player’s talent level I took the average of their ESPN and Scout star rating. For players only evaluated by one of the services I went with that rating. For junior college transfers I gave a star rating of 3-stars unless there was a rating on them coming out of high school that was different. This isn’t a perfect approach but the best I could do.

Finally, we’re adjusting for playing time. To create the final number I took the average star rating and multiplied it by the % of minutes they played during the season. Having a 5-star player on your roster doesn’t help if they’re on the bench. So if that 5-star player plays 20 minutes per game and a 2-star player plays 20 minutes per game, together they would average out the same as a 3.5-star player who plays all 40 minutes. Only players participating in at least 10% of total minutes (4 minutes per game) are included. To make up any gaps in minutes remaining I’m pretending it’s a 2-star player for all of the times when the bench is emptied and the walk-ons come in.

Here’s an example of what this looks like for the 2016-17 Washington Huskies.

To get the final talent rating I divided each player by 500 (5 positions * 100% of minutes) which makes the total digestible. This year’s Huskies ended with a Talent Rating of 3.69. On average, when a player was on the court for the team they were somewhere between a 3-star and 4-star player. Since 2011, there have been 82 Pac-12 basketball seasons (Colorado and Utah joined in 2012). This year’s UW team ranked 14th out of 82 in terms of pure talent on the court based on recruiting rankings. I didn’t go back and check but I’m pretty sure that more than 14 teams have had more than 2 conference wins during that time. Having talent certainly didn’t help the Huskies this year. What is the correlation then between efficiency margin and talent?

Pac-12 Recruiting Talent vs. Kenpom Rating

The r-squared value ends up being 0.33. That means that about 33% of the variation in final efficiency margin can be explained by the difference in talent on the floor via our system. The purple dot to the upper left is this year’s Fultz-led team. This method doesn’t take into account though one of the big things that factored into the arguments of those who wanted to give Romar another year. And that is experience.

While I only calculated values for Pac-12 schools, under this methodology Kentucky would be expected to be the #1 team in the country every single year. Having a roster full of 4 and 5 star players is great but a senior laden roster full of 3 and 4 star players is often just as good. Now, let’s take the initial talent ratings and add in an adjustment for experience. Freshmen get their regular star-rating but sophomores get 0.5 stars, juniors 1.0 stars, and seniors 1.5 stars. Under this system, a 3.5-star senior is just as good as a 5-star freshman.

Pac-12 Experience-Adjusted Talent vs. Kenpom Rating

We explain a little bit more of the variation this time around. 39% of the variation is explained through a combination of initial recruiting rankings and the class standing of the players on the court. That’s not completely irrelevant but isn’t enough to make you want to go to Vegas.

So how would a hypothetical 2017-2018 team have looked if Romar had stuck around? I made the assumption that Matthew Atewe and Dominic Green both would have transferred this off-season. Atewe was buried on the bench and Green looked completely lost by the end of the year so they seem reasonable assumptions. I then carved up minutes to reflect what likely rotations would have been. I tried to stay conservative and left the current starters with similar minutes, replaced Fultz with Porter Jr., and divided up remaining minutes among the trio of new guards mostly. Next year’s team would have had an experience-adjusted star-rating of 4.5.

That number would have qualified for 16th out of the now 83 teams in the sample. This year’s team was 31st under the experience adjustment method. The worst finish among any of the other teams to finish at a 4.5 or above was this season’s Stanford team (by a considerable margin) who went 14-17 and ended up 103rd in kenpom’s rankings. The Cardinal had 6 upperclassmen in their 9-man rotation and only one of those 9 players wasn’t rated at least a 4-star player by either ESPN or Scout. If we were to take the teams between a 4.3 and 4.7, the average kenpom efficiency margin was about a 12. This year, Cal finished with a 12.13 which qualified them as a #1 seed in the NIT but wasn’t good enough to get them into the big dance as an at large.

Conclusion

After all of that, what do we know? There is some relationship between a team’s level of talent coming out of high school and how they perform on the court. However, that relationship isn’t very strong on its own. It is improved by factoring in experience but there’s still a lot of unknowns to create the perfect formula including scheme, fit, talent development, and in-game coaching.

This year’s UW team certainly under-performed and likely should have been at least a bubble team based on their talent and experience combination. Had Romar been retained it’s clear next year would’ve been the most promising team on paper since the 2011 squad which got a #7 seed but was ranked 16th in kenpom’s efficiency metrics and barely lost a psuedo-road game against UNC. While most fans think it was a guarantee that a 2017-18 squad would have vastly under-performed once again, the 2016 team clearly overachieved by a decent amount using this methodology. Even with an average level coach, it was no guarantee that next year’s team would make the tournament but the expectation going in definitely should’ve been a top-5 squad in the conference.

Ultimately, about 60% of a Pac-12 team’s success is based off of what most people would put under the bucket of coaching. Clearly there are mis-evaluations by the recruiting folks but star ratings aren’t completely useless. Experience and talent are necessary if you want to compete for a Sweet 16 every year. Only 13 teams have made the Sweet 16 out of the Pac-12 in the last 7 years. 8 of them had a better experience-adjusted talent rating than the 2017-18 Huskies would have even with their banner recruiting class. Having a great coach is essential to winning at the level that UW aspires to but that only gets you part way up the mountain on its own.

A couple other random observations:

• Arizona and UCLA are quite clearly the class of the Pac-12 when it comes to a recruiting standpoint. Of the top-10 experience-adjusted seasons, 5 belong to Arizona, 4 to UCLA, and 1 to Stanford.
• However, there’s a little diversity when it comes to the actual finish on the court. Of the top-10 kenpom finishes Arizona still comes in 1st with 4, UCLA and Oregon each have 2, and Utah and UW had one apiece.
• The conference is very clearly one of haves and have nots. Wazzu, Oregon State, Utah, and Colorado have never had a season of at least a 4.0 experience-adjusted rating while Cal, Arizona State, and USC have only had 2 each.