How to Measure Coaches

In college football, the "winningest" (coach with most wins) is a stupid statistic. It's much more a measure of quantity, not quality. If you coach a long time, even if you suck, you have a better shot at being winningest than a younger high-quality coach. It's a participation trophy.

What matters is your percentage of wins, right? Yes - but it's also true that teams differ in the strength of their recruiting. What I didn't know until I read this article at watchstadium.com is that the strength of each incoming class is measured and ranked, just like teams are. Not surprisingly, there's a relationship: the better your incoming class, the more your team wins.

But it's not an absolute relationship (otherwise, why play the season out?) So if we compare class quality against the final record, this gives us a good way to measure the strength of coaching. For coaches, your input is the quality of your players, and your output is your win-loss record. Coaching is converting talent into wins. That's what you have control over. Not the strength of the rest of your conference, not how screwed up the ranking polls and CFP committee are. (A committee to determine who gets into the playoff? Ever wonder why they don't just use a transparent formula? It's so the bowls make more money. It only has to do with how good the teams are to the extent that if it's too obvious they're actually maximizing for revenue, people will spend less on bowls.)

So if a coach consistently wins less often than the strength of his recruiting class predicts, he's screwing up. If he ends up winning more often than the recruiting class predicts, he's taking silver and and turning it into gold, and he's a good coach.

The article I'm linking to shows scatter plots of programs from the Power Five, plotting number of wins from 2015 through 2018 against average incoming class quality rank for the same period. They don't give an r-value, but the curve does look sigmoidal. This is interesting, because sigmoidal curves often suggest network behavior, and the output here (number of wins) for each team is dependent on other teams. They also point out the relationship is weaker in basketball for a number of reasons including transfers.

On the plot below, the further above the line, the better the coaching. Bottom line: Mike Leach at Washington State is the best coach in the Power 5, with his team finishing 6 wins higher during 2015-18 than would be expected based on his recruiting class. Paul Chryst at Wisconsin, Kirk Ferentz at Iowa, and Dave Doeren at NC State, all tie for second best coach in the Power 5, finishing on average 5 wins higher than their recruiting classes. These are the coaches who are converting talent into wins most effectively.

Which coaches waste the talent they bring in? UNC and Maryland (6 wins lower), along with Nebraska and Texas (5 wins lower.) The full list is available at the article.

For Pennsylvanians: Paterno's Record vs Engle and Franklin

November 2011 felt very much like Pennsylvania's own 9/11, especially to someone for whom Penn State was the closest thing to a religion that they were raised with. Nowadays my relationship to PSU football is like a lapsed Catholic. Sure I don't really follow the season or know the players, but I go once a year and went to the Rose Bowl two years ago - like a Christmas-and-Easter Christian I guess. And just like a Christmas-and-Easter Christian, don't you dare tell me I'm not really a fan! I just follow Penn State in my own way. In my heart!

You can't bring up Paterno without addressing the Sandusky scandal, or make people wonder why you're not addressing the Sandusky scandal. That's completely appropriate. This is a game - a game, like dodgeball - and kids' lives are infinitely more important than guys running around with a ball. Fair enough. Joe knew, and he could've been a hero by immediately going to the police, but he thought that an entertainment group (which is what all sports teams are, period, end of discussion) had an internal moral system that was more important than criminal justice. If there were another school sports team, and one of the assistant coaches was molesting kids, and the head coach not only knew about it but actively discouraged efforts to take it to the police - would you want your kid playing on that coach's team?

Without further ado: as I implied, Paterno is deified as a coach. Is he really that much greater than anyone else? Let's look at the numbers (and by the way, I expect some people to get irrationally upset that I'm daring to do something so brazen as to actually evaluate a coach's performance by numbers - much like religious people don't want the history of their holy text scrutinized too much. "It's just true! Now stop it and don't worry so much about how it was written!") To be honest, part of my motivation here is that I like Franklin - for one thing, instead of Joe's arrogant "I won't run up the score" position, Franklin realizes that his job is to increase Penn State's ranking, and that means run up the score when possible, given the (stupid) college ranking system. I always thought Paterno's take was selfish more than principled, and it certainly cost Penn State ranking spots and revenues (I may write a post about this later), which tends to be what pro-athletics people use to justify the distracting influence of football programs on the colleges that host them. Also, Franklin once coached at good old Kutztown University in old Berks County!

	Engle	Paterno	Franklin
Win %	66.7	74.4	67.9
% Winning Seasons	93.8	90.3	100
Longest win streak	15	21	4
% bowl seasons[2]	25	82	100
% bowl wins	75	64.9	50
% AP ranked seasons	31.3	51.6	50
final AP, when ranked[3]	14.8	8.7	7.5

[1] Note, "longest win streak" means longest streak of winning seasons. A season has to have more wins than losses to be a winning season.

[2] Since # of bowls has increased without a concomitant increase in # of teams, if the team remains the same, it would get higher as time goes on. Case in point, in 2014 PSU won the Pinstripe Bowl but was still un-ranked!

[3] I don't know if there are historical rankings beyond the top 25, which would give a more accurate picture (and I could just use that to give a picture of team quality instead of this and the previous category.) Though I don't calculate it here because it would take a lot longer to round up the data, a better measure might be start-to-end-of-season improvement in rank, beating expectations. This is unlikely for legacy teams with big media markets (think Notre Dame and Penn State) for cynical business reasons. Legacy teams tend to be consistently over-ranked in the preseason and under-perform during the season, ending up lower; for new hotshots in small markets it's the obvious (think Boise State or TCU.) This is the way the NCAA underhandedly seeds the bowls in order to maximize revenues. Since we're only dealing with one team this may not have any effect here, or rather, it only will to the extent that the rating powers-that-be are changing their opinion over time about how much they want PSU in a bowl, and/or the extent to which they're using this technique.


So what does this show? Paterno does have the highest win rate, but a lower % of winning seasons. That suggests that under Paterno, when Penn State was good, they were great, but when they sucked, they really sucked. (Specifically, they sucked in the 21st century, but more on this in a bit.) But the statistical fallacy is that there is more noise in Paterno's history because his tenure was so much longer than Franklin's so far. Franklin gets to more bowls (see footnote #1) but does worse in them.

Taking out the win streak because it's so heavily dependent on tenure, in these categories, Engle wins for % of bowl games he won, Paterno wins on win % and % of seasons ending up in the AP rankings, and Franklin wins on % of winning seasons, % of bowl seasons, and final AP when ranked. That is to say, Paterno was in the ranked teams list a little more often than Franklin at the end of the seasons, but ranked lower.

What does it look like when we compare the first four seasons of all three coaches - that is, when Engle and Paterno were in Franklin's position near the start of their careers, how did they look?

	Engle-4	Paterno-4	Franklin
Win %	62.2	81.4	67.9
% Winning Seasons	75	75	100
Longest win streak	4	3	4
% bowl seasons	0	75.0	100
% bowl wins	N/A	62.5	50
% AP ranked seasons	0	75.0	50.0
final AP, when ranked	N/A	4.7	7.5

Paterno got off to a strong start, again with high win percentage - that doesn't go away. Franklin spreads the wins out more and is more consistent season to season. It's worth noting that Paterno wasn't taking over a still-shocked program that had trouble recruiting.


What about the 2000's?

Paterno had his first losing season in 1988 after a 21-year streak of winning seasons. After that, others followed: 2000, 2001, 2003, 2004. You can see the trend over time here. (Note, his first non-winning season was a tie, not a losing season, and was also his very first as the head coach in 1966.) It may be worth pointing out that when the slide started in around 1998, Paterno was already 71.



It might seem a little mean to pile on a coach after he's gone, when there was no shortage of people in the aughts saying that Joe should retire (and it looks like they were right, for more reasons than they knew.) But Penn State continued its rabid fanbase, and does so even now after the scandal. (Here's the reason why people stick with unpopular or losing teams.)

Future possible project: analyze rankings movements during Paterno's career based on margins, and if there is some estimate of how many points off the margin you lose by putting in (second string, third string, etc.), and you correlate end-of-season rankings with revenues, you can actually calculate how much money you cost a university by refusing to run up the score.