Kentucky Football: What is the best way to rate and rank recruits?

I think the coaches rankings and ratings are different from the "experts" who work for the ratings services. After all, the ratings are based on the opinions of those who rate and then rank. RBO uses the opinions of coaches through their offers. Personally, I will take the opinions of the coaches over the recruiting gurus opinions.

I have been spending time on the Rank By Offers (RBO) website trying to analyze the methodology. RBO rate(s) players on the number of offers received. Does that make more sense than opinion based ratings? On the surface it would seem to me that coaches know what they want in a player and then go after the players who they think can best help their team. RBO also considers the success of the teams making offers by using a three year average of Sagarin's Elo-Chess ratings as a weighting measure. See how it is done here. You can read about the proprietary methodology in the link in the opening above. I think the weighting and the methodology are both fair.

There is certainly a difference of opinion among the "experts" who rate the players and I am absolutely positive that the ratings are agenda driven. If the player attends one of Rivals' events, Rivals is going to give that player a higher rating. The same goes for 247 Sports, ESPN and Scout. The excuse is that if the player doesn't attend their event then they can't properly evaluate him. Then there are other opinions that don't match what the coaches or competitions say.

The 247 Sports data base of players is up to 5011 players. I maintain it is impossible to evaluate that many players which is why you see so many players with NRs (Not Rated or Not Ranked) beside a player's name. RBO has ranked 2,579 players who have at least one offer from a Division I school.

Here's an example of the disparity of opinion:

Joe Mixon has more offers (49) than any other player in the 2014 recruiting class. So, RBO has him listed as the number one player in the country based on offers. He also has more offers from Top 25 schools than all the other players.

There are a lot of coaches who want this RB from California, but the ratings services aren't so impressed. Rivals has him as the 15^th best player and the 4^th best running back behind Leonard Fournette from Louisiana, Sony Michel from Florida, and Elijah Hood from North Carolina. Scout has him as the 16^th best player in the land and #5 at his position. ESPN shows him at #137 overall and #9 at his position. 247 Sports, as you may or may not know, offers two ratings: their own and a composite rating that includes the others above as well as their own. I guess they don't trust themselves all that much. Anyway, they rate Mixon as the #37 player in the land and the #1 all-purpose back. Their composite shows Mixon as the 45^th best player overall and the #1 APB.

As an aside, 247 Sports doesn't include our Stanley "Boom" Williams in their top 247 players even though he has 16 offers. Rivals lists Williams at #96 overall and the #1 APB. I guess Williams didn't attend any 247 Sports sponsored events. How else can you explain the disparity between the two Rankings? The question, then, is who to believe?

I have chosen to use Rivals rankings this year and 247 Sports profiles in my articles concerning Kentucky's recruiting. I intend, however, to analyze the Rank By Offers methodology and may change how I do things next year. After all, offers and who have made the offers are an indicator of a player's value.

The team rankings, I believe, are also biased and agenda driven. Rank By Offers has Kentucky's 2014 class #3 in the country, Rivals has us at #2, ESPN at #12, Scout at #8 and 247 Sports at #10. So, who is right? It may just be me, but I see ESPN and 247 Sports having a bias against Kentucky (see Stanley Williams above). I submit that both ESPN and 247 Sports have their favorites.

Rank by Offers rates every player from the recruiting services that hold offers from Division I schools, but only publishes their top 250. Drew Barker isn't in this group because he "only" has 15-16 offers and is ranked #307. Based on the methodology, I say "fair enough." For me, that poses a problem since he's supposedly one of the top 11 quarterbacks and has participated in the Nike Elite 11 events and came out placed at #7 by the coaches involved in the event. Could the 15-16 offers be a result of other schools knowing they didn't stand a chance? Ohio State was recruiting him, but never offered. Why is that? Did they know that he was most likely to go with South Carolina or Kentucky and figured what's the use? Kentucky needed Barker's early commitment because it was instrumental in obtaining the commitments that followed.

Ever hear of this guy? He's the top QB, according to Rank By Offers. Ulmer is the #66 player in the nation and has 27 offers. While Ulmer is listed as a QB, he's such a versatile player that he can play any number of positions. Hence, the large number of offers. Here's a list of the Top 10 QBs by rank from RBO: (Rank, Name, Commitment)

#66 William Ulmer (Maryland): 27 offers total, 7 from the Top 25 schools

#108 Ralph Peavey (Arkansas): 23 offers, 6 from the Top 25 schools

#130 DeShone Kizer (Notre Dame): 21 offers, 7 from the Top 25 schools

#167 Kyle Allen (Texas A&M): 19 offers, 7 from the Top 25 schools - #2 in the Nike Elite 11 competition

#172 Deshuan Watson (Clemson): 18 offers, 7 from the Top 25 schools - #10 in the Nike Elite 11 competition

#176 Brandon Harris (Uncommitted): 18 offers, 7 from the Top 25 schools

#307 Drew Barker (Kentucky): 15 offers, 1 from the Top 25 schools - #7 in the Nike Elite 11 competition

#308 David Cornwell (Alabama): 14 offers, 5 from the Top 25 schools

#342 Jacob Park (Georgia): 13 offers, 5 from the Top 25 schools - #5 in the Nike Elite 11 competition

#351 Foster Sawyer (TCU): 13 offers, 5 from the Top 25 schools

Sean White (Uncommitted), the winner of the Nike Elite 11 competition is ranked #394 by RBO and has 14 offers, none from the Top 25 schools. Here's the list of the Elite 11 via twitter.

So, while I think RBO is on the right track, they haven't yet nailed down all the variables. And, I don't like using ESPN because they are very slow in updating information. Using 247 Sports would be better because they are sometimes even faster than Rivals with their updates. That being said, the kids view the ESPN ratings and rankings as having a lot of importance.

I have been in contact with RBO's owner/operator, Paul Nelson. Paul was kind enough to respond to my concerns:

Since the first offer-based ranking was published a little over a month ago, I've seen a lot of debate about recruiting rankings on message boards across the college football world. One thing is clear ... there is very little respect for the opinion-based rankings. Almost everyone agrees that they are distorted by bias. Opinions on offer-based rankings vary widely. Some are strongly in favor of having an objective measure. Others believe the approach is too flawed to ever be of use.

The biggest problem, in my opinion, is a temporary one ... data quality. The "self reporting" method by which offers are made public leads to a variety of problems. Some believe recruits claim offers that they really haven't received. I think recruits may exaggerate their total number of offers, but I doubt many publicly claim specific offers they haven't received. I know, however, that many recruits don't diligently report their offers. There is little incentive for them to do so at this point. If offer-based ranking does gain popularity, there would be an incentive and the problem would be resolved. I've already seen this occur, on a small scale, with the recruits who have contacted me directly because they weren't being given credit for all of their offers. Almost all of them worked with Rivals and ESPN to add the missing offers.

Another issue with data quality, as you pointed out, is that the services maintaining the offers aren't doing a very good job. None of their lists are complete. I started by gathering Rivals data alone, until I realized that they only show a limit of 200 offers per team, and that ESPN had many offers that Rivals didn't. I decided to supplement the Rivals offers with additional ones from ESPN. I'd like to add 247Sports as well, but their data is presented in a format that is difficult to extract.

I would think a three year study might help with a weighting by position and also how accurate the services are with how many offers a player has. As you can see in the two charts below there is a position distribution issue that may need to be addressed, and you have to ask why these services can't get their offers straight. People actually pay money for this kind of information?

Paul admits there are problems with early offers as I noted above. It might discourage additional offers. He is searching for a way to take that into consideration. And, he is also looking at a way to take into account offers by position, but isn't positive that is needed.

Some have said that the ranking should be adjusted because some positions are in more demand than others. That depends upon what the rankings measure. The traditional rankings seem to be attempting to measure a recruit's ability to play their specific position alone. The ranking by offers measures something slightly different ... perceived value to college coaches. Athletic ability is certainly an important consideration for coaches, but there are many other things they must take into account. One consideration is positional need for their particular teams. If every team has a dire need for skilled CBs, then proven CBs are going to be more valuable to college teams. They are being rewarded for excelling at a particularly difficult position to play. I don't see a need to try to undo any increased positional valuation the college coaches have assigned. This may represent a slight departure in thinking from the traditional rankings, but it makes more sense to me. Besides, if you want to see the ranking within stated position, that can still be accomplished as you've done in your analysis.

This table shows the distribution by position and the second shows the disparity between the services in the number of offers reported. Paul was kind enough to send me what his database shows for our commitments.