Up 3, final seconds, game on the line, do you foul or not?

2008 NCAA Basketball Championship title game. Unbeknownst to me, Kentucky's future coach is coaching Memphis against Kansas - GO MEMPHIS!!! (I grew up in Missouri, I hate Kansas!) Derrick Rose hits a freethrow, Memphis is up 3 with 10 seconds to go - YES!!! Kansas inbounds, I'm on my feet inches from the flatscreen screaming, "FOUL! FOUL! FOUL!" Memphis doesn't foul, Chalmers sinks a 3, Kansas forces overtime - NO!!! Why didn't he foul!?! Kansas wins - NO, NOT THE JAYHAWKS!!! Right then I wouldn't have spit on Calipari if he'd been on fire.  

Why didn't he foul!?! Later, Calipari explained the Tigers were trying to foul but couldn't get a call. However bad Calipari felt about it, I was devastated ... for all of about 15 minutes and then the dog needed a walk. C'est la vie. Heh, it wasn't the 'Cats. [Saved the NCAA from the later dilemma of whether to vacate their own championship game. The NCAA didn't vacate Duke's Championship over Corey Maggette.]

Now I read an ESPN "College Basketball Nation Blog" piece by Eamonn Brennan that brings memory of that Memphis/Kansas game back to me in all its bitter agony. Brennan has found an empirical study by the Harvard Sports Analysis Collective (HSAC) that concludes that it didn't matter if Memphis fouled or not.

More after the jump.

In the 2009-2010 season, I found 443 instances where a team held the ball down three points during their last possession of a period (either the end of the 2nd half or an overtime period). In 391 of those cases, the team leading did not foul. In 52 cases, the team chose to foul. [...]

Of the 52 teams that committed a foul, six lost the game for a winning percentage of 88.46%. Of the 391 teams that did not foul, 33 lost the game for a winning percentage of 91.56%. Both a two sample t-test of proportion and a Chi-squared test fail to reject the null hypothesis that there is a difference in winning percentage between the two strategies. In this sample, teams that did not foul won slightly more often. For the less statistically inclined, this means that there is no significant difference between the two strategies.

Interesting result, however..... as stated by Mark Twain, "There are three types of liars, “Liars, damn liars, and statistics”(He credits Benjamin Disraeli for the insight). Without casting any (further) aspersions on the integrity of the HSAC, let me point out some things about this thought-exercise HSAC conducted.

First, statistical hypothesis tests like the t-test and Chi-squared test used to support the HSAC's conclusion make vastly simplifying assumptions about the nature of the data being examined. These assumptions must be considered when choosing a test and when interpreting the results. Statistical results generally don't PROVE anything because the data and its characteristics don't normally adhere closely, let alone perfectly, to the test's assumptions. Think of it as using a straight ruler to measure the perimeter of a circular shape. The more curved and irregular a surface the less likely using the straight ruler is going to give accurate results.

In this case, one of the most important assumptions required by the tests comparing these two samples (foul/no foul) is that the samples be statistically identical except for the decision to foul. I'll go out on a limb here and suggest that isn't the case. Team and opponent strengths and weaknesses as well as game situations like who's in the game, fouls to give, fatigue, officiating, timeouts, etc. heavily bias the decision to foul and therefore skew the makeup of the samples (for you statistically inclined, the foul/no foul variable isn't randomly distributed and subject to selection bias) and therefore the tests don't accurately measure the statistical differences between the samples. Ignoring these issues is the way statistics can be used to predict population growth in the USA by examining sidewalk temperatures in Stockholm - absurd but there you are. To be fair, these problems don't mean the HSAC conclusions are wrong, fouling when up 3 in the final seconds may well not make any difference, this study just didn't clear-up the issue.

Nonetheless, I still find the study interesting. First, only 52 out of 443 coaches (11.7%) chose to foul when up by 3 in the final seconds. That fact is really puzzling, because, as pointed out by ESPN's Brennan, not fouling is contrary to conventional wisdom.

[...] the conventional wisdom -- forged over the past few years -- says that teams leading by three at the end of games should foul. The thinking is simple: It's harder for a team to make one free throw, miss it, and then grab a rebound and score than it is for that team to come down the court and make a quick, albeit challenged, three point shot.

So, unless you ascribe to the premise that coaches as a species are ignorant of strategies thought to enhance the chances of victory in general and this particular nugget of conventional wisdom in particular, this suggests the decision to not foul is as deliberate as the decision to commit the foul. Fans, players, ADs and, of course, coaches generally hold the position that coaching decisions make a difference in determining game outcomes. Certainly they are held responsible.

Hey, here's a chance to praise the acumen of coaches as opposed to suggest their decisions are irrelevant. The fact that 88.46% of those coaches who didn't foul won, as did 91.56% of those coaches who did foul, suggests the alternate hypothesis that coaches are able to assess the advisability of fouling given the game situation. Coaches appear to correctly choose to foul, or not when up 3 in the final seconds, approximately 90% of the time with wrong decisions and instances of inadequate execution accounting for the 10% losses. Good job, guys!