Numbers Can’t Measure Everything That Matters

If you follow baseball closely, you may have noticed there is a bit of a dust-up sometimes over baseball stats; specifically, if they are applied correctly to make decisions about the value of certain players. One of the main arguments is about players who compile prodigious regular season stats, especially the glamour numbers like HRs, RBIs, BA, OBP, and slugging percentage, and then fall far short of those numbers in the postseason.

The camps fall into two basic categories: those who basically think stats are the ultimate and final authority on questions about value, and those who think stats are definitely useful when applied correctly, but can be overused, or mis-applied, especially by those who don’t fully understand the weaknesses of statistics.

Let’s call them … “stat freaks” and “sane folks”. Heh. KIDDING!

The “stat freak” argument about post-season stats — and I’m trying to be fair and square here, because it’s a valid point — is that the sample size is usually too small to be statistically significant. I.e., if every player could, in theory, get enough at-bats in the post-season, they would all end up with basically the same hitting stats as they have during the regular season. “Tending to the mean”, I think it’s called. Stated another way, since most players who get to the postseason don’t get anywhere close to the number of ABs they’d get in a full season (600), or even a third of a season (200), we’re judging the great bulk of them based on the equivalent on the equivalent of a couple weeks of ABs, or maybe a month. And this makes the conclusions inherently unreliable.

And because of these small sample sizes, therefore: there is no way to prove or disprove the idea that “clutch” hitting, or peforming under pressure, exists. Stated another way, the main (only?) thing holding back all players from attaining their regular season equivalence in their postseason accomplishments is the ability to get to the plate enough times for their stats to become statistically significant. I can’t vouch for the truth of this, because I haven’t done the research — nor would I, as the outcome wouldn’t be interesting to me — but others have, and so to some degree at least, it must be considered valid. So let’s grant these premises, for purposes of discussion.

But let’s also turn this around, and look at it another way. Instead of focusing on sample sizes and stats, let’s look at people.

Let’s imagine that some players get nervous in the post-season, and will “choke” (to use the frankest possible term here, for brevity) quite often, especially in their early ABs (first 50-100, let’s say, just as an example). Let’s also imagine that some other players don’t get quite as nervous, but still do to some degree, and that still other players don’t feel such nerves at all, and are in fact precisely where they want to be in pressure situations.

Let’s further suppose that as time goes on, year by year, and as these first two groups of “chokers” acquire more and more post-season experience, they learn and adapt: i.e., how to suppress that emotion, or focus better, or tune out the crowd, or they just get more comfortable, or any number of things that help them perform better. In other words, they adapt. Plus, just to state the obvious, they’re getting older and (we hope) smarter along the way, which might contribute to their ability to adapt, as well. And they’re usually making more money, some of them insanely huge amounts of money, \$15M per year or more; this can sometimes put extra pressure on a player.

Once they get past 200-250 at bats, let’s say, some of them are finally as comfortable as during the regular season, and their hitting stats rise steadily along with that confidence, to the point where their postseason stats are now mostly in line with their regular season stats.

Now, to the outside observer, what is the difference between this scenario and the “tending to the mean” scenario above? Nothing at all, that I can see: there are no independently observable data points that prove it was either one scenario or the other. The only variables are internal to the player in question. And therefore, unquantifiable.

So, both explanations can fit the exact same data points, in other words.

And when multiple explanations can fit the exact same data points, they must be considered equally likely.

Stated another way, the fact that players tend to their career regular season stats after accumulating enough at-bats doesn’t prove or disprove anything about clutch hitting performance, if one accepts the premise that some players can adapt to the unique demands of the postseason. And other players, conversely, will never adapt, for various reasons.

Note that what is really in question here between the two scenarios is the assumptions about an unquantifiable event: what causes some players to have poor post-season performance in their first, say, 200 at bats? Is it just bad luck, or a slow start, like some players have every April and May? Is it “choking” under pressure? Is it extraneous annoyances, like dealing with the media, and family and friends who demand more of your time, possibly causing some players to lose their edge? Is it a continuation of the resumed dominance of pitching over hitting, with the return of cooler weather in September, into October? Is it some combination of all of these?

We don’t really know; we can’t know. All we do know is that some players don’t perform in the postseason as well as they do in the regular season, given their records as they currently stand. And any or all of these things could be factors. And the fact that these things are unquantifiable, and therefore stats simply cannot measure them, doesn’t mean they do not exist.

Players talk about such things often, and to suppose that they are talking about things that feel real to them, but yet since we can’t quantify them, they must be discounted, is simply nonsense. They are the ones on the field, actually being asked to get that hit to drive in the tying run, rather than sitting at their computers, like us, analyzing the exploits of others. I’ll go with their take on this one.

So to my mind, this means that the above “people” explanations are not just possible, but in fact, all probably do happen, to varying degrees, to just about every player in postseason play, in every sport.

Here is a truism: some players have no issues with pressure, and may even play better in big games; other players might have some issues with pressure, but adapt quicker than still others who may never learn to deal with the pressure.

And how do we know this? Because we are human, and so are they, and we know that we all have, to varying degrees, emotions that affect our ability to perform when the bright lights are on, and also to varying degrees, ability to adapt to new situations.

Ironcially, the inability to prove any of this with stats seems to prove just one thing: stats can’t prove some things we know to be true. And this is because the stats we’ve invented are not a perfect model of a sport.

In fact, I’d go so far as to say stats can never be such a model. As long as people are the ones playing the games, and as long as people have emotions, there are going to be factors that affect their performance that we can never measure, or likely even ever know about. This seems semi-obvious, I guess, but it is also a truism. And, a very good thing; who would want to watch a sport where stats rule over human emotion everywhere and always? Not me.

So. I’d be curious to hear the refutation of this argument by “stat freaks”. Note that I’m open to persuasion on this; if somebody can prove to me that human variables — that we all know everybody has, and that are internal to the mind and body of an athlete — are somehow accounted for, and controlled for, when evaluating external variables like batting stats, I’ll be happy to look into it.

Until such time, my take is that excessive devotion to statistics causes some to over-value those things that are quantifiable, and under-value those thing that are not. Which doesn’t make them wrong all the time; it just means that when the only tool you have is hammer, everything looks like a nail.

But that’s OK; statisticians do what they do, and that contribution is valuable, as far as it goes. But that isn’t quite far enough. And I haven’t even gone into the other weaknesses of stats, such as the inability to effectively measure fielding, or throwing, or baserunning, for various reasons. But that is a discussion for another day.

And as long as real people with real emotions are playing the games, stats will never be enough.

And that’s just exactly the way it’s supposed to be.