rehash: culture and habit by way of regression to the mean
Suiting up for my new job, I’m reading Leonard Mlodinow’s Drunkard’s Walk: How Randomness Rules our Lives, a look at randomness, probability, predictability, notions of certainty, math, and how our brains work for and against us. It’s in synch with the themes driving Super Crunchers and The Numerati, which I cover here and here, but also appears to have some good grounding material in statistics and probablility.
This entry is less about the larger argument of the book and more an idea sparked by something Mlodinow references. (I’ve seen it referenced in other places but it seems I finally got it today.) The story is from Daniel Kahneman, a psychologist who won the Nobel in Economics in 2002. Early in his teaching career, Kahneman was teaching a psych class to Israeli Air Force instructors. On the subject of positive and negative reinforcement, his students strenuously objected to a premise, indicating that their experience teaching pilots was quite the opposite of traditional views:
I’ve often praised people warmly for beautifully executed maneuvers, and the next time they always do worse . . . And I’ve screamed at people for badly executed maneuvers and by and large next time they improve. Don’t tell me that reward works and punishment doesn’t. My experience contradicts it.
So there are two data points the instructors put out:
The truth of the matter is that these two data points are blips in a larger, clearer story. Successful students have a learning path similar to the one in the chart below:
The individual performances or benchmarks will, of course, be scattered around that smooth curve, like so:
The two data points cited by the instructor would be the movements indicated by the arrows. On the left, the pilot had an unusually good performance, followed by one that was more the norm. On the right, an unusually bad performance was followed by a normal one. The unusual performances are actually normal. There are always spikes in performance. When they are followed by more norm-al performances, however, the contrast gets our attention and we look for causal links.
In the book this was a way to highlight “regression to the mean,” the tendency of data points to settle into the larger trends (move toward the mean), and how we tend to over-emphasize the aberrations as significant narratives from which we can learn. It was interesting to me since it highlighted an old saw around slow, accretive progress, the importance of cultural and habitual changes rather than quick fixes. Above, calling the pilot and idiot and calling him a prodigy were ultimately insignificant in the overall flow of the pilot’s education. The accretive, iterative learning process was what did the trick.
It’s also fun since, during a recent flu bout I was re-watching The West Wing and saw the episode “Post hoc, ergo propter hoc”, “after this, therefore because of this”, the fallacious assumption that something that follows an event was caused by the event. As President Bartlett says: “It means one thing follows the other therefore it was caused by the other. But it’s not always true in fact it’s hardly ever true.”