Experience and evidence

From Janet Maslin’s review of Michael Specter’s Denialism: How Irrational Thinking Hinders Progress, Harms the Planet and Threatens Lives, in the NYT of November 5:

What bothered [Specter] more than Dr. [Andrew] Weil’s advice [about taking an assortment of pills for various conditions] was Dr. Weil’s philosophy. “The idea that accruing data is simply one way to think about science has become a governing tenet of the alternative belief system,” Mr. Specter writes. And the additional idea that the evidence of experience is as important as the results of meticulous scientific testing is, in Mr. Specter’s view, one of the most dangerous forms of denialism, especially when it comes from a figure of Dr. Weil’s stature.

Advice literature of all kinds suffers from this inclination to fall back on personal experience rather than seeking evidence about the matters in question. This is an all-too-human tendency, with a variety of sources; in particular, our own experiences (and impressions, associations, etc.) and those of our acquaintances and people we’ve heard about are much more easily accessible than accurate assessments of the facts, which can take an awful lot to investigate. Worse, even our recollections of of our own experiences are colored by interfering effects of many kinds, including several kinds of selective attention and reshapings of memory. And our reasoning about events in our own experience is often flawed in many ways: we take post hoc to mean propter hoc, we affirm the consequent, we take an individual to represent some group they’re a member of, we suffer from confirmation bias, we search for single causes, we generalize from a few anecdotal examples, and so on.

So ordinary people will fall back on personal experience, impressions, anecdates, and speculation. People who set themselves up as givers of reliable advice should do better than that, especially if they display professional credentials. (Andrew Weil does both, which is why Michael Specter is so enraged by what Weil writes.)

These issues come up repeatedly in discussions of language, notably when someone asks, in a serious forum, about language, about the use or history of some  expression or construction. Often these requests explicitly ask for an authoritative answer, but what they mostly elicit is a kind of extended bull session of opinions, beliefs, personal anecdotes, and guesses — plus reports of what the responders had been told by someone (a friend, a teacher, or, alas, an advice writer).

(Such responses are themselves worthy of study, as data of folk linguistics. But as they stand, they aren’t data about language use or history. They’re metadata.)

Sometimes I can respond to such queries quickly, because I’ve studied the phenomenon in question or have easy access to research on it, (Even so, my responses are sometimes rejected outright, because the earlier writer says that’s not their experience — they know what they know — in which case I just have to give up.) More often, the questioner is asking about something whose details are largely, or even entirely, unknown, and the query serves as a request to start a study. Unfortunately, research on such questions is enormously complex.

4 Responses to “Experience and evidence”

  1. Rick S Says:

    There is, after all, survival value in generalizing that all tigers are dangerous after watching one eat your neighbor. We’re only human.

    But your (and the Log’s) occasional reminders that personal experience and speculation are poor substitutes for evidence have not been ignored. I comment far less often than I consider doing so, and when I do it’s to share a novel viewpoint that I think might spur thought in a new direction. I try hard not to make “me too” comments, and I have on several occasions spent 45 minutes or more crafting a reply, only to delete it without mercy for want of quality. So take heart, your appeals do fall on attentive ears.

  2. arnoldzwicky Says:

    To Rick S: I think this is the first time a reader has thanked me for asking for more thought and care in commenting. What I usually get is anger and complaint — in extreme cases, an accusation that I’m an authoritarian asshole who’s trying to censor people.

  3. More on experience and evidence « Arnold Zwicky's Blog Says:

    […] on experience and evidence By arnoldzwicky A little while back I posted on Michael Specter’s attack on Andrew Weil for what Specter sees as “one of the most […]

  4. Watch where you put that accent « Arnold Zwicky's Blog Says:

    […] been here before; see my postings “Experience and evidence” (of 12/17/09) and “More on experience and evidence” (of 1/19/10). The focus there was on […]

Leave a Reply

%d bloggers like this: