Archive for the ‘Testing’ Category

Dubious diagnostics

February 27, 2011

Letter from Lucille Lang Day of Oakland in the February 2011 Harper’s:

The Method in It

In her article on prodromal psychosis [“Which Way Madness Lies,” Report, December 2010], Rachel Aviv discusses an exam said to evaluate the risk of psychosis. The exam asks patients to describe “the similarities between an apple and a banana.” This is a very poor way of diagnosing the likelihood of mental illness.

Before I quote the rest of Day’s letter, think for a moment how you would answer this question.


Data points: N ellipsis, ambiguity 7/31/10

July 31, 2010

Just in, from my daughter, about her daughter:

On Friday, Opal does work that she brings home. Yesterday’s included the word problem: Mother cat had 6 kittens. 5 kittens went to new owners. How many were left?

Opal’s answer, duly checked off by the teacher, was “2 cats were left.”

Thank goodness she gave a complete non-elliptical sentence as an answer. That’s what you’re expected to do in school talk, even though it totally goes against ordinary language use, which sensibly enough goes for brevity.

Opal’s answer, which chooses one of the two ways of filling in the N ellipsis in the question’s how many? — ‘how many cats?’ vs. ‘how many kittens?’ — does, however, opt for the contextually less likely reading of the question: the set-up for the question is about kittens — the discourse topic is kittens — so the expected fill-in for the ellipsis would be kittens, not cats, even though the N cat is out there in the context. That is, the expected elliptical answer would be “1”, or somewhat less elliptically, “1 kitten”, or the complete sentence “1 kitten was left”. (Ok, a kid who responded with any of these might still have the answer marked WRONG by a stickler teacher, since they all have “1”, the numeral, instead of “one”, the number word. My god, school is a minefield.)

I suspect that Opal might have learned, by experience rather than explicit teaching, to anticipate pitfalls and tricks in test questions, which you can avoid only by talking with utter explicitness. It’s still an open question whether she honestly (and unconsciously) interpreted the question in the contextually less appropriate way, or whether she was (perhaps without thinking it through) being clever and tweaking the person who wrote the question, or whether she meta-reasoned (unconsciously) that 6-minus-5 was just too stupidly easy a question, so that some answer less obvious than “1” was called for. (Not that we could find out by asking her: if she could supply an answer to the question “Why did you say ‘2 cats’?” at all, the answer she gave would be likely to be a construction based on her interpretation of the reasons behind our question, since she would have been extremely unlikely to have had insight into the springs of her behavior at the time she wrote her answer — whatever they might believe, people aren’t at all good at getting access to their unconscious thought processes — and she’s even less likely to remember these details now. So I’m having breakfast with Opal and her mother in a few minutes but won’t ask her why she gave that answer.)

Oh dear, this has gone way past just a brief reporting of a data point. So goes the academic life!