False dilemma: The either-or fallacy and the standardized testing debate

"They call me Mr. Knowitall
I will not compromise.
I will not be told what to do.
I shall not step aside.

They call me Mr. Knowitall
I have no time to waste.
My mouth it spews pure intellect.
And I've such elegant taste! [...]

They call me Mr. Knowitall
I am so eloquent!
Perfection is my middle name,
And whatever rhymes with... eloquent."

                                         ("Mr. Knowitall," Primus)

A few days ago, I came across a blog post by a fellow who chose to write about standardized testing, something that has been written about eighteen or so trillion times.

But this article was different.  It didn’t recycle old rhetoric and clichés, nor was it unduly burdened with melodrama and gross hyperbole, two sins that many, including perhaps yours truly, indulge in from time to time – some deliberately (cheesy and manipulative writers), some accidentally (bad writers).

Instead, I found its tone revelatory in its simplicity.  It says only what it needs to.  It is clear – both precise and concise.   It is so short that my attempts to excerpt it would lead me to duplicate almost the entire article, which only clocks in at just over 800 words.  It is eminently readable, and it accomplishes this without dumbing down the jargon, technical language or rhetoric.  

A few choice snippets:

“As a teacher, if my livelihood is based on test results, then I will do everything possible to ensure high marks, including narrowing the curriculum and prepping fiercely for the test. The choice between an interesting project and a paycheck is no choice at all. These are amazing disincentives to student learning. Tying teachers' careers to standardized tests does not foster creative, passionate, skillful young adults. It does exactly the opposite.”

“As a large body of research suggests, standardized test data are imprecise for two main reasons. First, they do not account for individual and environmental factors affecting student performance, factors over which teachers have no control. (Think: commitment, social class, family.) Second, high-stakes, one-time tests increase the likelihood of random variation so that scores fluctuate in arbitrary ways not linked to teacher efficacy. (Think: sleep, allergies, the heartache of a recent breakup.)”

“Because standardized tests are an inexact estimate of a teacher's ability, they are also unfair. By focusing on a sliver of the curriculum -- often rote facts --standardized tests do not measure meaningful understanding. (Think: ‘Who was the last French monarch?’ versus ‘How much violence is justified in revolution?’) And unless you believe bubbling the letter of the best answer is crucial in the 21st century, standardized tests exclude evidence of important skill development.”

And I love the way the author, Jack McKay, describes testing as America’s “new national pastime.” Hell, baseball is all about the stats now, why shouldn’t teaching be so as well?  Then they could print test scores and passing rates on the back of teachers' I.D. badges -- like baseball cards!  *sigh*

What most caught my attention, however, was this phrase:  “[E]ducation reform has been cast as a false dilemma between students and teachers…”

I wrote my Masters thesis on the either-or fallacy in public policy (specifically, as pertains to education and pedagogy) decision-making.  The either –or fallacy (also called false dilemma or the black-and-white fallacy) is, simply put, the assertion that any set of options is necessarily reduced to either x or y.  I am a passionate opponent of this manner of oversimplified, prone-to-extremes type of issue analysis.  (Think: America’s 2-party system.)

People seem to naturally gravitate towards this simplified manner  of thinking, unfortunately. Have you ever noticed that when you disagree passionately with a person, you often abandon a moderate stance in order to more vehemently oppose that person?  It’s easier to defend an extreme position, because it eliminates the complexities and nuance of any argument, and liberates you of the need to make concessions.

Think about it in debate terms.  There are two ways to approach debating:  One is to use the friction of debate to unlock new lines of inquiry, probe diverse viewpoints, better understand the gestalt of an issue, and reach a better (or at least more acceptable) solution to a problem.  I’ll call this constructive debate.  The second way is to “win,” whatever the cost, in other words, to take down your opponent.  I’ll call this destructive debate. 

There’s nothing harder than engaging in constructive debate with an adversary who is hell-bent on destructive debate.  You want to look at multiple sides, make the necessary concessions to show that you’re not a one-note ideologue; you want your discussion to be rich, nuanced, erudite, but your efforts to recognize the aspects of your adversary’s arguments with which you might agree only fuel his rhetorical zeal, and he in return gives you no such consideration. “Score-wise,” you have capitulated, whereas he has not.  He sees your acquiescence as a sign of weakness, your lack of sureness as to your own core beliefs. Clearly, he tells you, you have no consistent principles.  You feel the center of the debate shifting unacceptably towards your adversary; your confidence slides as he is further emboldened.  This creates a new problem.  You certainly don’t want your supporters to see you agreeing with aspects of your opponent’s position, while not winning any points of your own.  They might think your opponent’s position has merit!  So what do you do? You ratchet up your end of the dialogue, take a slightly more extreme position to “titrate” the discussion more into balance.

Oh, but this has the opposite effect, rather like trying to steer out of a skid.  Ere long, you and he are at radical opposite ends of the spectrum, two polarized extremes, an either-or.  Another debate that could have been a constructive discussion, turned into a destructive battle.

Even the best of us falls into this trap occasionally.  Even, it seems, the esteemed author of the blog article I’m currently lauding.

He writes:

“I believe student results from standardized tests should not be used to evaluate teachers because the data are imprecise and the effects are pernicious.”

“Let's decouple high-stakes testing from teacher evaluations for the sake of students and teachers alike.”

Hey, I agree with him! Well, mostly, that is…

And isn’t that the lure of the either-or trap?  It satisfies our driving human need to emote.  To want to say “Hell yeah!” or “Hell no!”  It’s much less cathartic to scream out, “That’s pretty much it, but there are a few details I’d like to take issue with!”  The either-or fallacy satisfies our need for easy categorization (are you a pro or a con?) and our need for fraternization (it’s easier to match black with black and white with white than it is to color-match an infinite number of shades of grey).  No public speaker is going to rouse a teeming crowd of would-be hangers-on to orgasmic outbursts of applause with “Pretty much!  That’s basically it!”

Look at these modified versions of his sentences:

“I believe student results from standardized tests should not be used as the main metric to evaluate teachers because the data tend to be imprecise and the effects can be pernicious.”

“Let's drastically reduce the impact of high-stakes testing results on teacher evaluations for the sake of students and teachers alike.”

I don’t think the utter elimination of high-stakes testing is the answer, necessarily.  That's the "or" to the "either" of testing. Any assessment can show us something useful.  If four physics teachers at a high school all give the same assessment to their three classes of 75 kids, and the results are radically different, there is a good chance that there’s some reason why.  The score disparity might just alert the teachers, or the department, or the administration to a conversation that needs to be had, a need that is not being met, or simply an artifact of the testing process itself that should be discussed, if for no other reason than to eliminate it from consideration as a failure of one aspect or other of the system.   Those conversations cannot happen if there are no diagnostics of any kind, and standardization (or "norming") is the best way I can think of to achieve, or at least approach, a state of ceteris paribus.

Now, and this is important, the reason is NOT AUTOMATICALLY teacher inefficacy, which would automatically be the assumption under the very real current educational regime that McKay describes.  Therein lies the problem – not so much the assessment itself (although we do way the hell too much of it, and at the expense of instructional time, no less), but the high-stakes, make-or-break impact of what we do with the result.

Simply put:  We trust the numbers more than we trust the people.  In fact, one might to moved to say (if one were wont to hyperbolize) that we trust the numbers to the exclusion of trusting people.   And as McKay correctly points out, the results are catastrophic.  I mean, “can be” catastrophic.  (Modal verbs make great qualifiers.  Consider the subtle, almost subliminal, impact of qualifiers on a debater's presentation by checking this out .)

Anyhow, I enjoyed the McKay piece, and I suggest you read it.  It’s short, much shorter than this (1,300+ words at this point already). And I think he is possibly more eloquent than I.  He doesn’t claim to be a Mr. Know-it-all; he pleads with us for debate (constructive, not destructive), dialogue, a discussion, a conversation.

Works for me.  I may disagree with you, but I’ll always try to talk to you, not at you.  Unless you piss me off, then I’ma have to take you down…

Can blood-feud policy adversaries ever sit at a table and have this kind of constructive discussion without resorting to the cheap and ultimately destructive rhetorical shortcuts of partisan false dilemmas?  Oh, snap, by resorting to the binary of "constructive" and "destructive," did I just create my own either-or?

I suspect I know the sad answers, but still, I’m A.S.K.ing…