Avoiding the audit

Why I don't use rubrics when assessing student writing

J. Michael Rifenburg

English Department

assessing individual writing development

assessing writing programs

3 developments in writing assessment:

1961 ETS research bulletin

1994 Assessing Writing article

2003 book What We Really Value: Beyond Rubrics in Teaching and Assessing Writing

Diederich, Paul B., John W. French, and Sydell T. Carlton. 1961. Factors in the Judgment of Writing Ability. (ETS Research Bulletin 61-15). Princeton, Educational Testing Service.

Birth of modern writing assessment.

 

"disturbing" that "94% of the 300 [papers] received either seven, eight, or nine of the nine possible grades; that no paper received less than five different grades" (Abstract)

Thus the five-point rubric was born

"Simplified, ordered, well-controlled representation"

v

"dissent, diversity, context-sensitivity, and ambiguity"

(Broad 6)

The authors "traded in the rhetorical truth confronting them (that readers value texts differently) in exchange for the grail of high inter-rater agreement" (Broad 8)

The authors gave writing assessment 

Wiggins, Grant. 1994. The Constant Danger of Sacrificing Validity to Reliability: Making Writing Assessment Serve Writers. Assessing Writing 1(1): 129-39.

First issue of Assessing Writing

"In performance tests of writing we are too often sacrificing validity for reliability; we sacrifice insight for efficiency; we sacrifice authenticity for ease of scoring. Assessments should improve performance . . . not just audit it" (129)

Broad, Bob. 2003. What We Really Value: Beyond Rubrics in Teaching and Assessing Writing. Logan: Utah State UP.

equates use of rubrics born in the 1960s as using a Viking map from 985 CE to navigate the oceans today .

introduces Dynamic Criteria Mapping

getting after what "we really value" in our writing programs and in student writing.

do our rubrics represent the work of our department?