125 (Spring 2000): 54-55
|
|
|
|
This Is Not a Test! Assessing Students Assessing the University
TAMARA HARVEY
DURING spring 1998, I helped lead a women's studies outreach program in two eleventh-grade English classes at Santa Ana High School. In our module we asked students to think about the social construction of gender in literary works, artworks, advertisements, and videos and then write about personal photographs using the questions and approaches we had developed throughout the unit. We were also required to come up with a formal exercise for evaluating our program. Our assessment endeavors were aimed at three groups: at those in the institution who evaluate and fund the Humanities Out There program; at us teachers who sought to develop an effectively collaborative module; and at the students whose impressions of our program, women's studies, and the university would be shaped in part by this exercise. By far the most difficult task we faced in developing this assessment tool was to represent our program and the university positively to students without distorting the philosophy of our module. How could we combine our emphasis on questioning institutions with an apparatus that often seems clinical and alienating?
At the beginning and end of our module we had students respond to the same questionnaire. At the top of the page was the caveat, "This is not a test!," followed by a series of questions about definitions and examples of myth and about students' plans for college and the influences behind those plans, followed by an ad analysis exercise. All this looked very much like a test. Our rationale was that the first section on myths was most testlike and the second on college plans and influences most personal, whereas the ad analysis built on these personal and objective components to allow students to practice the formal skills of analysis.
Structurally, then, this looked like a test, though students were aware, especially at the end of the session, that their responses would not be graded. We felt that meeting students' expectations about university practices through a testlike set of questions would foreground the institutional apparatuses students negotiate and will continue to negotiate regardless of their educational plans. In a program that emphasizes respect for students' intelligence and analytical abilities, masking this aspect of the institution would be hypocritical and ineffective. Moreover, the familiar test structure gives students a chance to perform--their answers to the myth question in particular were filled with responses reflecting cultural awareness they felt, rightly, few of the tutors and instructors shared.
We were hopeful that the ad analysis section would most substantially transform this test into not-a-test. Ideally, this section would reveal the university and women's studies as places where society is analyzed in a manner that, while informed by the personal, is not just about personal empowerment (which would turn out to be an important part of the students' senses of the university in their answers to part 2). Active social analysis and critique are skills and attitudes toward knowledge and toward the university. We hoped our assessment questionnaire would not be a test if it conveyed this sense of social critique.
In responding to our questionnaires, students did practice institutional critique, but not in ways we anticipated. At the beginning of the module, responses to our request for examples of myths, our most testlike question, were varied, creative, and even rebellious (examples of characters from myths included Zeus, Athena, Narcissus, Lallorona, Chupachas, Freddy Kruger, and Jerry Springer), while the responses to plans about college were more uniform and the analyses of a cigarette ad we had chosen for its objectification of women led primarily to assertions that the students would never smoke. The right answer, students seemed to feel, was that college was good, smoking bad. When we gave students the same questions at the end of the semester, they did not take them seriously or put much thought into answering them. In short, their responses reflected institutional awareness primarily by giving us what they perceived to be the right answer the first time and by resisting a repetition the second time. Their responses to classroom assignments such as an exercise in which they wrote analytical responses to Madonna's Borderline video on newsprint posted around the room proved much more effective in eliciting thoughtful cultural analysis.
Since our ad hoc efforts, HOT has implemented assessment methods based on a program developed by Rigoberto Rodriguez, a graduate student in social ecology at UC Irvine. Students in HOT programs and control groups from nonparticipating classrooms are now evaluated for grade-level writing, grade-level reading, critical thinking skills, positive literacy attitudes, and college motivation, using a combination of standardized test data and results from an attitudinal survey administered by HOT. A new series of questionnaires filled out by tutors, teachers, and coordinators also helps assess program implementation and curriculum development. These data will be analyzed to assess HOT's efficacy in improving student literacy skills and increasing postsecondary enrollment. Finally, HOT has created regular guidelines for participation by the growing number of people with whom it is associated.
The quality and creativity of students' final projects remain far more powerful evidence of the efficacy of HOT than our questionnaires do, and the data obtained from Rodriguez's assessment model promise to add more objective support to this evidence while providing a diagnostic tool for classroom teaching and institutional decision making. Nonetheless, making the assessment tools we use into a site of analysis for students as well as teachers and administrators ought to be an important goal, since students often regard these tools as a largely alienating apparatus of the university. Our questionnaire was in many ways a test, despite our disclaimer, and, moreover, a test with no consequences. We wanted it to move toward cultural analysis that could be used to critique all institutions. Instead, the implicit critique was of our expectations to the degree that they were perceived as manipulative. Changing the questions and objects of analysis the second time around or at least explaining why it is interesting to examine the same questions through a different lens might make the exercise more stimulating. Couching the questions as drafting tools for college applications and job interviews might provide another way to engage students and give them concrete information about the rhetoric and logistics of college applications. Given our goals, bracketing each module with statistical and attitudinal assessment tools aimed primarily at administrators and teachers seems inadequate and potentially counterproductive. We should include with these surveys questions that allow students to engage critically with the institutions we represent. But we must make sure that they cannot be interpreted as a test.
The author is Assistant Professor of English at the University of Southern Mississippi. This paper was presented at the 1999 CCCC Annual Convention in Atlanta.
Rodriguez, Rigoberto. Humanities Out There Program Evaluation Plan. Irvine: School of Social Ecology, U of California, Irvine, 1998.
© 2000 by the Association of Departments of English. All Rights Reserved.
ADE Bulletin 125 (Spring 2000): 54-55 |
|
|---|
