Announcing
PhD in Second Language Studies Dissertation Defense
Gerriet Janssen
English Proficiency and Entrance to Doctoral Studies at One Colombian University: A Standard Setting Study
Chair: James Dean Brown
Tuesday April 26, 2:00 p.m.
Moore Hall, Room 155A
Critically important to test and program development is the enterprise of standard setting. This activity establishes descriptions of different performance levels and then uses a principled methodology to construct a cut score on a test: a point on a score scale that represents the separation of these different performance levels. To ensure that the creation of this cut score has been done is a way that is 鈥渟ystematic, reproducible, objective, and defensible鈥� (Cizek & Bunch, 2007, p. 8), standard setting draws upon a great body of theoretical and empirical resources, protecting test users from the negative consequences of arbitrary decision making.
Recently, one Colombian university developed coursework and a placement exam for an EAP program supporting PhD student publication and presentation skills. During this program鈥檚 development, a classification exam was created to place the PhD students into different course levels. Though not part of this exam鈥檚 uses, university leaders soon established that certain course levels would now be an entrance requirement for different PhD programs. This decision was arbitrary in that it was made without understanding (a) what decision makers wanted their PhD students to be able to do in English or (b) how these skills intersected with the different course levels that were being offered. In short, this decision was made without taking advantage of the technology of standard setting.
Responding to this situation, this study鈥攗sing the Body of Work Method (for the performance writing and speaking sections of the exam) and the Bookmark Method (for the multiple-choice grammar-vocabulary-reading comprehension section)鈥攅licited from this university鈥檚 PhD community their conceptualization of the minimum English proficiencies for PhD study at this university. From these individual recommendations, one cut score was initially calculated; however, the different procedural, external, and internal validity evidence indicated that two cut scores鈥攐ne more lenient, one more severe鈥攚ould be a better solution for this context. However, in light of other evidence from the validity argument, this dissertation suggests as its final recommendation that the university prioritize which tools of learning it would like to develop, instead of worrying about which limitations it should impose.