Negative multiple choice marking: Not safe for boys ...

[MrJonno]

Multiple choice is poor quality low cost examinations anyway. It should be possible for someone to demonstrate at least partial knowledge on a question instead of getting 0 or 100%

[apophenia]

There are a couple subtle effects not being acknowledged here (I skimmed, forgive if I missed something). First, MCQ without negative grading unfairly advantages boys, who, being less risk averse, will take more guesses and by sheer chance get more correct than a more risk averse female. And I imagine that, if all tests were conformed, over the lifetime of a student, that would put the female at a disadvantage. Second, even with negative grading, there will be a zone where less risk averse students scores will improve, rather than worsen, where they are unsure of the answer, but their hunch is correct; female students won't benefit from this "confidence boost" and it would have to be studied under what conditions and which segment of the male student populace is affected. And we also have to keep in mind that these are statistical norms, not fixed traits; there will be male students who are more risk averse than a given female student and so on. Regardless of the overlap, and which gender is more or less risk averse, the goal should be to encourage those traits we wish to nourish in our students. I would fathom that where one sits on the scale of relative risk aversion, tests should concentrate on measuring mastery of the material, and not building in questionable bonus points for having the right kind of personality - whatever that might be. Ultimately, a lot will come down to money - non-MCQ testing is expensive to administer and grade, and costs a student in more study time for equivalent mastery - politics, and probably some poorly thought out science and pseudo-science as well.

[JimC]

Some interesting points, Apophenia. In many ways, it depends how the test is constructed, and how the potential answers are generated.

In maths exams, in Oz there is usually a multiple choice section plus a short answer section, where demonstrating the correct sequence of working out contributes to the mark.

As it should...

When generating potential answers, one can either make 4 random ones, and one correct (I always make it ABCDE), or you can be sneaky, and generate a couple of answers that would come from typical student errors...

However, in the end, I feel it is even more important to get the balance right between relatively easy, moderately hard, and fiendishly difficult questions...

I actually enjoy making assessment tasks. I am perhaps a little strange...

[Ronja]

When I was last involved in designing MCQ questions (in the late 1990's) my then-boss (a really nice prof) was adamant about two things:

1) neither he nor us teaching assistants (TA) should be tortured with grading answers to essay questions, if the student clearly had not studied for the exam. The reason: our team handled three big modules per academic year, with 350 / 250 / 100 students, i.e. minimum 1400 midterms to grade every year, plus hundreds of end-of-term exams for those who had failed the first midterm, and there was a non-trivial number of students who kept coming to exams again and again, seldom much better prepared.

Therefore: use the MCQ as a criterion for whether the student's essay answers would be read at all (50 % was required to pass the MCQ part and 50% to pass the essay part).

2) as the MCQ got this very important gatekeeper role, the questions and the answer alternatives had to be designed very carefully. Especially guessing the correct answer based on how the alternatives were worded was a big no-no, and making that hard was a LOT of work. We ended up with a system where for each question there were six answer alternatives: one worth +1, two +1/2s, one -1 and two -1/2s. Not answering was worth 0. There were never fewer than 10 MSQs and never more than 16 per exam.

Over the years we constructed a battery of over 70 MC questions, with very carefully worded answer alternatives, and that was a lot of work. But then the MCQ part of the exam was really quick to check, and interestingly a larger percentage of students seemed to prepare at least tolerably for exams after they realized that they had to pass the MCQs to pass the exam at all - that there were less chances of getting "lucky" by spouting creative horse-shit and hoping for an inexperienced, unconcentrated or charitable-feeling TA to be one's grader.

Most likely our system was not perfect, but it kept us TAs motivated, as we knew that we did not have to read the worst essay answers (we got enough of creative horse-shit to read with the first home assignments of the first bachelor course, before the students cottoned on to that one could not do that and hope to pass with us).

[Azathoth]

Multiple choice is poor quality low cost examinations anyway. It should be possible for someone to demonstrate at least partial knowledge on a question instead of getting 0 or 100%

Multiple choice is bullshit. It tests memorisation of facts not understanding

[Tero]

I get confused. You have to make questions so I can answer yes or no plainly. I don't seem to understand double negatives at least not in English.

But the multiple choice thing is clear, I could figure that one out. It should not be that hard.