Negative multiple choice marking: Not safe for boys ...

[klr]

... or so the story goes.

What is negative marking? When examining students via standard multiple-choice questions (MCQs), it is possible to award a negative mark for a question attempted and wrongly answered. "Neutral" negative marking will use a negative mark such that where a student answers questions at random, they will on average get 0 marks for these questions. So if each question has 4 options, and 1 mark for each correct answer, then -1/3 for each wrong answer => 0 on average for randomly answered questions.

Sometimes examiners use "punitive" negative marking (where answering at random would on average get a cumulative negative mark), or "benign" negative marking with the reverse effect. Or none at all, which encourages students to answer all questions even where they are clueless.

As far as I'm concerned, there should always be negative marking, and it should always be "neutral".

Now here's the thing: Apparently, negative marking has fallen completely out of favour in some places because it "penalises" boys, who are less risk-averse than girls. I say "apparently" because all I've come across is some off-hand references in a discussion thread on an Irish forum, but no concrete leads - yet.

Yes, males are on average greater risk-takers than females, but would that justify warping the examination system to such a degree? And where on earth would it end?

[klr]

Bingo! Found a "source" article:

http://www.collegetribune.ie/index.php/ ... mcq-exams/

October 26, 2010

The School of Economics has decided to discontinue the use of negative marking in multiple choice question exams (MCQs) as of this semester. The decision was made last April, after a student brought the apparent injustice of the negative marking procedure to the attention of the School.

The School of Economics’ decision comes after the valiant efforts of Brian Woods, a third year mature student of Economics and Politics. Mr Woods first brought the issue to the attention of the University two years ago.
According to Mr Woods, negative marking in MCQs is profoundly unfair. Some people will guess and do better than expected or deserved, while others may have worked hard, only to achieve a mark that do not reflect the effort they put in. “Also, how is the examiner to know who is guessing, and who knows what they’re talking about? There is no way.”

Negative marking in MCQs is used in universities worldwide. This method of examination awards points for the questions answered correctly, no points for those unanswered and, most importantly, points deducted for all wrong answers. Guessing can obviously be penalized or awarded, depending on each person’s decision making. On top of this, qualities such as risk taking vary from person to person. A report on negative marking by Imperial College London has shown that female students are “less prone to taking assessment risks than male.” Either way, the fear of losing marks by ticking the wrong box will put students off answering some of the questions at all.

Professor David Madden, Head of the School of Economics told The College Tribune that negative marking had had a negative effect within the school saying that “it made students very risk averse, so that in a situation where someone was, say 80% certain of the right answer, they might still be afraid to put down that answer for fear of being wrong and losing a portion of a mark.”

Prof Madden “graciously accepted [Woods’] submission on negative marking and decided to abandon negative marking in [economics] exams,” following a staff-student meeting where the issue was raised.

Mr Woods, who spent a lot of time researching the issue of negative marking, found that there was very little literature on the controversial procedure. The scarcity of information means that there is no practical evidence on trends in results. Prof Madden says “it will almost certainly lead to an improvement in results” in the School of Economics. “Hopefully we will also see improvements for those students who previously were nervous about putting down an answer even though they were fairly confident they were correct. And if marks for these students improve, this is to be welcomed.”

Woods is now calling on other schools in UCD to abandon the negative marking process – a system which he describes as an “outright theft of marks.”

[Schneibster]

This is why exams that require discussion are fairer. If a mistake is made, the exact mistake can be graded; if it's a minor mistake, the examiner can judge that. MCQs are, IMO, inferior to interviews for determining a student's level of achievement. Unfortunately, interviews are more likely to be biased (see, for example, Heisenberg's examiner who reprimanded him for not knowing the physics of a microscope, which he ambushed him with because he didn't like him personally). Overall the fairest way is to use a combination of both; and in addition, it appears that charging a negative for wrong answers has some problems.

However, I note that the article does not discuss the option where -1/3 credit is taken for a wrong answer, rather than -1. Or did I miss something?

[Geoff]

... or so the story goes.

What is negative marking? When examining students via standard multiple-choice questions (MCQs), it is possible to award a negative mark for a question attempted and wrongly answered. "Neutral" negative marking will use a negative mark such that where a student answers questions at random, they will on average get 0 marks for these questions. So if each question has 4 options, and 1 mark for each correct answer, then -1/3 for each wrong answer => 0 on average for randomly answered questions.

Sometimes examiners use "punitive" negative marking (where answering at random would on average get a cumulative negative mark), or "benign" negative marking with the reverse effect. Or none at all, which encourages students to answer all questions even where they are clueless.

As far as I'm concerned, there should always be negative marking, and it should always be "neutral".

Now here's the thing: Apparently, negative marking has fallen completely out of favour in some places because it "penalises" boys, who are less risk-averse than girls. I say "apparently" because all I've come across is some off-hand references in a discussion thread on an Irish forum, but no concrete leads - yet.

Yes, males are on average greater risk-takers than females, but would that justify warping the examination system to such a degree? And where on earth would it end?

I see the point of negative marking, but I'd prefer it if answers that were "more wrong" were weighted more negatively.

[Jason]

"Do not answer the question if you are unsure." I can't recall how many times I've heard that or somethings similar. Switching it around would be a bastardly thing to do.

[klr]

This is why exams that require discussion are fairer. If a mistake is made, the exact mistake can be graded; if it's a minor mistake, the examiner can judge that. MCQs are, IMO, inferior to interviews for determining a student's level of achievement. Unfortunately, interviews are more likely to be biased (see, for example, Heisenberg's examiner who reprimanded him for not knowing the physics of a microscope, which he ambushed him with because he didn't like him personally). Overall the fairest way is to use a combination of both; and in addition, it appears that charging a negative for wrong answers has some problems.

However, I note that the article does not discuss the option where -1/3 credit is taken for a wrong answer, rather than -1. Or did I miss something?

The article does not seem to mention the "severity" of the negative marking used, never mind there being a correlation between this and how students might behave.

I have my doubts that there was any serious thought put into this decision. It smell of a publicity stunt/pandering to students. All it will ultimately do is exacerbate "grade inflation".

There's a much larger issue here (IMHO) as to whether MCQs are the right means of examination at all in many cases. It's not as if people go through their working lives making the sorts of decisions that one is faced with in a typical MCQ examination. As you say, traditional examinations are often better. But guess what? MCQs mean less work for examiners ...

"Do not answer the question if you are unsure." I can't recall how many times I've heard that or somethings similar. Switching it around would be a bastardly thing to do.

If "neutral" negative marking were used, then on average it wouldn't matter if you took a complete pot-shot at a question or just left it alone. That's exactly what it is designed to do. But where you could eliminate (say) one or two choices for a question, then on average you are better off attempting an answer.

...

I see the point of negative marking, but I'd prefer it if answers that were "more wrong" were weighted more negatively.

Now there's an interesting concept. Maybe you could have more right answers as well. Or more than one answer with the same positive/negative value. I doubt it would ever take off though, because of the extra work involved in devising questions. A pity ...

[JimC]

When I test in high school, either physics or maths, typically about one third of the test or less will be alternative answers. I don't negatively mark, and neither do the examiners in the externally marked exams in the final year.

What's more important to me is getting the balance right between easy, medium and hard questions, so that a student who has only a moderate grasp of the subject can achieve a pass, but to get an A or better, you have to really know your stuff, and be able to apply it in tricky questions.

[Xamonas Chegwé]

When I test in high school, either physics or maths, typically about one third of the test or less will be alternative answers. I don't negatively mark, and neither do the examiners in the externally marked exams in the final year.

What's more important to me is getting the balance right between easy, medium and hard questions, so that a student who has only a moderate grasp of the subject can achieve a pass, but to get an A or better, you have to really know your stuff, and be able to apply it in tricky questions.

The (exam) rule of thirds (which, coincidentally, also applies to setting pub quizzes!)

Category A: 1/3 of the questions should be so obvious that anyone with a measurable IQ that attempts them, awake and sober, should be able to answer.
Category B: 1/3 of the questions should be slightly trickier but the average student, that has done an average amount of cramming, should have no trouble answering them.
Category C: 1/3 of the questions should actively select for students that have paid attention in every class and revised until their brains bleed!

Currently, answering only category A questions correctly results in a fail, answering only category A and B questions correctly results in a bare pass and answering all questions correctly results in a distinction. Obviously, the world would quickly become a far better place if the consequences were changed to: -

As only correct - compulsory sterilisation + no voting rights.
As and Bs correct - no voting rights.
As, Bs and Cs correct - voting rights plus admission to the breeding program.

...not that I am proposing eugenics or anything...

[Pappa]

So, with punitive negative marking, it's possible to have an overall score of -33% if there were 4 options on each question if they were all answered incorrectly.

[Tyrannical]

A lot harder to guess fill in the black questions.
Or should everything be short answer and essay? That introduces a whole different level of subjective grading.

[klr]

So, with punitive negative marking, it's possible to have an overall score of -33% if there were 4 options on each question if they were all answered incorrectly.

No, that would be with just "neutral" negative marking. I've heard of cases where it's -1 for each incorrect answer, meaning that you could get -100%. With such a punitive system, there would be no point in answering a question unless you were very sure you were picking the right option.

[Jason]

So, with punitive negative marking, it's possible to have an overall score of -33% if there were 4 options on each question if they were all answered incorrectly.

No, that would be with just "neutral" negative marking. I've heard of cases where it's -1 for each incorrect answer, meaning that you could get -100%. With such a punitive system, there would be no point in answering a question unless you were very sure you were picking the right option.

That's the system I know. You're not penalized for not answering, but wrong answers are.

I hate multiple choice exams anyway.

[klr]

So, with punitive negative marking, it's possible to have an overall score of -33% if there were 4 options on each question if they were all answered incorrectly.

No, that would be with just "neutral" negative marking. I've heard of cases where it's -1 for each incorrect answer, meaning that you could get -100%. With such a punitive system, there would be no point in answering a question unless you were very sure you were picking the right option.

That's the system I know. You're not penalized for not answering, but wrong answers are.

I hate multiple choice exams anyway.

I don't like them either. The only examinations I've taken since I finished my degree were for a project management course last year. The questions were almost all multiple choice, although many of them were in a range of very complex formats - not the plain "choose one answer" format. Anyway, I felt like a battery hen, or someone assembling consumer electronics in a Chinese sweatshop.

[Horwood Beer-Master]

...I see the point of negative marking, but I'd prefer it if answers that were "more wrong" were weighted more negatively.

And answers that are "not even wrong" are weighted the most negatively.[/Pauli]

[hadespussercats]

Negative marking doesn't really make sense to me. But then, sounds like I'm a boy when it comes to multiple choice tests-- or, well, any tests, when I was in school. I always guess if I don't know. It's fun to try to make a smart guess. And on essay tests, if I didn't know, I'd guess, then attempt skillful bullshitting. Which often worked.

But that's why I wouldn't be a good Jeopardy contestant, even though I get a lot of right answers, playing at home. But if I were losing money for every wrong guess I shout out, well...

Why is this a gender issue? Figure out what's fair, and apply it equally.