Tuesday, November 5, 2013

Tips for Writing Multiple Choice Questions

--> Hosting a game show in a large event is a great way to keep everyone engaged, review information, and add emotional impact with a bit of competition. When playing to a large audience most game shows are going to feature multiple choice questions. This way, everyone can play along using keypads, the game play moves fairly quickly, and participants get an extra shot of information from your presentations and sessions. 

Here are some tips to keep in mind as you create questions for your multiple choice game in your large event:

Make sure the content is neither too difficult, nor too easy.
Neither type of question—too difficult or too easy—leads to a competitive game. A too-difficult game has very low scores and quickly becomes discouraging.

A too-easy game has very high scores across the board, which is not very competitive.

If a question is going to be one or the other, it’s best to err on the side of easy, which will still engage the participants and can be useful for review.

There are multiple tricks you can employ to make a question more difficult without making it convoluted. These include:
  • Asking for specific or precise recall (i.e. specific budget numbers) with similar distractors
  • Going deeper into the content by asking follow-up questions (start out with an easier question and increase the level of difficulty)
  • Having very good distractors (more on this later, but essentially: having good alternate answer options aside from the correct answer can provide more challenge)
Not every answer has to be the same level of difficulty. It’s okay—and, in fact, sometimes preferable—to have your questions start out at a more basic or novice level and progress in difficulty as the game goes on and the points increase.


Make questions clear.
Don’t seek to increase difficulty by making your question a tongue-twister or a mind-bender. This is particularly important with an international audience but, also, you don’t want your contestants to have to think too hard about parsing the question in relation to the actual answer.

Avoid using negative questions unless they’re clearly and simply stated without additional negatives. For instance, “Which of the following is NOT a benefit of our new HR program?” can be a good question if the answer options are also clear and not-confusing. This, for instance, would be a nightmare question to process as a contestant:

Which of the following is NOT a benefit of our new HR program?
a)    Not getting under 3 weeks off
b)    Having to apply for benefits annually
c)    Not getting to apply for benefits on a monthly basis
d)    Neither having to complete the diversity course nor the culture compliance course if you’ve not done them in the last 6 months
e)    Huh?

Make your questions specific
For instance, asking “Who is the current president of the United States?” is a better question than “Who is the president?” This is not usually as much of an issue in multiple choice questions, where the answer options will often guide a contestant to a correct context—but if answer options are also somewhat ambiguous it can cause confusion.

Make the question short/concise.
Billy may have taken a train to a city and you want to identify that city, but your question should never look like this:

Billy gets on a train going from Orlando to Minneapolis; the train now contains 42 people. On the way, two passengers get off in Little Rock and 4 more get on. If, in St. Louis, thirteen get off and half the sum of the original passengers plus the number of passengers that got on in Little Rock get on, what is the name of the city Billy is going to?

Questions should be as clear and concise as possible—containing only the information necessary to ask the question and find an appropriate answer.

Instead of asking something like:

You’ve run into a person who you think is having a heart attack, what are the three most common signs to determine if this is actually the case?

Ask:

What are the three most common symptoms of a heart attack?


But keep in mind when paring down your questions: don’t use your answers to convey information that the question needs to ask. While you should strive to be direct in your questions, you shouldn’t leave out so much information that it has to be revealed in the answers. For example, instead of asking a question like this:

John Smith is:
a)    The Vice President of Operations for the Eastern Branch
b)    The Vice President of Operations for the Western Branch
c)    The Vice President of Operations for the Southern Branch

Ask:

John Smith is the Vice President of Operations for which branch?
a)    Eastern
b)    Western
c)    Southern


In a scenario where you want to use a longer question—i.e. you have a more complex case study to analyze—you can:
  • Include information before the question is asked, either through a presentation, information screen, or verbal description
  • Break the question into smaller pieces and have multiple questions around the same topic/area of interest
Have only one correct answer.
You may know exactly what answer you’re looking for, but if the question is unclear, or could be thought of in a different way--could someone come up with another answer? Could someone simply thinking outside the box come up with a different, but valid and correct response?

If there are two technically valid answers, either try grouping answer options together, or go with the A) answer a, B) answer b, C) answer a & b, D) none of the above route. HOWEVER…

Avoid all-of-the-above answer options to increase difficulty.
Let’s be honest, when there’s an all-of-the-above option the answer is always all-of-the-above… unless the question is deliberately designed to trick someone. Either way, you end up with a too-easy or too-difficult (by means of trickery and not content--which is frustrating) question.

A few all-of-the-above answer questions can be fine within a game—either to warm the audience up as an easier question, or as a question that leads to a more difficult, specific set of questions. However, they should be avoided in a game designed to be challenging throughout.

Use plausible distractors.
The distractors are the answer options you give that aren’t the real, correct answer. This can be one of the easiest ways to modify the difficulty level within your game show and they make a huge impact on the flow of the question.

In a four-option multiple choice question, at least 3 of the 4 answer options should be plausible, one of THOSE should be correct, and you can either have an additional plausible distractor, or incorporate something off-base or humorous.

To get good plausible distractors:

  • All answers should have roughly the same length and level of detail. i.e. When most people design answer options, the longest is usually the correct one because it’s stuffed with information that needed to be in the explanation before the question. Don’t do that.
  • All numerical answer options should be relatively close.
  • Answer options should skew (plausibly) toward your content. I.e. if you want to make a price seem like a good deal, the distractor options should be greater than the answer.
This:

How much is the NEW price for our X345 Widget?
a)    $145
b)    $165
c)    $195
d)    $250


…sounds much cheaper than this:

How much is the NEW price for our X345 Widget?
a)    $65
b)    $99
c)    $115
d)    $145

Be mindful of your question order
Don’t give anything away in one question that is going to appear in another question.

Proof your questions and answers
This may seem obvious, but when you’re the only one looking at content over and over again, things may slip by you that other people will catch (and could potentially make a difference in the correctness of one answer over another). Have someone proof-read your answers not just for correctness and simple mistakes, but also for difficulty and content.

Watch out for brand standards to avoid having to make last-minute changes. If your company colloquially refers to something by a shortened name, but the Brand Standard doesn’t reflect that, you may end up having to make revisions down the line. For instance, we worked for a credit card company that always referred to their clients as Card Members (capitalization, always) in branded pieces, so questions had to be revised to reflect that standard.

No comments: