O: The ultimate goal of testing is to measure what the students actually understand and according to the book Dr. G lent me, the most difficult part in creating these exams is that they are inherently nonobjective because question structure and wording are undeniably subjective and can greatly influence a student’s success on a question regardless of their understanding of the material (this is something Dr. G has addressed with the front page of our exams). The structure and wording of a question and its choices can make for a tough question even if you know the material forwards and backwards. So, the point I would like to focus on is the difference between good, challenging questions, and poor, tricky questions.
A couple of examples of what I consider hard and acceptable questions are question 28 from test 2 and question 5 from test 1. I believe it’s okay to have tricklike questions, which is what I see these two as. I would consider question 6 from test 2 as a tricky question as we discussed the ambiguity in the wording of choice C.
What separates these questions to me is that the tricky ones are difficult to understand due to structure and word choice, whereas the tricklike questions are testing your understanding of the material by having similar answer choices or asking the question in a different, possibly “tricky” way. However, the literature I read on this subject said that you should essentially leave the question in its barebones to really test students’ understanding of the material and to not decrease the reliability and validity of the exam. I imagine this would make it harder to write rich, application questions, which should demonstrate a higher level of understanding, so my question to you all is: where is the line between easy, cupcake questions that just involve regurgitating definitions and the challenging, possibly ambiguous, questions that really test your grasp of the material? If the professor really wants to see how well they are doing in instructing you and how well you understand the material, should they play it safe and see if you can memorize the textbook, or should they risk it and ask the tougher questions that may leave students scratching their heads?
J: Adding onto what Oscar said, given the difference between questions testing rote memorization and tougher, conceptual questions, what are your opinions on closed-book tests versus open-book tests? I have found that some of the hardest tests I have taken in my academic career so far have been open-book tests, such as in Physics, because they tend to be much more difficult conceptually.
Also, continuing on the point of asking good questions on a test, the Vanderbilt Center for Teaching says that multiple choice questions are good for assessing students because they have a lot of versatility, reliability, and validity (for more specific definitions, check out he link!). What do you guys think makes a “good” test in terms of question types? Could a full multiple choice test be good, or a full short answer? More specifically on science tests, what mix do you think is most successful in assessing students?
Finally, the Center for Teaching says that an effective multiple choice question should have an effective stem, meaning that it should clearly present a question, should not contain irrelevant material (which I found interesting, since I have taken tests before where there was extra information throw in to “trick” students), and should only be negatively phrased when the item is important. As for the answers, they should all be plausible, stated clearly (so as to test learning rather than reading ability), mutually exclusive (I also found it interesting that the CFT advised against excessive us of “trick” items, since they can erode trust for the testing process), should not include answers such as “all of the above” or “none of the above,” and should be free from clues about which response is correct. In short, the answers should all be plausible and should not allow students to arrive at the correct answer with only partial knowledge.
Given the above guidelines, I invite you to try and craft a good multiple choice question of your own!
I hope this gives us something good to talk about.
