Asking Difficult Survey Questions (04-09-10)

Designers often want to ask people questions on surveys that may be difficult to answer honestly – not because survey-takers don’t know the answer to the questions, but because they feel there will be negative repercussions to answering them honestly.  For example, workers in an open plan office spaces may feel that if they state that they have trouble concentrating in their workstation they will be judged as somehow deficient.  Bokenholt and van der Heijden have developed the mixture-item randomized response method for use in this situation.  In the test of their methodology reported, people on disability assistance were asked questions such as “In the last 12 months, have you taken a small job alone or together with your friends that you got paid for without informing the social welfare agency.”  Participants were told to answer this question “yes” when an electronic simulated dice roll incorporated into the survey software totaled 2, 3, or 4; to answer “no” when the dice totaled 11 or 12, and truthfully in all cases.”  The number of yes and no responses that could be expected based on the “forced” yes or no answers can be compared to the total number received, to understand the truthful responses to the question raised.  This method has been shown in practice to “motivate most of the respondents to answer truthfully.”

Denise Gellene.  2010.  “Surveying Sensitive Topics:  New Tools Help Correct for Survey Bias.”  Kellogg Insights, http:///insight.kellogg.northwestern.edu.