OBJECTIVES: There is a growing interest in eliciting stated preferences from patients or other harder to reach groups. Consequently, surveys are increasingly administered online to maximise respondent access and allow for randomisation, skips and forced choices to prevent illogical answers. This study aimed to investigate if, and how, the access device affected survey-related behaviour, choices and/or preference estimates.
METHODS: A discrete-choice experiment (DCE) was administered online to members of the public. The survey comprised 10 choice-sets generated using a Bayesian D-efficient experimental design incorporating priors from a pilot study. After the choice-sets, respondents were asked sociodemographic questions as well as self-reported ease of choice making and attribute attendance. The DCE was administered online using Sawtooth software capturing the respondents’ device type, browser, operating system, the time spent on each webpage, and the point of drop-out. After entropy balancing to reweight the sample of personal computer (PC) responders and respondents using handheld devices, differences in key outcomes (dropouts, response time, attention, understanding, failure of an internal validity test, self-reported attribute non-attendance, and willingness-to-pay) were analysed. Interaction terms for device type were included in conditional, heteroskedastic conditional and random parameter logit models for the analysis of choice data.
RESULTS: The most popular device was a PC (n=1,270) with a Microsoft (n=1,148) or Apple (n=102) operating system. In total, 322 respondents accessed the survey on a handheld device. Analyses were conducted on a final sample of 1,000 completed using either a PC (n=785) or handheld device (n=215). Significant differences (p=0.017) were found in response time; those accessing the survey on a handheld device were almost 10% (9.6%) less likely to ‘speed’ through questions. Most other survey-related behaviours, choices, and preference estimates were robust to device type.
CONCLUSIONS: This study indicates that allowing respondents to use their preferred device did not influence preference measures.
