Resource allocation decisions require information about individuals' preferences for goods and services. Survey based stated preference methods, such as discrete choice experiments (DCEs), are used to elicit preferences for non-market goods. A critique of stated preference research is that respondents to hypothetical surveys may not provide careful and thoughtful responses that reveal rational preferences. Choice certainty has been used to measure survey respondents' task engagement. Researchers assume that respondents who are certain about their choices provide deliberative responses. In the case of DCE, we argue that the variability of choice certainty is also important. We present a novel framework to identify thoughtful / deliberative respondents. The framework combines respondents’ certainty with their variability in certainty across a set of choice tasks. We test our framework empirically using data from two case studies. We find respondents with higher mean certainty and variability (i) seldom use decision heuristics, (ii) are more likely to have monotonic preferences, (iii) have longer response times, (iv) make choices that have higher interval validity, and (v) have higher choice consistency. We discuss the relevance of alternative ex-post calibration strategies with a view to improve the precision and accuracy of DCE-based welfare estimates.
