Inconsistent Responses in Three Preference-elicitation Methods for Health States

Values for health states obtained from the general population can be used in the development of cost-utility algorithms such as Quality Adjusted Life Years (QALYs) to aid in resource allocation decisions. These values are obtained using methods such as the visual analogue scale, the time trade-off or the standard gamble. However, all of these methods suffer, to some extent, from the problem of inconsistent responses. In this paper, we examine the degree to which three preference-elicitation methods-ranking, VAS and TTO--produce inconsistent responses, the effect of sociodemographic and health variables on the numbers of inconsistencies produced, and the effect of including inconsistent responses on the ordering of health states in EQ-5D VAS and TTO tariffs. Health states valued were a sub-set of 43 health states generated by the EuroQol-5D instrument, which were valued as part of a larger study to obtain a tariff of VAS and TTO values for use with the Spanish version of the EQ-5D. Two types of inconsistency--internal and criterion inconsistency were tested, and both the numbers of inconsistencies produced by each method, and the 'size' of those inconsistencies, i.e. the distance in terms of severity between health states rated inconsistently, were tested. The study showed that although the TTO produces higher numbers of inconsistent responses, the numbers of inconsistent responses in all methods were low, and did not affect rankings in the final tariff of values. Older respondents and those with lower levels of education produced significantly higher numbers of inconsistencies, though health characteristics were not significant. On the ranking and VAS methods, the number of inconsistencies doubled between the two interviewers. Efforts should be made to reduce inconsistencies, particularly among older and lesser educated respondents. Future research should concentrate on determining how inconsistencies might affect the values assigned to health states in the final tariff, and not simply the order of those states.