Technical Report

M. Artaso and F. J. Díez. Empirical comparison of visual analog scales and three versions of the time trade-off. A study on cochlear implantation. Technical Report CISIAD-16-01. Dpto. Inteligencia Artificial, UNED, Madrid, 2016.

18 pages. PDF (487 KB), BibTeX entry.


Purposes To measure the increase in health-related quality of life (QoL) from unilateral to bilateral cochlear implantation (CI) in children. To analyze whether the framing of questions in the time trade-off (TTO) method affects the measurement. To measure the effect of incentives on the response rate.
Methods We invited 3,465 students to answer an online questionnaire. Each respondent was randomly assigned to one of four techniques for measuring QoL: visual analog scales (VAS), time trade-off giving up life time (TTOtime), time trade-off giving up quality of life (TTOqol), and time trade-off based on equivalence (TTOequiv). Each respondent was offered feedback, either from the VAS to the TTO or vice versa, and the possibility of revising their answer. Subjects who gave implausible answers were warned.
Results 583 respondents filled in the personal data screen. The incentive, which consisted of the possibility of entering a lottery, caused a small—but statistically signiffcant—increase in the response rate. Some of the subjects assigned to TTOtime and TTOqol declined to enter the trade-off game. Others abandoned later. For 455 of the respondents we obtained at least one QoL estimate. The average initial (i.e., before the feedback) estimate of the QoL with unilateral CI was 0.581 for the VAS, 0.787 for the TTOtime, 0.654 for the TTOequiv and 0.486 for the TTOqol. Differences between these TTO estimates are statistically signiffcant, with p < 0:0003 for each of the three pairwise comparisons. The average initial increase in QoL from unilateral to bilateral CI was 0.247, 0.159, 0.111, and 0.231 for the four techniques, respectively. The proportion of implausible answers was 15.9%, 82.4%, 41.0%, and 69.1%; after the feedback these percentages were 29.9%, 69.5%, 27.8%, and 63.0% respectively.
Conclusion The increase in QoL due to bilateral CI ranges from 0.106 to 0.293, depending on the elicitation technique. The status-quo trap, a well known psychologycal bias that makes people reluctant to give up what they alredy have, leads the the TTOtime to overstimate the QoL and the TTOqol to underestimate it. When applying the TTO, we recommend the TTOequiv version because it seems to be more neutral, as it involves no give-up; additionally, it yields fewer implausible responses and is deemed easier by respondents. The feedback reduces to some extent the number of implausible responses for all the versions of the TTO but it is not suffcient to eliminate them. Therefore, it would be advisable to train ("calibrate") the respondents before applying this method.