This paper presents a discussion of precautions within the category rating procedure.
One problem first described in this article is grouping bias. Consider using a 5-point response scale to evaluate 20 stimuli: on average 4 stimuli must be allocated to each response point which means the respondent is prevented from discriminating between them. To counter this, the number of stimuli examined should not greatly exceed the number of response points in the rating scale.
This paper descries a pilot study comparing rating scales with different numbers of response points. Variation in these can influence how people respond yet it is rarely if ever questioned in studies, using, for example, the 9-point de Boer scale without question. Much further work is required to continue this investigation.
Note: please see Hyvarinen 2015 for a criticism of the statistical analysis in this paper.
This is a review of the study by Kakitsuba attempting to validate the Kruithof curve. Unfortunately, the choice of stimuli and centering tendency means this study was unable to do anything but ‘validate’ Kruithof.
This discussion of category rating focuses on the assumed truth of responses. This assumption is that respondents have valid experience on which to base their response, have a valid recollection of that experience, have properly read the question, understand the question in the same way as the experimenter intended, and are then able to accurately indicate their opinion. In many cases, if not all, these are assumptions too far.