Web surveys are an important method of data collection in social science research. Such an approach, in addition to capturing substantive data, also enables the collection of digital traces, called paradata, which record the respondent's activity while filling out the questionnaire. Paradata are especially valuable in web surveys because their completion takes place without the presence of interviewers. In this context, response times – the time required to complete a question, page, or entire survey – have become the most commonly used type of paradata to address the flow and quality of the response process.
Although the technical recording of timestamps at the page level in web survey software is well established, the basic question remains whether the default calculated response times can be used without hesitation in further research. Namely, the dilemma arises whether the measured response times or the corresponding completion speed only reflect the cognitive processes of the respondent and the characteristics of the survey questions. Completion speed estimation can be biased by confounding factors: pauses, multitasking behavior, and returning to previously visited pages mean that response time is fundamentally misestimated; answering open-ended questions and the item »Other« artificially creates the appearance of slower completion speed; not answering the questions and not being exposed to the questions due to branching give the appearance of faster completion speed.
The main goal of the master's thesis is the conceptualization of the confounding factors in web surveys, the proposal of their treatment (i.e. neutralization of their effect), and the technical implementation of a standardized procedure for handling response times in the R programming language.
In the empirical part, predictions of the expected response time for different types of survey questions were first made. The practical treatment of confounding factors was tackled by adjusting the response times of each questionnaire’s page in the event of their presence using appropriate procedures. This makes the response times of every page comparable to each other. Only now can we calculate the respondent's completion speed of the online questionnaire and use response times to evaluate the quality of the response process.
|
