Purpose
Existing research on the measurability of information quality (IQ) has delivered poor results and demonstrated low inter-rater agreement measured by Intra-Class Correlation (ICC) in evaluating IQ dimensions. Low ICC could result in a questionable interpretation of IQ. The purpose of this paper is to analyse whether assessors’ motivation can facilitate ICC.
Methodology
To acquire the participants’ views of IQ, we designed a survey as a gamified process. Additionally, we selected Web study to reach a broader audience. We increased the validity of the research by including a diverse set of participants (i.e. individuals with different education, demographic and social backgrounds).
Findings
The study results indicate that motivation improved the ICC of IQ on average by 0.27, demonstrating an increase in measurability from poor (0.29) to moderate (0.56). The results reveal a positive correlation between motivation level and ICC, with a significant overall increase in ICC relative to previous studies. The research also identified trends in ICC for different dimensions of IQ with the best results achieved for completeness and accuracy.
Practical implications
The work has important practical implications for future IQ research and suggests valuable guidelines. The results of this study imply that considering raters’ motivation improves the measurability of IQ substantially.
Originality
Previous studies addressed ICC in IQ dimension evaluation. However, assessors’ motivation has been neglected. This study investigates the impact of assessors’ motivation on the measurability of IQ. Compared to the results in related work, the level of agreement achieved with the most motivated group of participants was superior.
| Language: | English |
|---|
| Keywords: | information quality, motivation, information quality assessment, inter-rater agreement, task-at-hand, gameful design, sports, online encyclopedias, research assessment, games, video games, measurement, psychometrics |
|---|
| Work type: | Article |
|---|
| Typology: | 1.01 - Original Scientific Article |
|---|
| Organization: | FRI - Faculty of Computer and Information Science
|
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Year: | 2022 |
|---|
| Number of pages: | 22 str. |
|---|
| Numbering: | Vol. 17, no. 10, art. e0274811 |
|---|
| PID: | 20.500.12556/RUL-144622  |
|---|
| UDC: | 004 |
|---|
| ISSN on article: | 1932-6203 |
|---|
| DOI: | 10.1371/journal.pone.0274811 |
|---|
| COBISS.SI-ID: | 127711235 |
|---|
| Publication date in RUL: | 03.03.2023 |
|---|
| Views: | 593 |
|---|
| Downloads: | 83 |
|---|
| Metadata: |    |
|---|
|
:
|
Copy citation |
|---|
| Share: |  |
|---|
