Oblikovanje in vrednotenje sistemov ekspertnega ocenjevanja v znanosti : doktorska disertacija

Dekleva Smrekar, Doris (author), Južnič, Primož (mentor)

V doktorski disertaciji poteka proučevanje sistemov ekspertnega ocenjevanja z dveh vidikov: upravnega, katerega cilj je poenostavitev postopkov, in vsebinskega, katerega cilj je dosega maksimalne objektivnosti ocenjevanja. Pri tem so pomembna stališča treh glavnih vključenih subjektov: raziskovalcev (prijaviteljev), ekspertnih ocenjevalcev (recenzentov) in administratorjev (strokovnih sodelavcev), ki vodijo postopke ocenjevanja. Iz znanstvene literature so povzeta izhodišča, pomembna za razumevanje delovanja sistemov ekspertnega ocenjevanja v znanosti. Bistven pomen znanstvenega raziskovanja je ustvarjanje znanja. Nanj odločilno vpliva obstoj znanstvene skupnosti, ki jo povezuje informacijski proces znanstvenega informiranja in komuniciranja. Mehanizem ekspertnega (kolegialnega) ocenjevanja je zgodovinsko utemeljen način zagotavljanja kakovosti v znanosti, s katerim znanstvena skupnost zagotavlja avtonomnost in verodostojnost znanstvenih dognanj. Poleg ekspertnosti je odločilen dejavnik pri odločanju tudi človeški faktor, zato so poglavitnega pomena profesionalne in osebne etične norme. Izvedena empirična metoda temelji na izkušnjah, na podlagi katerih so se odkrivali in pojasnjevali določeni pojavi v delovanju sistemov ekspertnega ocenjevanja. Opravljena je bila raziskava v sklopu nacionalnega sistema dodeljevanja sredstev za raziskave, kot kombinacija več raziskovalnih metod. Narejene so bile analiza regulative, študija testne implementacije posebnega programskega orodja za iskanje ekspertnih ocenjevalcev pri razpisih za dodeljevanje sredstev za raziskave, analiza prijav in eksperiment uporabnosti tega orodja za tiste, ki niso strokovnjaki za posamezna znanstvena področja. S pomočjo anket in usmerjenih intervjujev so bila zbrana mnenja vseh treh glavnih akterjev v sistemih ekspertnega ocenjevanja. Rezultati raziskave podajajo usmeritve za oblikovanje in vrednotenje sistemov ekspertnega ocenjevanja v znanosti ter aplikativne možnosti za njihovo racionalizacijo in posodabljanje.

• Language: Slovenian
• Keywords: znanost, ekspertno ocenjevanje, kakovost znanstvene produkcije
• Work type: Dissertation
• Typology: 2.08 - Doctoral Dissertation
• Organization: FF - Faculty of Arts
• Place of publishing: Ljubljana
• Publisher: [D. Dekleva Smrekar]
• Year: 2021
• Number of pages: 180 f., [46] f. pril.
• PID: 20.500.12556/RUL-127708
• UDC: 001.89:005.585(043.3)
• COBISS.SI-ID: 67706115
• Publication date in RUL: 21.06.2021

Secondary language

• Language: English

Abstract

In the doctoral thesis the examination of the expert evaluation (peer review) systems is based on two aspects: an administrative one aimed at simplifying procedures and a substantive one aimed at maximizing the objectivity of the evaluation. The views of the three main actors involved are important: researchers (applicants), expert evaluators (peer reviewers) and administrators (professional assistants) leading the evaluation procedures. From the scientific literature the starting points that are important for understanding the operations of expert evaluation systems in science are summarized. The essential importance of scientific research is the creation of knowledge. It is decisively influenced by the existence of the scientific community, which is connected by the information process of scientific information and communication. The mechanism of expert (peer) evaluation is a historically grounded way of quality assurance in science, by which the scientific community ensures the autonomy and credibility of scientific findings. In addition to expertise, the human factor is also a decisive factor, so professional and personal ethical norms are of paramount importance. The performed empirical method is based on the experience on the basis of which certain phenomena in the operation of expert evaluation systems were identified and explained. The research was carried out as part of a national research grant system, as a combination of several research methods. An analysis of the regulation, a study of the test implementation of a special software tool for finding expert evaluators in tenders for research funding, an analysis of applications and an experiment in the use of this tool for those who are not experts in particular scientific fields were made. Through surveys and targeted interviews, the opinions of all three main actors in the expert evaluation systems were gathered. The results of the research provide guidelines for the design and assessment of expert evaluation systems in science and application possibilities for their rationalization and updating.