Periodic Evaluation

The Periodic Evaluation of Research, Development, Artistic and Other Creative Activities (hereinafter referred to as the “Periodic Evaluation”) is a peer-review assessment of the publication and other creative activities of Slovak scientific and research institutions (public universities and public research institutions). The inspiration for this reform came from proven foreign systems of science evaluation, in particular the British Research Excellence Framework (abbreviated as the “REF”).

Thanks to the Slovak Republic’s Recovery and Resilience Plan, in 2022, panels and subpanels composed mainly of foreign experts from 19 countries and 66 universities (e.g. Princeton, McGill, Melbourne, Cambridge, TU Munich, ETH Zurich, Paris-Saclay, Sapienza, etc.) were established to assess the level of Slovak science from the perspective of the international scientific community, and thus provide us with insights into the performance and creative output of the Slovak science and research ecosystem.

The subject of the assessment were the workplaces of Slovak public universities (hereinafter referred to as “HEIs”) and public research institutions (hereinafter referred to as “PRIs”) and their research publication outputs, as long as the workplace had at least 5 male and female scientists (employed full-time in the case of HEIs, or at least half-time in the case of PRIs, in each case they had to have been working at the workplace for at least four years during the period under review) in the period 2014-2019.
Each HEI and PRI, or parts thereof, could submit applications for evaluation in 28 research areas (falling under 7 groups of research areas). Each application contained a maximum of 25 outputs (scientific articles, monographs, conference papers, artistic outputs, etc.) in a specific research area. A maximum of 5 outputs were submitted for each scientist. In case an individual did not produce 5 outputs, so-called ‘zero outputs’ were added for him/her.

Example: a faculty of a selected HEI has applied for a periodic evaluation in the research area ‘chemical sciences’. In the application for this research area, it has listed five staff members who fulfilled the conditions for inclusion [a minimum of 48 months of 100 per cent full-time service at the faculty concerned during the years 2014-2019]. Four of the five staff members produced at least five outputs (for each staff member) during the periodic review period, while the fifth included staff member produced three scholarly outputs during the periodic review period. Therefore, a total of 23 creative outputs (five outputs each from four of the staff members, and three from the staff member with quantitatively lower output during the period under review) and two zero outputs were included in the periodic review request for the research area ‘chemical sciences’.

The final selection of outputs for each periodic review request was set on the basis of an attempt to respect the principles of representativeness and evenness.

In order to comply with the representativeness criterion, the applicant had to select from a list of all outputs (produced during the period under review by staff eligible for the periodic review in the research area concerned) a final selection of 25 outputs, based on the chosen substantive criterion for ranking the outputs. According to Article 2(g) of Directive No 36/2022 on the periodic evaluation of research, development, artistic and other creative activities, the substantive criterion for the ranking of outputs is a justified principle for the ordering of outputs, which may also include the number of references, the categories of bibliographic classifications, the scientometric data of periodical publications and non-periodical publications and, where appropriate, combinations of these. This means that the applicant had to rank and divide the outputs of all staff members included in the evaluation into groups according to the substantive criterion chosen by the applicant, and then select an equal (or approximately equal) number of outputs from each of these groups for the final selection of outputs.

Example: A particular HEI submits a periodic evaluation application in which all five staff members under evaluation report a total of 150 outputs for the years 2014-2019. The institution has decided to consider the number of citations for each of the outputs as a ranking criterion, and to rank all those outputs according to this criterion, and then to divide them into five groups (from the group of outputs with the highest number of citations to the group of outputs with the lowest number of citations; with 30 outputs in each of the five groups). In order to comply with the principle of representativeness, a particular HEI had to subsequently include in the final selection (with 25 outputs to be assessed for their quality) five outputs from each of the five groups, which were created and ranked with respect to the chosen substantive ranking criterion.

In order to respect the principle of evenness, the final selection for each application did not include only the best outputs, but tried to reflect the creative publishing activity of the whole institution, or the whole research unit in a given field of research (i.e. the activity of each staff member listed in the application for periodic evaluation who met the conditions for inclusion in the periodic evaluation).

Example: The selected PRI has applied for a periodic evaluation in the area of ‘legal studies’, including 7 staff members who meet the eligibility criteria. In this case, in order for the selection of outputs to meet the criterion of evenness, the applicant must submit four outputs for four of these staff members and three outputs for three of these staff members for the final selection (25 outputs in total).

In accordance with Section 26aa of Act No 172/2005 Coll. on the organisation of State support for research and development, on the basis of a call issued by the Ministry of Education, Science, Research and Sport, HEIs and PRIs shall undergo a periodic evaluation of their research, development or artistic and other creative activities (hereinafter referred to as ‘periodic evaluation’) in the individual fields of research once every six years. Only the quality of outputs shall be assessed in the current periodic evaluation. However, according to Article 3(2) of Directive No 36/2022 on the periodic evaluation of research, development, artistic and other creative activities, in future the periodic evaluation will also include an assessment of the societal impact of research, development, artistic and other creative activities, as well as an assessment of the environment of research, development, artistic and other creative activities.