FAQ

Why does DORA recommend not using journal impact factor?

The journal impact factor (JIF) was developed for the purpose of selecting journal subscriptions, but it has been mistakenly used to assess the quality of individual research papers in the past decades. Considering the skewed distribution of citations to journals, the JIF of a scholarly journal is attributed to very few highly cited papers; it does not accurately represent the quality of individual research papers published in a given scholarly journal (Larivière & Sugimoto, 2019). Additionally, the JIF is subject to and a reflection of systemic biases (Davies, et al. (2021). Decisions based on JIF and journal prestige risk reflecting systemic biases in their hiring and promotion processes.  

Nobel laureates (Schekman, 2013; Nobel Prize, 2017) are speaking out against JIF 

Will assessment reforms that accept both traditional and non-traditional research outputs will dilute the meaning of research excellence and quality? 

The call for more responsible and fair evaluation requires the development of standards and structure in the assessment process. An improved research assessment approach will discourage overreliance on purely quantitative comparisons and instead be based on intentional and meaningful decisions that consider a range of appropriate quantitative and qualitative measures as supporting evidence of broad research objectives and impact practices

One of the principles of DORA is to avoid reliance on impact factors and focus on the quality of the scholarly output. How do we assess this in the context of merit or promotion review in a heterogeneous faculty?    
 

Quality might take different forms under different research objectives. DORA-aligned clarifications have been included in the University of Calgary GFC Academic Staff Criteria & Processes Handbook (lastly updated in July 2023). For example:

1.2.3 "...High-quality research and scholarship will be measured by peer recognition and/or advancement to the discipline, and/or innovation, and/or creativity, and/or impact on society and community etc". 

1.2.4 " Activities in research and scholarship vary among Faculties... Such activities shall normally be measured by the quality, originality, innovation, impact, entrepreneurial spirit, knowledge engagement and community impact, and the pattern of the academic staff member’s work appropriate to the discipline, field, or community." 

How do we continue to reward excellence when people put in a lot of time, work, and effort to publish in high-impact journals?  

Under DORA we still value the work and effort that was put into publishing in more competitive venues, but we value it because it is a high-quality work and not because of the impact factor or the prestige of the journal. Traditional scholarly research outputs continue to be highly valued and assessed based on their own merit, which maintains the integrity of important activities such as peer review.