Declaration On Research Assessment (DORA)
Knowledge to Impact
Motivation for Change: Pursuing Impact
The Declaration on Research Assessment (DORA) is a global initiative to encourage the development and promotion of best practices in evaluation of researchers and the outputs of scholarly research.
Research assessment shapes research careers, institutional decisions, and academic cultures. The Declaration affirms a university-wide commitment to societal impact, with validation of multiple forms of output from UCalgary research, scholarship, and innovation activities. DORA also emphasizes the university’s accountability to rigorous assessment and reporting of impact both internally and externally. DORA-aligned research assessment will promote clearer processes of assessment and will enable researchers to be recognized for diverse outputs and impact practices that matter to them, their scholarly field, and to key users of their work allowing them to communicate their impact more clearly to funders, collaborators, partners or philanthropists.
DORA Implementation at UCalgary
The University of Calgary signed DORA in January 2021, joining thousands of institutions and individuals globally. Major Canadian funders such as NSERC, CIHR, SSHRC, Canadian Foundation for Innovation and Genome Canada have also signed DORA.
The University of Calgary adheres to the principles of the San Francisco Declaration on Research Assessment (DORA) and strives to responsibly and inclusively evaluate the research quality, significance, and impact of current and prospective academics. We do this by explicitly considering the value and impact of various activities, outputs, and scholarly contributions that are appropriate to the field or discipline and to key beneficiaries when assessing research and scholarship activities.
UCalgary Policies, Plans and Resources
GFC Academic Staff Criteria & Processes Handbook (2024)
Faculty Guidelines
Research and Innovation Plan
Academic Innovation Plan
Academic Selection Toolkit
Good Research Assessment Practices at UCalgary
Guided by DORA
Responsible assessment
Bibliometric indicators are not used or considered in isolation but rather in combination with qualitative methods for demonstrating evidence of achievements. More specifically, journal-based metrics are not used for research assessment of individuals (if relevant to that Faculty) and instead discipline-appropriate quantitative and qualitative indicators such as document-level indicators, peer recognition, advancement to the discipline, innovation, creativity, and/or impact on society and community are considered.
Transparent assessment
Transparency of the research assessment process through an open and clear overview of how quality and impact are considered. For academic staff, the GFC Academic Staff Criteria & Processes Handbook and Faculty Guidelines provides clear descriptions of appropriate venues, methods, communication outlets, and when collaborations with academic and non-academic actors are recognized.
Inclusive assessment
Agreed-upon criteria for assessment will depend on the academic selection decision and also consider the individual’s scholarly context (their scholarly field, time demands to reach full development, community engagement demands, and the probability of use/ impact on key intended partners and users). See our Most Significant Contribution Statement Guide on how to highlight an individual’s strengths for assessment decisions.
Joining DORA is another step toward more meaningfully evaluating research output and impact. Our scholars are increasingly engaged in new forms of knowledge creation, and as a university we have emphasized the need to better demonstrate the impact of our research. Including an assessment of impact along with more traditional metrics will help us to fully recognize the value of their work.
Dr. Marcello Tonelli
Professor, Cumming School of Medicine,
Former Associate Vice-President (Research)
Additional Resources | DORA
DORA Reference Links
You are invited to actively engage in the DORA implementation process in all phases - within your department, faculty or more broadly. You can share your questions, feedback, or recommendations with the team, or request more information at: knowledge.impact@ucalgary.ca
- Designing a responsible research assessment process
SCOPE Framework for Research Evaluation | INORMS - Rethinking Research Assessment (DORA resources)
The three major federal funding agencies (CIHR, NSERC and SSHRC) will be gradually transitioning to a new narrative-style CV for competitions. Narrative CVs prioritize written descriptions of an applicant’s research contributions, allowing users to highlight a wide range of research outputs and to describe their career trajectories in more detail. This format values societal research outcomes, such as influence on policy or mentorship, alongside more traditional research outputs like publications. See: Tri-Agency CV
The Most Significant Contributions Statement is a core component of many narrative CVs, including the Tri-Agency CV. The Most Significant Contributions Statement Guide helps researchers create statements to communicate the quality, significance and impact of their work.
UCalgary Libraries & Cultural Resources:
- Measuring Research Impact
- Manage Your Research Identity and Track Your Impact
- Altmetrics
- Scholarly Communication
The Knowledge Engagement Impact Assessment Toolkit provides a framework for evaluating the impact of partnered research both quantitatively and qualitatively and can be adapted as desired.
CIHR - including guidance for applicants, guidance for reviewers, FAQs and more.
NSERC - including revised guidelines for assessment of contributions
SSHRC - including merit review
Tri-Agency CV Announcement | FAQs (CIHR)
Manage Your Research Identity and Track Your Impact
Faculty Library Support Guide – Scholarly Publishing
Find recordings from our past sessions below:
The Evolution of Research Assessment and Responsible Use of Metrics, Dr. Vincent Larivière.
View Recording
Stop Using the h-index: Educating Researchers About Scholarly Metrics, Dr. Stefanie Haustein.
View Recording
Tools for Advancing Research Assessment, Ruth Schmidt.
View Recording
DORA FAQs
Why does DORA recommend not using journal impact factor?
The journal impact factor (JIF) was developed for the purpose of selecting journal subscriptions, but it has been mistakenly used to assess the quality of individual research papers in the past decades. Considering the skewed distribution of citations to journals, the JIF of a scholarly journal is attributed to very few highly cited papers; it does not accurately represent the quality of individual research papers published in a given scholarly journal (Larivière & Sugimoto, 2019). Additionally, the JIF is subject to and a reflection of systemic biases (Davies, et al. (2021). Decisions based on JIF and journal prestige risk reflecting systemic biases in their hiring and promotion processes.
Nobel laureates (Schekman, 2013; Nobel Prize, 2017) are speaking out against JIF
Will assessment reforms that accept both traditional and non-traditional research outputs will dilute the meaning of research excellence and quality?
The call for more responsible and fair evaluation requires the development of standards and structure in the assessment process. An improved research assessment approach will discourage overreliance on purely quantitative comparisons and instead be based on intentional and meaningful decisions that consider a range of appropriate quantitative and qualitative measures as supporting evidence of broad research objectives and impact practices
One of the principles of DORA is to avoid reliance on impact factors and focus on the quality of the scholarly output. How do we assess this in the context of merit or promotion review in a heterogeneous faculty?
Quality might take different forms under different research objectives. DORA-aligned clarifications have been included in the University of Calgary GFC Academic Staff Criteria & Processes Handbook (lastly updated in July 2023). For example:
- 1.2.3 "...High-quality research and scholarship will be measured by peer recognition and/or advancement to the discipline, and/or innovation, and/or creativity, and/or impact on society and community etc".
- 1.2.4 " Activities in research and scholarship vary among Faculties... Such activities shall normally be measured by the quality, originality, innovation, impact, entrepreneurial spirit, knowledge engagement and community impact, and the pattern of the academic staff member’s work appropriate to the discipline, field, or community."
How do we continue to reward excellence when people put in a lot of time, work, and effort to publish in high-impact journals?
Under DORA we still value the work and effort that was put into publishing in more competitive venues, but we value it because it is a high-quality work and not because of the impact factor or the prestige of the journal. Traditional scholarly research outputs continue to be highly valued and assessed based on their own merit, which maintains the integrity of important activities such as peer review.