DORA aims to improve the ways in which the outputs of scholarly research are evaluated. The objectives of DORA are to 1) call attention to new tools and processes in research assessment and the responsible use of metrics that align with core academic values and promote consistency and transparency in decision-making; 2) aid development of new policies and practices for hiring, promotion, and funding decisions; 3) spread research assessment reform broadly by working across scholarly disciplines and globally; and 4) call for broader representation of researchers in the design of research assessment practices that directly address the structural inequalities in academia.

UCalgary signed DORA to change how the university assesses research. We believe that an enhanced focus on research content and impact, rather than on bibliometrics alone, is expected to benefit all types of research, from curiosity-driven inquiry to the most applied forms of investigation. This will also help to align policies and practices with those of leading funders, many of whom have also signed DORA.

In 2021, the Vice-President (Research) established a Research Impact Assessment Working Group to review existing research assessment practices and implement the recommendations of DORA at UCalgary. A 5-stage action plan was proposed to begin the DORA implementation across campus in 2021-2023. Meaningful change takes time, communication and collaboration, and we expect full DORA implementation to be iterative and ongoing in the coming years. 

As a researcher, you are encouraged to follow the DORA recommendations for researchers, and consider how they apply in your discipline and individual context. You are also invited to actively engage in the DORA implementation process in all phases - within your department, faculty or more broadly. You can share your questions, feedback, or recommendations with the team, or request more information at DORA@ucalgary.ca. Please check back regularly for updates, events and new opportunities.

Is DORA a mandate for future research assessment?

DORA is not a mandate or accreditation, but a commitment on behalf of signing individuals and institutions. In November 2022, DORA published a policy on engagement and outreach for organizational signatories. In this statement, DORA asks organizations that sign the declaration to share a public statement detailing their commitment to DORA and responsible research assessment to their communities. It also aims to ensure that an organization's implementation of the DORA principles is informed by ongoing dialogue with staff and students who are involved in research and research-enabling activities. 

Hiring, tenure and promotion, awards, and grant application:

As stated in the recommendations for institutions and individual researchers, DORA emphasizes transparent and responsible practices across all aspects of research assessment. This includes a shift away from journal-based metrics such as the Journal Impact Factor in assessing individual research contributions or researchers, and an emphasis on considering a broader range of research outputs and impacts. This will be interpreted in ways appropriate for each Faculty and Department, with guidance from relevant governing bodies for institutional processes (for example, academic tenure, promotion and merit). Importantly, researchers will still be expected to contribute high-quality original research outputs that peer scholars within that field recognize. 

DORA stands to benefit all researchers. The principle that researchers should be recognized for the full array of research outputs (papers, data, code, protocols, etc.) enables researchers to gain credit for work that matters to them and to their scholarly field. Ensuring greater transparency in research assessment promotes equity and appropriate assessment methods for researchers in diverse disciplines and at various career stages. Traditional scholarly research outputs continue to be highly valued and assessed based on their own merit, which maintains the integrity of important activities such as peer review. By considering a broader range of impact measures in research assessment, researchers who conduct important community-engaged work may look forward to clearer processes to assess and value these time-intensive research activities. 

Graduate students, postdocs, new faculty hires, etc.

As a Declaration, DORA represents a commitment by the University of Calgary to implement the recommendations across all aspects of the University that constitute research assessment – including early career researchers. DORA's emphasis on transparency in reaching hiring, tenure, promotion and merit decisions and assessing research outputs on their own merit rather than on journal-based measures such as the Journal Impact Factor is likely to be welcome news for early career researchers. 

Major Canadian funders CIHR, NSERC and SSHRC, as well as the Canada Foundation for Innovation (CFI) and Genome Canada, have all signed DORA, stating their commitment to applying DORA recommendations to funding applications. Individual grant applications are increasingly including reference to DORA, and careful attention should be paid to the specific criteria of each call. In 2022, NSERC updated their Guidelines on the assessment of contributions to research, training and mentoring. 

DORA encourages institutions to consider a broad range of impact measures, including qualitative indicators of research impact, such as influence on policy and practice. Although traditional research assessment practices do not always recognize connection to and impact on community, positive impacts of research on local communities are expected to be encouraged and promoted by research assessment following DORA recommendations.

The Journal Impact Factor (JIF) has been used as an indicator for research assessment for several decades. 

The JIF was developed to measure the quality of scholarly journals for the purpose of selecting journal subscriptions, but it has been mistakenly used to assess the quality of individual research papers in the past decades. Considering the skewed distribution of citations to journals, the JIF of a scholarly journal is attributed to very few highly cited papers; it does not accurately represent the quality of individual research papers published in a given scholarly journal on average (Larivière & Sugimoto, 2019). Thus, DORA highly recommends not using the JIF to assess individual research papers. In consideration of disciplinary differences, DORA does not provide a replacement to the JIF but recommends using a combination of various indicators (quantitative or qualitative) for research assessment across different disciplines. 

No. 

DORA objects to the abusive use of metrics (i.e. a single bibliometric indicator) in research assessment but recommends the responsible use of metrics. Since each bibliometric indicator has its applicable scope and limitations, DORA recommends selecting the most appropriate indicators in research assessment. 

Yes. 

Although traditional metrics (e.g., number of publications, number of citations, etc.) could represent the scholarly impact of research, they cannot be used to measure non-scholarly impact of research, including impact on economic development, society, community, policy and so on. Some non-traditional metrics (i.e., altmetrics) could be used in research assessment as a supplement. 

...particularly those in which the JIF is less used?

Although originally written to address concerns in the field of cell biology, DORA principles have much to offer in disciplines in which the JIF or journal publications are not the primary mode of research output or assessment. Importantly, DORA highlights the importance of assessing research on its own merits and applies a broad definition for research outputs and impact, not limited to journal publications. 

Specific requirements for hiring, tenure, promotion and merit will be determined by the appropriate policies and procedures in the researcher's Faculty and the University of Calgary GFC Academic Staff Criteria & Processes Handbook. According to the DORA Recommendations for Researchers, researchers should focus on the content of the scholarly contribution rather than simply publication metrics or the identity of the journal in which it was published. 

DORA also recommends researchers be recognized for research outputs in various formats; not only publications but also patents, software, license, dataset, reagents, intellectual property, highly trained new researchers, and so on. 

Meanwhile, researchers could demonstrate their different types of research impact, including not only the scholarly impact (citations, reach, uptake within the scholarly community) but also broader impact on policy, practice, community, etc. For journal publications, researchers are reminded to focus on article-level metrics rather than journal-level. 

For more details, please contact us at DORA@ucalgary.ca.

Various resources (bibliometric databases such as Scopus, Web of Science, etc.) or tools (e.g., various metrics, toolkits, etc.) may be used to help researchers report their research impact. We recommend UCalgary researchers use the tools and resources supported by Libraries and Cultural Resource to compile metrics regarding their research performance. 

Please visit the Manage Your Research Identity and Track Your Impact guide to learn more. The Knowledge Engagement Impact Assessment Toolkit provides a framework for evaluating the impact of partnered research both quantitatively and qualitatively, and can be adapted as desired. 

Researchers could use either altmetric indicators or narratives (e.g., narrative CV) to describe their non-scholarly impact including their research contribution to society, community, or policies. Consider combining both numbers and stories in order to provide a complete picture of your research contributions. Watch this page for new tools and resources for communicating research impact.

DORA applies a broad definition of research outputs, not limited to publications (journal articles, books, monographs, and book chapters). DORA encourages considering research outputs in various formats such as patents, software, licenses, datasets, training, reagents, public engagement, oral presentation, intellectual property, etc. that are relevant to the scholarly discipline.