Stikkord: SCOPE

Stikkordarkiv: SCOPE

Brand new INORMS Research Evaluation Group Guide to SCOPE Framework for responsible research evaluation

“The guide is designed by practitioners for practitioners seeking to design and run responsible research evaluations in their organisations. We believe the SCOPE framework bridges the gap between high-level statements of principle for responsible metrics, and the actual practical implementation of those principles through the design and delivery of responsible evaluations. We hope this new guide, complete with practical case studies, will be a significant help to busy practitioners seeking to deliver on their responsible research evaluation aspirations”, Tanja Strøm, Deputy Chair of INORMS Research Evaluation GroupThe International Network of Research Management Societies (INORMS)

Research Evaluation Group (REG) has released November, 22 an 18-page guide to their framework for responsible research evaluation called SCOPE.

Established in 2018, the INORMS REG has been working to better understand and develop best practices in responsible research evaluation with the aim of providing practitioners and senior leaders the tools they need to effectively and responsibly evaluate research. The SCOPE Framework for responsible research evaluation resulted from their work.

Over the past year the Group have partnered with a range of funders, institutions and publishers to apply SCOPE to a series of real-life evaluation problems with the aim of testing the model. The guide draws on the learnings of these explorations, many of which have been included as case studies. The aim of the guide is to assist any organisation seeking to design a responsible research evaluation. It was designed by their publishing partner, Emerald Publishing, and is now available as an open resource under a CC-BY licence.

The SCOPE framework for responsible research evaluation comprises of five stages:

START with what you value
CONTEXT considerations
OPTIONS for measuring
PROBE deeply
EVALUATE your evaluation

This content is available online, including one-page summaries in various languages, and is available for no charge. In addition to the SCOPE guide, additional materials have been developed including fuller case studies using the framework from a variety of international contexts including research universities, publishing companies, and funding agencies. More information and materials can be found online here: https://inorms.net/research-evaluation-group/.

The REG is available for additional workshops and program development in the use of SCOPE. Contact the group by reaching out to the Chair, Dr. Elizabeth Gadd, e.a.gadd@lboro.ac.uk or the Deputy Chair, Tanja Strom, tanja.strom@oslomet.no.

Rating the Rankers!

Initial findings from an international research evaluation working group suggests that the organisations behind university world rankings merit some scrutiny themselves.

Looking at six of the largest and most influential world university rankings, members of the International Network of Research Management Societies (INORMS) Research Evaluation Working Group used their five-step process called SCOPE to assess them on a number of community-developed criteria centred around good governance, transparency, measuring what matters and rigour.

The group found that while most of the ranking organisations made some efforts towards good governance, there were clear weaknesses in terms of declaring conflicts of interest.

The rankers’ aims and methods were generally transparent, although this was not necessarily borne out by others’ ability to replicate the data, data availability or financial transparency.

Most rankings underperformed when it came to measuring what matters, all failing to tailor their offer to different audiences and showing unfair bias to some groups. Finally, university rankings, which are most criticised for their methodological invalidity, generally scored very poorly when it came to implementing rigorous methods.

Convenor of the INORMS Research Evaluation Working Group, Dr Lizzie Gadd, said: “there is clearly work to be done here, and we hope that our rating clearly highlights what needs to be done and by whom. The world university rankings currently fail to meet community expectations around fair, meaningful, and responsible evaluation. We hope that this work will provide ranking organisations, and those that rely on them for decision-making, an opportunity to reflect and reconsider their approach.”

For further information, Rethinking the rankings, a blogpost by Lizzie Gadd and Richard Holmes can be found on the ARMA website at https://arma.ac.uk/rethinking-the-rankings/

Contact Lizzie Gadd at e.a.gadd@lboro.ac.uk, Twitter: @lizziegadd

For further information on INORMS, visit www.inorms.net.

INORMS har to norske medlemmer i dette arbeidet; professor Nils Pharo og seniorrådgiver Tanja Strøm ved OsloMet

The INORMS Research Evaluation Working Group’s two work-packages

The INORMS Research Evaluation Working Group’s two work-packages are now nearing completion and we have had a lot of positive feedback about our work. Of particular interest has been the development of the SCOPE model – a five stage process for evaluating responsibly. We are now keen to understand how this process is being used and by whom, so we can share case studies with the wider community and build on this work. To this end, if you’re using the SCOPE model in any way – for training, for assessing existing evaluations or developing new ones, we’d really love to hear from you! Similarly, if you’re not yet using it, but would like to provide a comment on its value or how you see it being used, do get in touch.

You may either respond to tanja.strom@oslomet.no, or directly to sub-group leader, Laura Himanen (TAU) laura.himanen@tuni.fi

We look forward to hearing from you!

A webinar from the INORMS Research Evaluation Working Group: SCOPE: A five-step process for evaluating responsibly

The International Network of Research Management Societies (INORMS) established the Research Evaluation Working Group in 2018 to consider how to best ensure that research evaluation is meaningful, responsible and effective. The Working Group is truly global, it has members from all continents, with the exception of Antarctica!

The Research Evaluation Working Group invites research managers and other research evaluation practitioners involved in planning, implementing and interpreting research evaluations to a webinar on ‘SCOPE’ – a five step process for evaluating responsibly.

The webinar will be held on Monday 20^th April 2020 at 1-2 pm BST [In Norway at 2-3 pm]

San Francisco Declaration on Research Assessment (DORA), the Leiden Manifesto and the Metric Tide give the Higher Education sector a framework for evaluating research responsibly. However, practitioners who plan and organize evaluations may still be left asking “now what?”. How do we actually do research evaluation responsibly?

The INORMS Research Evaluation Working Group has developed a five-stage process for evaluating responsibly, that may answer that question! The process is called ‘SCOPE’.This webinar presents the SCOPE model in detail and provides a practical framework for research managers, and others, seeking to conduct responsible research evaluations.

The webinar can host up to 150 participants. All who register will receive the link to the webinar, and the first 150 to join the webinar can attend live. A recording of the webinar will be sent to all who register.

Hope you can join us!

The INORMS Research Evaluation Working Group

For more information, please contact Laura Himanen (laura.himanen(at)tuni.fi)