Stikkord: SCOPE

Stikkordarkiv: SCOPE

Rating the Rankers!

Initial findings from an international research evaluation working group suggests that the organisations behind university world rankings merit some scrutiny themselves.

Looking at six of the largest and most influential world university rankings, members of the International Network of Research Management Societies (INORMS) Research Evaluation Working Group used their five-step process called SCOPE to assess them on a number of community-developed criteria centred around good governance, transparency, measuring what matters and rigour.

The group found that while most of the ranking organisations made some efforts towards good governance, there were clear weaknesses in terms of declaring conflicts of interest.

The rankers’ aims and methods were generally transparent, although this was not necessarily borne out by others’ ability to replicate the data, data availability or financial transparency.

Most rankings underperformed when it came to measuring what matters, all failing to tailor their offer to different audiences and showing unfair bias to some groups. Finally, university rankings, which are most criticised for their methodological invalidity, generally scored very poorly when it came to implementing rigorous methods.

Convenor of the INORMS Research Evaluation Working Group, Dr Lizzie Gadd, said:there is clearly work to be done here, and we hope that our rating clearly highlights what needs to be done and by whom. The world university rankings currently fail to meet community expectations around fair, meaningful, and responsible evaluation. We hope that this work will provide ranking organisations, and those that rely on them for decision-making, an opportunity to reflect and reconsider their approach.”

For further information, Rethinking the rankings, a blogpost by Lizzie Gadd and Richard Holmes can be found on the ARMA website at

Contact Lizzie Gadd at,  Twitter: @lizziegadd

For further information on INORMS, visit

INORMS har to norske medlemmer i dette arbeidet; professor Nils Pharo og seniorrådgiver Tanja Strøm ved OsloMet

The INORMS Research Evaluation Working Group’s two work-packages

The INORMS Research Evaluation Working Group’s two work-packages are now nearing completion and we have had a lot of positive feedback about our work.  Of particular interest has been the development of the SCOPE model – a five stage process for evaluating responsibly. We are now keen to understand how this process is being used and by whom, so we can share case studies with the wider community and build on this work.  To this end, if you’re using the SCOPE model in any way – for training, for assessing existing evaluations or developing new ones, we’d really love to hear from you!  Similarly, if you’re not yet using it, but would like to provide a comment on its value or how you see it being used, do get in touch.

You may either respond to, or directly to sub-group leader, Laura Himanen (TAU)

We look forward to hearing from you!

A webinar from the INORMS Research Evaluation Working Group: SCOPE: A five-step process for evaluating responsibly

The International Network of Research Management Societies (INORMS) established the Research Evaluation Working Group in 2018 to consider how to best ensure that research evaluation is meaningful, responsible and effective. The Working Group is truly global, it has members from all continents, with the exception of Antarctica!

The Research Evaluation Working Group invites research managers and other research evaluation practitioners involved in planning, implementing and interpreting research evaluations to a webinar on ‘SCOPE’ – a five step process for evaluating responsibly.

The webinar will be held on Monday 20th April 2020 at 1-2 pm BST [In Norway at 2-3 pm]

San Francisco Declaration on Research Assessment (DORA), the Leiden Manifesto and the Metric Tide give the Higher Education sector a framework for evaluating research responsibly. However, practitioners who plan and organize evaluations may still be left asking “now what?”. How do we actually do research evaluation responsibly?

The INORMS Research Evaluation Working Group has developed a five-stage process for evaluating responsibly, that may answer that question! The process is called ‘SCOPE’.This webinar presents the SCOPE model in detail and provides a practical framework for research managers, and others, seeking to conduct responsible research evaluations.

Register for the webinar

The webinar can host up to 150 participants. All who register will receive the link to the webinar, and the first 150 to join the webinar can attend live. A recording of the webinar will be sent to all who register.

Hope you can join us!

The INORMS Research Evaluation Working Group

For more information, please contact Laura Himanen (laura.himanen(at)