The Responsible Research and Innovation (RRI) framework that was launched by the European Commission has the aim to ensure that researchers work together with citizens, policy makers, businesses and third organisations during the whole research and innovation process. For initiatives within this framework to be successful, recognition is needed of the broad range of activities researchers engage in. The academic reward system in this sense can play an important role in whether true responsible research and innovation is achieved, and conversely, a lack of recognising RRI approaches would obstruct this goal.
During the Marie Curie Alumni Association (MCAA) annual conference in Vienna, Karen Stroobants, vice-chair of the MCAA policy working group, went in conversation with David Bogle (Pro-Vice-Provost of the University College London Doctoral School), Anne Loeber (A/Prof University of Amsterdam, NewHoRRIzon), Katrien Maes (Deputy Secretary-General of League of European Research Universities), Gareth O’Neill (president of European Council for Doctoral Candidates and Junior Researchers) and Katie Wheat (Head of Higher Education Engagement, Vitae) to discuss research(er) evaluation under the RRI framework. Here I summarise the main points discussed.
The Responsible Research and Innovation framework as an opportunity to reward more than publications
The evaluation of researchers still more often than not is heavily focused on the publications a researcher has produced, and the journal in which those publications are published (e.g. via the journal impact factor – JIF). Researchers of course are engaged in a much wider spectrum of activities, but those count substantially less than “high impact” publications when they are evaluated. Even where there is positive movement, a general perception held by researchers sustains the perverse expectation to publish, as ‘high impact’ as possible (despite the well-recognized detrimental effects on research of this ‘impact factor game’), and as much as possible. In contrast, when researchers discuss what they truly value, at various events and internal conversations in their research group, excellent leadership, engagement and diversity are since many years mentioned as often as qualitative publications. However words are only rarely met with actions, and the scale of what we measure continues to give disproportionate weight to publications. Rather than seeing this as a barrier to reach RRI, the panel suggested that a conscious desire to reach RRI can form a real opportunity to change the reward system, which will be the beginning, rather than the end, for changing research culture as a whole.
Signing declarations won’t be enough, transparency and accountability need to be institutionalised
But changing culture is easier said than done. The San Francisco Declaration of Researchers Assessment and the Leiden Manifesto, both aimed at moving away from researcher evaluation based on the JIF, have been around for some time. Yet many funders and research institutions have not signed these expressions of commitment, and many of those who did sign arguably have not been able to make the necessary changes to honour their commitment. Indeed, signing any declaration will not be enough; researchers need leadership teams at their institutions and funding bodies to take responsibility and drive change. There are efforts underway to replace the JIF by more ‘responsible’ metrics, workshops to address mental health issues of researchers are in abundance, and each institution promotes initiatives to fix the gender balance in science. Yet what we really need is more responsible people, a reward system that values emphatic leaders whose first concern is the best support for the next generation rather than their own prestige, and evaluation procedures that institutionalise diversity.
Towards recognition of diverse profiles rather than extending the box-ticking exercise
The diversity mentioned does not only refer to diversity in the sense of ethnic and gender diversity, but also refers to the kind of profiles that are considered for academic positions. If we want to reward a broader range of activities; open science, ethics, public and policy engagement, teaching, leadership etc, that should not mean that every researcher needs to do all of those things. Instead of extending the box-ticking exercise, departments and funders should for each position they offer, define what is needed; will this be a leadership position, or are we looking for someone who can play a considerable role in outreach activities? What is evaluated should be in line with the competencies that will be needed to fulfil the role. And if in their totality, we expect research teams to take an RRI approach, positions that focus proportionally more on excellence in leadership, science communication, open data management etc, should exist alongside each other. Researchers in this context need to learn how to develop a narrative around their own set of skills, which only will happen if this is what is expected of them to get hired or promoted.
Wider reward systems in practise, universities in the Netherlands and the UK are leading the way
Several funders and institutions, particularly in the UK and the Netherlands, are at the forefront of driving change, and a lot can be learned from them already. In the UK, funders such as UK Research and Innovation and Wellcome Trust are putting research culture forward as one of their major areas of work, and institutions like University College London already look at teaching, enterprise, public engagement and institutional citizenship alongside research quality in evaluation and promotion processes. In the Netherlands, institutions such as the University Medical Centre, University of Utrecht have been ahead of their time, using a portfolio-based evaluation process for the promotion of researchers since many years, and the VSNU (Association of Universities in the Netherlands), NWO (Netherlands Organisation for Scientific Research), NFU (Netherlands Federation of University Medical Centres) and ZonMw (Netherlands Organisation for Health Research and Development) have recently expressed their commitment to search for a new approach to recognising and rewarding academics. They have announced their support for renewal of existing frameworks for scholarly recognition and reward, with a focus on three areas; differentiation of career pathways, new research assessment methodologies and team science. Change is clearly on the horizon, and it will be up to each funder and institution to choose whether they want to widen their definitions of success to stimulate broad talent development and boost quality and impact, or whether they want to stay behind until the system forces them to follow suit.
An edited version will be published on the Marie Curie Alumni Association blog Medium
