LM provides inspiration for Ghent University vision for research evaluation

While the Leiden Manifesto is widely respected as a statement of best practice in applying metrics to research evaluation, many struggle with its application.  Insight into how to design an evaluation system inspired by the Leiden Manifesto can be sought in Ghent University’s vision for evaluation of its research.  The LM provided a reference and source of inspiration for Ghent’s deliberation. 

The discussions at Ghent surfaced these strengths of the LM:

• The recommendations are concrete and immediately applicable
• There is no polarization of quality versus quantity, nor of metrics versus peer review methods
• Emphasis on the research mission of the individual/group/institution
• Importance of differentiation between disciplines
• Multiple indicators rather than focus on one single (or one composite) indicator

In addition, these limits were noted:

• The Leiden Manifesto is a good start but other elements also play a part when evaluating research
• We should attend to the process of research when we evaluate (e.g. data management, leadership, team work, scientific integrity) rather than merely focus on the product
• We also need standards/methodologies for peer review, not just for metrics. Something is not “excellent” because an expert says it is “excellent”
• Institutions do not work in isolation, nor are they entirely autonomous. If national funding mechanisms or allocation models reward particular performances or favour certain metrics such as impact factors, it is impossible to develop internal recommendations against this. At best, you can set up a ‘safe haven’ within your institution where you try to limit but cannot entirely wipe out these pressures. This is particularly difficult in Belgium where funding is heavily dependent on output-based performance indicators.

 
Ghent University's Board of Governors agreed on eight building blocks for a quality evaluation of research:

1. The choice of an appropriate evaluation method for research is in line with the objective of the evaluation.
2. The evaluation takes into account the intended impact of the research; strictly academic, economic, societal, or a combination of these.
3. The evaluation takes into account the diversity between disciplines.
4. For each chosen evaluation method, the simplicity of the procedure is weighed up against the complexity of the research
5. The evaluation criteria are drawn up and communicated to all stakeholders in advance.
6. There are sufficient experts on the evaluation committee who are in a position to adequately assess the quality of the research
7. The above principles are implemented by means of a smart choice of evaluation indicators and by adopting a holistic approach to peer review.
8. Any committee or policy measure evaluating research, makes a best effort commitment to translate the above principles into practice.

 A translation of the full vision statement is available here.