Research funding and assessment conference
12 October 2006

Workshop facilitated by Lee Elliot Major (Times Higher)

Session one: DfES review and metrics

•	For metrics to be effective, they would need to be complex, which would diminish transparency & the original desire for simplicity in funding methodology
•	Measuring the volume of publication is not a good measure of research activity
•	Some research topics are fashionable and get lots of citations, but not others, regardless of research quality
•	Responses to the government’s RAE/metrics consultation indicate a desire for a range of metrics combined with expert review
•	It was felt that the government was looking for a type of metrics which favoured those HEIs the government wanted to receive the bulk of funding
•	UUK’s ‘Eureka’ publication showed that good research comes from all sorts of HEIs, not just the top 10 the Treasury wants to have the bulk of funding
•	A funding system without a judgment on quality would undermine the international reputation of UK research
•	UCU role to try and ameliorate the government’s desire to concentrate funding; to make any system user-friendly and inclusive of ordinary HEIs doing both teaching and research [as opposed to research-intensive ones]
•	Treasury ignoring evidence that post92 HEIs were more effective in drawing in other types of research funding e.g. applied. 
•	Desire to reduce dependence on public funds is making HEIs like UCL & LSE concentrate on overseas undergrads and postgrads
•	Metrics will be related to maximising output, but that would probably be accompanied by lowering of quality
•	An effective metrics system would put significant bureaucratic demands on HEIs, which would negate attempt to save money on the RAE

Session two: alternatives

•	While rejecting the RAE and metrics as proposed, the UCU survey indicated support for various forms of peer review of research together with a range of metrics including impact assessment and dissemination
•	There was a suggestion of a benchmark for a department including assessment of teaching, research and knowledge transfer/outreach, which might refocus priorities away from research and towards teaching
•	Also suggested were light touch rolling reviews, as opposed to the once-every-five-years RAE model, combined with a set of 5-10 principles/guidelines of what UCU would find acceptable in the assessment of research, including research-teaching synergy
•	It was important to build alliances with other groups in the sector where we had views in common; also important to have a succinct message which could be simply expressed
•	Important to draw attention to the negative impact on teaching of greater concentration of research funding, with fewer HEIs doing teaching in a research-rich environment
•	The move by some HEIs to a global status, with lots of overseas students and postgrads, could marginalise home students and undermine efforts to widen participation
•	Important to build around the negative effects of research concentration on widening participation: the more concentration through the proposed metrics system, the less likelihood of WP students having access to research-enriched teaching or the opportunity to be involved in research projects; WP students were more likely to live at home and go to the local HEI, which was unlikely to be a research-intensive one; it was important to widen access to research, at the undergrad and postgrad levels.

Workshop facilitated by Sean McWhinnie (Royal Society of Chemistry)

Session one: DfES review and metrics

•	There was no support for the ‘mono-metrics’ approach adopted in the DfES report.
•	Concern was also expressed about the potential timetable under consideration – e.g. metrics influencing funding from 2008-9. 
•	It was recognised that all forms of metrics – including citations - do not work as well in the humanities and social sciences. 
•	The importance of collaboration – both subject and institutional forms – was missing from the current debate. 
•	The potentially negative impact on institutional finances and career development (FTCs, new researchers, work-life balance) was also stressed. 
•	There was little need to heavily concentrate funding in areas like law and history – though it was more important in expensive laboratory and medical subjects. 
•	Some people argued that the UCU should oppose all forms of ‘metrics’; others felt their arrival was inevitable and that we should push for a more sophisticated ‘basket of metrics’ alongside peer review.  
•	Implementing a robust set of metrics results in ‘escalating complexity’ and may not be compatible with the government’s shortened timescale. 
•	Metrics can be more responsive than the RAE – for example, the DfES models show that post 92 universities are punching above their weight in terms of grant funding.  
•	Others were sceptical about the redistributive effects of metrics. The historical bias in funding was unlikely to be reversed and new researchers, subject areas and even institutions were unlikely to do well under the DfES proposals. On the contrary, the real Treasury goal was likely to be further concentration, i.e. all research funding going to only 10 rather than 25 universities.  
•	The Treasury’s longer-term goal may be the end of the dual support system – with a Tertiary Education Funding Council dealing with all HE/FE teaching funding and all HE research funding allocated via the research councils and the DTI’s, Office of Science and Innovation.  
•	Some questioned whether it would be completely negative to funnel all funding via the research councils, e.g. the criteria on assessing impact can be more progressive than research funded out of the block grant.  

Session two: alternatives

•	Contrary to comments in the plenary session, the RAE had not been particularly kind to university chemistry – in fact, it had contributed to some departmental closures. 
•	There is a need to tackle subject and institutional divisions within HE research. Some positive examples include small-scale academic co-ops (Sweden) and possibility of strategic alliances amongst related disciplines (e.g. sociology, politics, and psychology).  
•	Important to learn from the more collaborative Scottish model – particularly in the physical sciences. 
•	More ‘capacity building’ funds are needed – for example, for new researchers, new disciplines and for cross-institutional co-operation. 
•	The Californian model of higher education is another possible model. It involves a greater institutional separation of the research and teaching functions but teaching-led universities are not seen as ‘third tier’. 
•	Some support for a return to per capita, non-competitive funding for research, particularly in arts & social sciences. 
•	Others felt it is no longer viable to separate the allocation of research funding from some form of quality assurance. The RAE had been successful partly because it tapped into ‘practitioner power’, i.e. the academy’s desire to rank and grade research. 
•	The UCU should open up the debate over what is meant by research- for example, clinical academic training, or the inclusion of textbooks in the RAE. This fed into a wider debate about what is meant by higher education and what is means to be a university lecturer.  
•	Research assessment system should incorporate the economic and social value of HE research.  
•	Need to ‘humanise’ the peer review system – for example, wider array of practitioners and users.  

Workshop facilitated by Dennis Hayes (UCU)

Session one: DfES review and metrics

•	Various views about why the government has intervened in the RAE: recognition that it’s not working; Treasury “efficiency” agenda; attack on peer review.
•	There needs to be some system of measures, although some members were opposed to the linking of assessment to funding.
•	Use of metrics likely to further distort assessment in favour of articles in “top” journals and away from other forms of publication; implications for academic freedom.
•	If we are going to be lumbered with metrics we should try to make their use as palatable as possible.
•	Little support for any of the five models suggested by the government.
•	Any alternative should apply across all subject disciplines, although there was also a case made for differential treatment, particularly in relation to the arts and humanities.
•	Interdisciplinary research may suffer under metrics.
•	There should be a rolling programme of assessment rather than a “big bang” RAE assessment every five or six years.
•	Although peer review has its faults, UCU should argue for its retention and for its continuation as the core method of research assessment.
•	Danger of duplication of effort in the collection of research-related data; the various sources and types of data should be rationalised and streamlined.

Session two: alternatives

•	Should UCU play according to the government’s terms and engage with their metrics agenda? Or should we simply argue for the abolition of the RAE and concentrate on campaigning for proper funding for all higher education teaching, research and scholarship? What is the political scope for achieving fundamental change?
•	Some members favoured a campaigning approach; others argued that some system of research assessment was inevitable given the need for universities to be accountable for taxpayers’ money.
•	If the academic community could not agree on an alternative to the RAE, the dual support system might not survive; transfer of all government research funding to the research councils would be worse than the RAE.
•	Of Roger Brown’s alternatives, there was some support for a return to some form of block funding provided that funding was increased significantly. The need to broaden the definition of research to embrace the full range of scholarly activities was strongly supported. There was also support for the idea of limiting RAE-type funding to the most expensive areas of science and engineering.
•	The meeting discussed the question of whether it was possible to imagine a fair system of research assessment which would be inclusive both of staff and of all research/scholarship outputs. Could the UCU come up with such a model?
•	A fair system of assessment might differ according to subject; it may need to embrace work related to, for example, social inclusion and community/business links, rather than just traditional academic research. There might be a case for bringing together currently separate funding streams.
•	Self-assessment might figure as a major element in an alternative system as in the Netherlands.
•	The UCU should make the case for a broad understanding of how research and scholarship contribute to the public good and not accept the government’s narrow utilitarian outlook.