Research Assessment Exercise
Encyclopedia
The Research Assessment Exercise (RAE) is an exercise undertaken approximately every 5 years on behalf of the four UK higher education
funding councils (HEFCE
, SHEFC, HEFCW, DELNI
) to evaluate the quality of research undertaken by British higher education institutions. RAE submissions from each subject area (or unit of assessment) are given a rank by a subject specialist peer review panel. The rankings are used to inform the allocation of quality weighted research funding (QR) each higher education institution receives from their national funding council. Previous RAEs took place in 1986, 1989, 1992, 1996 and 2001. The most recent results were published in December 2008.
Various media have produced league tables of institutions and disciplines based on the 2008 RAE results. Different methodologies lead to similar but non-identical rankings.
Government. It was conducted by the University Grants Committee
, a predecessor of the present Higher Education Funding Councils. The purpose of the exercise was to determine the allocation of funding to UK Universities at a time of tight budgetary restrictions. The committee received submissions of research statements from subject areas ("cost centres") within Universities, and issued quality rankings labelled "outstanding", "above average", "average" or "below average". The research funding allocated to Universities (called "quality-related" funding) depended on the quality ratings of the subject areas.
Two subsequent research assessments were conducted in 1989 and 1992 under the name "research selectivity exercise" by the Universities Funding Council
. These were followed by "research assessment exercises" conducted in 1996, 2001 and 2008 jointly by the various UK Higher Education Funding Councils.
The quality levels—based on assessment of research outputs, research environment and indicators of esteem—are defined as:
Each unit of assessment was given a quality profile - a five-column histogram - indicating the proportion of the research that meets each of four quality levels or is unclassified.
These ratings have been applied to "units of assessment", such as French or Chemistry, which often broadly equate to university departments. Various unofficial league tables have been created of university research capability by aggregating the results from units of assessment. Compiling league tables of universities based on the RAE is problematic, as volume and quality are both significant factors.
In response to criticism of earlier assessments, and developments in employment law, the 2008 RAE does more to take into account part-time workers or those new to a sufficient level of seniority to be included in the process.
Since 1996 the AUT
, now incorporated within the UCU
, has maintained a policy of opposition to the Research Assessment Exercise. In its view,
The official Review of Research Assessment, the 2003 "Roberts Report" commissioned by the UK funding bodies, recommended changes to research assessment, partly in response to such criticisms.
The House of Commons Science and Technology Select Committee considered the Roberts report, and took a more optimistic view, asserting that, "the RAE had had positive effects: it had stimulated universities into managing their research and had ensured that funds were targeted at areas of research excellence", it concluded that "there had been a marked improvement in universities' research performance". Nevertheless, it argued that "the RAE in its present form had had its day", and proposed a reformed RAE, largely based on Roberts' recommendations.
that after the 2008 exercise a system of metrics
would be developed in order to inform future allocations of QR funding. Following initial consultation with the higher education sector, it is thought that the Higher Education Funding Councils will introduce a metrics based system of assessment for subjects in science, technology, engineering and medicine. A process of peer review is likely to remain for mathematics, statistics, arts, humanities and social studies subjects.
HEFCE is working to develop the new arrangements, known as the Research Excellence Framework
(REF), which will be introduced after the 2008 RAE.
Higher education
Higher, post-secondary, tertiary, or third level education refers to the stage of learning that occurs at universities, academies, colleges, seminaries, and institutes of technology...
funding councils (HEFCE
Higher Education Funding Council for England
The Higher Education Funding Council for England is a non-departmental public body of the Department for Business, Innovation and Skills in the United Kingdom, which has been responsible for the distribution of funding to Universities and Colleges of Higher and Further Education in England since...
, SHEFC, HEFCW, DELNI
Department for Employment and Learning
The Department for Employment and Learning , formerly the Department of Higher and Further Education, Training and Employment , is a devolved Northern Ireland government department in the Northern Ireland Executive...
) to evaluate the quality of research undertaken by British higher education institutions. RAE submissions from each subject area (or unit of assessment) are given a rank by a subject specialist peer review panel. The rankings are used to inform the allocation of quality weighted research funding (QR) each higher education institution receives from their national funding council. Previous RAEs took place in 1986, 1989, 1992, 1996 and 2001. The most recent results were published in December 2008.
Various media have produced league tables of institutions and disciplines based on the 2008 RAE results. Different methodologies lead to similar but non-identical rankings.
History
The first exercise of assessing University research in the UK took place in 1986 under the Margaret ThatcherMargaret Thatcher
Margaret Hilda Thatcher, Baroness Thatcher, was Prime Minister of the United Kingdom from 1979 to 1990...
Government. It was conducted by the University Grants Committee
University Grants Committee
University Grants Committee may refer to:*University Grants Committee *University Grants Committee...
, a predecessor of the present Higher Education Funding Councils. The purpose of the exercise was to determine the allocation of funding to UK Universities at a time of tight budgetary restrictions. The committee received submissions of research statements from subject areas ("cost centres") within Universities, and issued quality rankings labelled "outstanding", "above average", "average" or "below average". The research funding allocated to Universities (called "quality-related" funding) depended on the quality ratings of the subject areas.
Two subsequent research assessments were conducted in 1989 and 1992 under the name "research selectivity exercise" by the Universities Funding Council
Universities Funding Council
The Universities Funding Council was a UK body established under the Education Reform Act 1988 replacing the University Grants Committee. It distributed funds provided by central government to universities for the provision of education and the undertaking of research...
. These were followed by "research assessment exercises" conducted in 1996, 2001 and 2008 jointly by the various UK Higher Education Funding Councils.
2008
The 2008 RAE used a four-point quality scale, and returned a profile, rather than a single aggregate quality score, for each unit.The quality levels—based on assessment of research outputs, research environment and indicators of esteem—are defined as:
| Rating | Description |
|---|---|
| 4* | Quality that is world-leading in terms of originality, significance and rigour |
| 3* | Quality that is internationally excellent in terms of originality, significance and rigour but which nonetheless falls short of the highest standards of excellence |
| 2* | Quality that is recognised internationally in terms of originality, significance and rigour |
| 1* | Quality that is recognised nationally in terms of originality, significance and rigour |
| Unclassified | Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment. |
Each unit of assessment was given a quality profile - a five-column histogram - indicating the proportion of the research that meets each of four quality levels or is unclassified.
Earlier Assessments
In 1992, 1996 and 2001, the following descriptions were used for each of the ratings.| 2001 & 1996 Rating | 1992 Rating | Description |
|---|---|---|
| 5* | 5* | Research quality that equates to attainable levels of international excellence in more than half of the research activity submitted and attainable levels of national excellence in the remainders. |
| 5 | 5 | Research quality that equates to attainable levels of international excellence in up to half of the research activity submitted and to attainable levels of national excellence in virtually all of the remainder. (Same definition) |
| 4 | 4 | Research quality that equates to attainable levels of national excellence in virtually all of the research activity submitted, showing some evidence of international excellence. (Same definition) |
| 3a | 3 | Research quality that equates to attainable levels of national excellence in over two-thirds of the research activity submitted, possibly showing evidence of international excellence. (Research quality that equates to attainable levels of national excellence in a majority of the sub-areas of activity, or to international level in some) |
| 3b | 3 | Research quality that equates to attainable levels of national excellence in more than half of the research activity submitted. (Research quality that equates to attainable levels of national excellence in a majority of the sub-areas of activity, or to international level in some) |
| 2 | 2 | Research quality that equates to attainable levels of national excellence in up to half of the research activity submitted. (Same definition) |
| 1 | 1 | Research quality that equates to attainable levels of national excellence in none, or virtually none, of the research activity submitted. (Same definition) |
These ratings have been applied to "units of assessment", such as French or Chemistry, which often broadly equate to university departments. Various unofficial league tables have been created of university research capability by aggregating the results from units of assessment. Compiling league tables of universities based on the RAE is problematic, as volume and quality are both significant factors.
Assessment process
The assessment process for the RAE focuses on quality of research outputs (which usually means papers published in academic journals and conference proceedings), research environment, and indicators of esteem. Each subject panel determines precise rules within general guidance. For RAE 2008, institutions are invited to submit four research outputs, published between January 2001 and December 2007, for each full-time member of staff selected for inclusion.In response to criticism of earlier assessments, and developments in employment law, the 2008 RAE does more to take into account part-time workers or those new to a sufficient level of seniority to be included in the process.
Criticism
The RAE has not been without its critics. Amongst the criticisms is the fact that it explicitly ignores the publications of most full-time researchers in the UK, on the grounds that they are employed on fixed term contracts. According to the RAE 2008 guidelines, most research assistants are "not eligible to be listed as research active staff". Publications by researchers on fixed term contracts are excluded from the Assessment Exercise unless those publications can be credited to a member of staff who is eligible for the RAE. This applies even if the member of staff being assessed only made a minor contribution to the article. Another issue is that it is doubtful whether panels of experts have the necessary expertise to evaluate the quality of research outputs, as experts perform much less well as soon as they are outside their particular area of specialisation.Since 1996 the AUT
Association of University Teachers
The Association of University Teachers was the trade union and professional association that represented academic and academic-related staff at pre-1992 universities in the United Kingdom...
, now incorporated within the UCU
University and College Union
The University and College Union is a British trade union formed by the merger in 2006 of the Association of University Teachers and the National Association of Teachers in Further and Higher Education ....
, has maintained a policy of opposition to the Research Assessment Exercise. In its view,
The official Review of Research Assessment, the 2003 "Roberts Report" commissioned by the UK funding bodies, recommended changes to research assessment, partly in response to such criticisms.
The House of Commons Science and Technology Select Committee considered the Roberts report, and took a more optimistic view, asserting that, "the RAE had had positive effects: it had stimulated universities into managing their research and had ensured that funds were targeted at areas of research excellence", it concluded that "there had been a marked improvement in universities' research performance". Nevertheless, it argued that "the RAE in its present form had had its day", and proposed a reformed RAE, largely based on Roberts' recommendations.
Planned changes to RAE system
It was announced in the 2006 BudgetBudget
A budget is a financial plan and a list of all planned expenses and revenues. It is a plan for saving, borrowing and spending. A budget is an important concept in microeconomics, which uses a budget line to illustrate the trade-offs between two or more goods...
that after the 2008 exercise a system of metrics
Metric (mathematics)
In mathematics, a metric or distance function is a function which defines a distance between elements of a set. A set with a metric is called a metric space. A metric induces a topology on a set but not all topologies can be generated by a metric...
would be developed in order to inform future allocations of QR funding. Following initial consultation with the higher education sector, it is thought that the Higher Education Funding Councils will introduce a metrics based system of assessment for subjects in science, technology, engineering and medicine. A process of peer review is likely to remain for mathematics, statistics, arts, humanities and social studies subjects.
HEFCE is working to develop the new arrangements, known as the Research Excellence Framework
Research Excellence Framework
The Research Excellence Framework is the successor to the Research Assessment Exercise, a method of assessing the research of British higher education institutions...
(REF), which will be introduced after the 2008 RAE.
External links
- Links to 2008 League Tables (The Independent)
- Links to 2008 League Tables (Research Fortnight)
- Links to 2008 League Tables (Guardian)
- 2001 RAE Institution Ranking (Guardian)
- Current official RAE website for 2008 exercise
- Archived RAE website for 2001 exercise
- Chanson, H. (2007). "Research Quality, Publications and Impact in Civil Engineering into the 21st Century. Publish or Perish, Commercial versus Open Access, Internet versus Libraries ?" Canadian Journal of Civil Engineering, NRC, Vol. 34, No. 8, pp. 946-951 (DOI:10.1169/L07-027) (ISSN 0315-1468).
- Chanson, H. (2008). "Some Experience with Digitial Publishing, Open Access Repository, Research Impact and Quality." Proc.Open Access Collection workshop, Australian Partnership for Sustainable Repositories, Brisbane, 14 Feb., 12 pages.