Effects of RAE/REF on Academic Life: Empirical Study
This study explores the impact of the RAE/REF on research performance, diversity, academic freedom, motivation, and more across different academic disciplines. With a qualitative approach, the research delves into the connections between the academic, scientific, and political spheres.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Performance Assessment in Science andAcademia II Effects of the RAE/REF on Academic Life.
Outline I 1. Introduction 2. Empirical procedure 2.1 Selection process 2.2 Questionnaire 3. Empirical results 3.1 The academic and scientific sphere 3.1.1 Research performance 3.1.1 Inequality 3.1.2 Diversity 3.1.3 Academic freedom 3.1.4 Recruitment policies 3.1.5 Relationship between teaching and research 3.1.6 Research motivation 3.1.7 Power
Outline II 3.2 Connections between the academic, scientific and political sphere 3.2.1 Research performance 3.2.1 Inequality 3.2.2 Diversity 3.2.3 Academic freedom 3.2.4 Recruitment policies 3.2.5 Relationship between teaching and research 3.2.6 Research motivation 3.2.7 Power 4. Summary 4.1 Academic and scientific sphere 4.2 Connections between the academic, scientific and political sphere 4.3 In the light of the state of research 5. Further research 6. Literature
1. Introduction Literature mentions many effects with regard to the impact of RAE/REF (see first seminar) To be able to discuss these effects in more detail I conducted a qualitative study It covered three different institutions and three different disciplines and included actors across the academic, scientific and political spheres
1. Introduction Nicholas Stern is actually running a review of the Research Excellence Framework (REF) and there is much debate in the academic, media and political sphere as to the effects the RAE/REF has on academic life (cf. Matthews 2015a, 2015b, 2016a, 2016b, 2016c, 2016d; Oancea 2016; Bentley 2016; Cuthbertson 2016; Locke 2016)
1. Introduction Hence, my research questions concern the RAE's effects a) on research performance; b) on inequality; c) on diversity; d) on academic freedom; e) on research and teaching; f) on recruitment policies; g) on motivation; and h) on power
2. Empirical procedure 2.1 Selection process Ethically confirmed by the Federal Ministry of Education and Research of Germany Random process of selection of scientists and academics (heads, professors, lecturers and readers) Selection by institutional rank (low, middle, high) and discipline (chemistry, sociology and history) Response rate of 10 percent
2. Empirical procedure 2.1 Selection process Interview time, date and location Pretest of questionnaire with scientists and academics from three different disciplines Testing understanding, accuracy and duplication of content of the questions Actual sample is N=23
2. Empirical procedure 2.1 Selection process Institution Discipline Total 1 2 3 1 3 3 3 9 2 2 3 1 6 3 2 3 3 8 Total 7 9 7 23
2. Empirical procedure 2.1 Selection process Interviews with central political actors in science and academia Policy advisors from a funding body, policy advisors from higher education agency, research experts in the field of the RAE/REF, leading representative of science to the government
2. Empirical procedure 2.1 Selection process Institution Position Number Funding Body Policy Advisor 5 Higher Education Agency Policy Analist 1 Leading researcher in the field of the RAE/REF Research experts 2 Scientist Leading advisor to the government 1 9
2. Empirical procedure 2.2 Analytic strategy and questionnaire First interpretation of the data according to notes taken during the interview Concentration on the similarities rather than the institutional and the disciplinary differences
2. Empirical procedure 2.2 Analytic strategy and questionnaire Academic and scientific communifty Historical changes Government Peer review and panel members University management Funding bodies Policy advisors
2. Empirical procedure 2.2 Analytic strategy and questionnaire Introduction with a general question concerning the interviewees' work experience General changes in the university landscape in the wake of the RAE/REF Effects of RAE/REF on a) research performance b) inequality c) diversity d) academic freedom e) research and teaching f) recruitment policies g) research motivation h) power
3.1 The scientific and academic sphere 3.1.1 Research performance British science and academia is a world leader in research in terms of publication and citation rates (01_01; 01_02; 01_03; 01_04; 01_05; 01_06; 02_01; 02_03; 02_04; 02_05; 02_06; 02_08; 02_09; 03_01; 03_02; 03_03; 03_04; 03_05; 03_06; 03_07) Numbers are increasingly crucial for decision-making processes and there is a belief in the increase of performance in science and academia
3.1 The scientific and academic sphere 3.1.1 Research performance Underlines the functional model of performance assessment in science and academia People are looking more and more at league tables, when they refer to the worldwide leading position of the UK s science and academia There is concern about the long-term perspective of the RAE/REF
3.1 The scientific and academic sphere 3.1.1 Research performance Academics and scientists said that it inhibits long- term speculative, blue sky and risky research due to its short termism (01_02; 01_03; 01_04; 02_05; 02_07; 02_09; 03_02; 03_03; 03_05; 03_06) Encourages the researchers to plan and organize their research (01_01; 02_2; 02_03; 02_04) 18 scientists and academics did not mention the effect at all
3.1 The scientific and academic sphere 3.1.2 Inequality Main funding goes to the leading, established universities (01_04; 01_05; 02_01; 03_02; 03_03). There is concern about the relative academic chances of women (02_02; 02_03; 02_05; 02_06; 02_07; 02_08)
3.1 The scientific and academic sphere 3.1.2 Inequality Women are more unlikely to be nominated for the REF cycle. Furthermore, women are likely to be assigned administrative tasks rather than activly engage in research to boost their research profile (03_03) Furthermore there are concerns about the chances of early career researchers who are overloaded with work and need REFable results for career progressions (01_03; 01_04; 01_06; 02_02; 02_03; 02_05; 02_06; 02_07; 02_08; 03_05; 03_06; 03_07)
3.1 The scientific and academic sphere 3.1.3 Diversity They did not change their research focus as a result of the RAE/REF (01_01; 01_02; 01_03; 01_04; 01_05; 01_06; 02_01; 02_02; 02_03; 02_04; 02_05; 02_06; 02_07; 02_08; 02_09; 03_01; 03_02; 03_04; 03_05; 03_06; 03_07) Interviewees identified conformity with the RAE/REF time cycle (01_04; 01_05; 02_06; 02_07; 02_08; 02_09; 03_01; 03_03; 03_04; 03_06; 03_07) Likely that RAE/REF does not enable them to undertake major shifts in their research focus
3.1 The scientific and academic sphere 3.1.3 Diversity This conformity could lead to a lesser likelihood of taking fundamentally new research paths and trying to find new directions in their research areas
3.1 The scientific and academic sphere 3.1.4 Academic freedom Interviewees replied that the RAE/REF had affected the choice of research themes (01_02; 02_01; 02_04; 02_08; 02_09; 03_01; 03_05; 03_06) Impact (01_02; 01_04; 02_02; 02_03; 02_05; 02_06; 02_08; 02_09; 02_09; 03_01; 03_02; 03_04; 03_06), time problems (02_01) and creating High Impact Journal research (01_04; 02_01; 02_02; 02_03; 02_04; 02_06; 02_08; 03_04; 03_05; 03_07)
3.1 The scientific and academic sphere 3.1.4 Academic freedom Concern about impact that prevents them from behaving critically towards government, NGOs and charities Contribution to academic community is inhibited (01_03; 02_05; 02_09; 03_01; 03_03; 03_06) Collaborations are encouraged (01_02; 01_04; 02_08; 03_01) Little or no effects on their teaching freedom (01_01; 01_02; 01_03; 01_06; 02_02; 02_03; 02_04; 02_05; 02_06; 02_07; 02_09; 03_01; 03_02; 03_03; 03_04)
3.1 The scientific and academic sphere 3.1.4 Academic freedom In relation to freedom of learning there are only little effects (01_02; 01_06; 02_01; 02_06; 02_08; 02_09; 03_02; 03_04) Some interviewees indicated that there are restrictions to student learning, time pressures to research and recruitment processes to fit in REFable criteria
3.1 The scientific and academic sphere 3.1.5 Recruitment policies Major effect Ever more strategies are used to increase and improve RAE/REF scores This creates an instrumental strategic mode to look at different rankings to further stimulate research collaborations at the lower ranked universities It is accompanied by historical shifts in the way the science and academia system develops There is a global trend for rankings and assessment of science and academia systems
3.1 The scientific and academic sphere 3.1.5 Recruitment policies Top researchers are recruited by means of REFable criteria to boost REF scores (01_02; 01_03; 01_04; 01_05; 02_02; 02_03; 02_04; 02_08; 02_09; 03_01; 03_02; 03_03; 03_05; 03_07) In recruitment more and more bibliometric tools are used to judge the quality of the new researcher Managerial and departmental peer review processes before and after the RAE/REF to improve research results (01_04; 01_06; 02_03; 02_04; 02_06; 02_09; 03_05; 03_07)
3.1 The scientific and academic sphere 3.1.6 Teaching and research Teaching is now more research led (01_01; 01_02; 01_03; 01_04; 02_02; 02_03; 02_08: 03_03; 03_06) Implies that the chance to discuss new topics which are unrelated to that research is more and more difficult to develop All restrictions coming up with the RAE/REF are focused more directly on the students' academic life
3.1 The scientific and academic sphere 3.1.7 Research motivation and pressure Some interviewees do not feel effected (01_03; 02_05; 02_08; 03_01; 03_04) For some it is more like an extra motivation (02_03; 02_04; 02_07) For others it feels depressing (01_01), pressuring (01_04; 03_07), leading to burn out (01_05), did not help much (02_01), creates stress (02_02; 03_06) and demotivates (03_03; 03_05)
3.1 The scientific and academic sphere 3.1.7 Research motivation and pressure It creates pressure on academics and scientists (01_01; 01_02; 01_04; 02_01; 02_02; 02_04; 02_05; 02_07; 02_09; 03_01; 03_03; 03_05; 03_06; 03_07) It is likely for the RAE/REF to increase pressure, which, in its turn, decreases motivation for quality Interviewees from a research intensive university indicated that pressure has a decreasing effect on their research motivation (01_01; 01_02; 01_04; 01_05) Discipline chemistry seems to be under high pressure (01_01; 01_02; 02_01; 02_04; 03_01; 03_05)
3.1 The scientific and academic sphere 3.1.8 Power Interviewees replied that they have evidence of power relationships influencing performance assessment in science and academia (01_01; 01_02; 01_06; 02_01; 02_02; 02_03; 02_05; 02_06; 02_08; 02_09; 03_01; 03_02; 03_04; 03_05; 03_06; 03_07) Power of important areas (01_01), competitive politics to lead a brand (01_02), managerialism, vice chancellor and academic senat (01_06), people favouring their own agenda (02_01)
3.1 The scientific and academic sphere 3.1.8 Power Top down power (02_03), management in the departments for better RAE/REF outcomes (02_05), who is on the panel, senior roles, gatekeepers to the REF and personal experience (02_06) Pressure which drives people (02_08), gender bias (02_09; 03_07), more prestigious universities (03_02), society and influencing mechanisms in panels (03_04; 03_05)
3.2 Connections between the academic, scientific and political sphere 3.2.1 Research performance Performance in the UK has increased as a result of the latest REF (04_02; 04_03; 04_05) Exerts pressure on academics to raise quality and it is a bureaucratic and costly procedure (04_01; 04_02; 04_05) Increased managerialism at the universities (04_01; 04_02; 04_03; 04_04) Therefore, the effects are due to increased management of RAE/REF rather than displaying a direct effect of the RAE/REF procedure
3.2 Connections between the academic, scientific and political sphere 3.2.1 Research performance Selective system of resource distribution Leads to a concentration of funding (04_01; 04_02; 04_04) The drive for bibliometrics comes from the government (04_01; 04_02) and the latter also stimulated the introduction of impact (04_01) The major problems are the decreasing level of GDP and the need to maintain the resource level for universities (04_01; 04_02; 04_03; 04_05;)
3.2 Connections between the academic, scientific and political sphere 3.2.1 Research performance An incentive to produce high quality research by peers (04_01; 04_02; 04_03; 04_04; 04_05) It is a more legitimate activity There are knock on effects, more micromanagement, more support and more strategic behaviour (04_06) It encourages a short term approach and a drive towards the norm of mean panel requirements A reduction and control of quality becomes Money is a driving force for decisions (04_07)
3.2 Connections between the academic, scientific and political sphere 3.2.2 Inequality Establish gender imbalances in the staff selected Codes of practice and the possibility to reduce the amount of publications to submit to the RAE/REF (04_01) It is possible to reduce the number of publications under special circumstances (period of maternity leave, level of life events, illness and disability) (04_01; 04_03; 04_04)
3.2 Connections between the academic, scientific and political sphere 3.2.3 Diversity Short term work is encouraged rather than longterm risky research (04_03) Interdisciplinary research is discouraged (04_04) An effect on multidisciplinary research and humanities research
3.2 Connections between the academic, scientific and political sphere 3.2.4 Academic freedom Interdisciplinary research is discouraged (04_04) and impact favours topics (04_01; 04_04) The discipline structure is likely to inhibit interdisciplinary research (04_02) But interdisciplinary research scored as high as other research
3.2 Connections between the academic, scientific and political sphere 3.2.4 Academic freedom Citations are not used as criteria in the REF process (04_05) There is conservatism in the sector which might prevend some output types and areas It is more symptomatic It is a highly managed context (04_07)
3.2 Connections between the academic, scientific and political sphere 3.2.5 Recruitment policies Major effect It creates a transfer market (04_01; 04_02; 04_03) A rise in staff movement can be observed by the census date (04_03) There is continued poaching and enticement of research stars (04_02) It discourages international researchers (04_02)
3.2 Connections between the academic, scientific and political sphere 3.2.5 Recruitment policies Fixed dates and timing of recruitment are relevant (04_04) Early career researchers are struggling in the assessment cycle (04_04) It enables greater mobility of researchers, encourages to hire the best people (04_05)
3.2 Connections between the academic, scientific and political sphere 3.2.6 Relationship between teaching and research Emphasis on research rather than teaching, but this will change with the introduction of the Teaching Excellence Framework (TEF) (04_02; 04_05)
3.2 Connections between the academic, scientific and political sphere 3.2.7 Research motivation One interviewee pointed out that the RAE/REF does not have an effect on this Rather it is the way the management and performance assessment is implemented in universities (04_01) You have to differentiate between the people who are submitted to the RAE/REF and those who were not selected Confirmation of success and being highly employable vs. experiencing of a loss of morale (04_02)
3.2 Connections between the academic, scientific and political sphere 3.2.8 Power Policy advisors see wide evidence for this situation (04_01; 04_02; 04_03; 04_04) There is a growing need to justify public investments in the spending review and to provide evidence for the treasury (04_01) A further vehicle of power structures is the peer review The people know each other very well There are good and bad relationships between the assessors and the assessed people
3.2 Connections between the academic, scientific and political sphere 3.2.8 Power Use their power to achieve better outcomes in the RAE/REF (04_02) Institution is seen as a driving power (04_03), on the individual, the institutional, the panel and the researcher level (04_04), government (04_04), lobby groups, societies as strong voice, scientific communities and academic unions (04_01) Evidence of management power, clique of experts hegemonial definitions and hierarchies of esteem in the disciplines (04_07)
4. Summary 4.1 Academic and scientifc sphere Britain is a worldwide leading country in science and academia It is not yet clear whether this is attributable to the RAE/REF Scientists, academics believe in the performance increase of British science and academia in terms of publication and citation rates
4. Summary 4.1 Academic and scientifc sphere The RAE/REF and its interpretation in universities by management, departments and staff leads to unequal treatment in early career developments, progression and promotion and promotes gender inequality
4. Summary 4.1 Academic and scientific sphere RAE/REF is likely to inhibit diversity in research topics due to scientists and academics conforming to the RAE/REF timecycle RAE/REF does not enable the scientists and academics to change their research tracks fundamentally It does not force academics and scientists to take up research on specific areas only
