Gender and the Research Excellence Framework

In: EqualBITE
Open Access

If the inline PDF is not rendering correctly, you can download the PDF file here.

Outcomes of research evaluation are arguably playing an ongoing and increasingly important role in academic careers and success, but there are several factors that hold the potential to militate against fairness, gender equality and equality of opportunity (Yarrow, 2016). This article discusses my recent PhD research into women’s lived experiences of research evaluation in a UK Russell Group university and explores some of the factors that may affect submissions to the Research Excellence Framework (REF). The last run of the REF occurred in 2014 with results being published in December 2015; 154 UK institutions submitted to the exercise. Currently there is a national requirement at the institutional level for a combination of research outputs (65%), the impact of research (20%), and the institutional research environment (15%), though at the individual level, each university currently decides which academics’ work will be included or excluded. However, the inclusion criteria are under review for the 2020 REF after the publication of the Stern Review (Stern, 2016).

The Stern Review is also significant in that currently recommendations for the next REF are being made, as well as the Stern Review strategically informing not only how the REF should be adapted, but also implemented at the national and institutional level in the future. However, I argue that the recommendations made in terms of equality and diversity issues, are not clear or tangible and that there is a clear need for further investigation into the equality and diversity issues surrounding research evaluation. Conversely, it is noteworthy to consider the potentially very significant game-changer which is proposed (Stern, 2016), to include all research-active staff in future exercises and allocate them to a UOA (unit of assessment). Although this may appear to remedy career issues associated with non-inclusion, it is currently not clear whether this recommendation will be recognised and subsequently implemented, and so there is an ongoing need to better understand the current method of selective inclusion.

It is important to demonstrate that whilst the exercise is designed to evaluate research outputs and not individual academics per se, based on my findings I argue that the two are inextricably linked. It is further of note that I found high levels of anxiety surrounding individual submissions that have ramifications not only for individual academics’ identities, but also stress and academic well-being (Yarrow, 2016).

My study focused on female academics’ lived experiences and career aspirations in the context of the 2014 REF, as well as the views of research directors and heads of school in the study entitled National Research Evaluation and its effects on female academics’ careers in the UK – A Case Study (Yarrow, 2016). The sample in the case study drew on 80 semi-structured, life-history inspired interviews with academics across humanities and social sciences in an anonymous UK university, made up of 65 female academics and 15 key respondents. The sample covered a broad age range, as well as a range of female academics spanning ECRs (early career researchers) to professors, as well as career academics and individuals who had been in practice prior to pursuing an academic career. It is important to make clear that my research was not conducted at the University of Edinburgh, but another Russell Group University, though a number of the findings may be generalised across UK higher education institutions that submit to the REF.

This article will cast light on why women are not only less likely to be submitted to research evaluation exercises in the UK, but also the role that informal networks and unconscious bias may play in some academic careers.

Why are women still less likely to be submitted to the REF?

It is clear that there is a stark disconnect between women’s ongoing under-representation in leadership positions, for example, and the increasing representation of female students (Grove, 2012). Women in the UK continue to be under-represented not only in leadership positions in industry and academia, but in the upper echelons of academia (HESA, 2015; Fletcher, 2007), particularly in the professoriate, where only around 23% of professors are female (HESA, 2015). Conversely, women are over-represented in temporary, or part-time academic positions (ECU, 2015).

It is of note that women at the national level are still less likely to be submitted to REF than their male colleagues (HEFCE, 2015), despite gender equality initiatives such as Athena SWAN, and the Equality Challenge Unit’s equality charter mark scheme. In the last REF in 2014, nationally around 67% of men were entered compared to around 51% of women. However, the HEFCE report also found UOA differences in that:

In Education, 62 per cent of the eligible staff were female and the average percentage of staff selected was 21 per cent; whereas in Electrical and Electronic Engineering, Metallurgy and Materials, where only 12 per cent of the eligible staff were female, the average proportion of staff selected was 65 per cent. (HEFCE, 2015, p. 9, point 51)

This therefore demonstrated that there are indeed differences between disciplines, but that these tend to be in areas where women are proportionately well represented, however that overall women are still less likely to be submitted to the REF.

More male scholars are included in programmes of national research evaluation than their female counterparts in the UK, but for what reasons?

It is clear that women who reach full professorship are in the minority compared to their male counterparts; it is argued here that outcomes of research evaluation play an increasingly important role in this. Contemporary female careers in the academy are argued by Van den Brink & Benschop to be still to an extent marred and constrained by practices of inequality that have no relation to merit (2011, p. 518), it is clear from the study that there are gendered perceptions of merit which appear to affect women’s lived experiences not only of research evaluation, but of academic careers more broadly (Yarrow, 2016).

Findings from my research suggest that outcomes of research evaluation, and whether an individual is included in the REF or not, may play a role in continuing vertical gender segregation in humanities and social sciences, because of the increasing importance of REF inclusion, also often referred to colloquially as ‘REF-ability’ and the associated career leverage gains in an increasingly ‘marketised’ academy, characterised by ever-increasing competition and corporatisation (Rogers et al., 2014; Deem, 1998; Willmott, 1995; 2003).

The long-term impact of maternity leave and time taken out on academic careers, particularly during an REF cycle, is an ongoing issue. This appears to be a factor that is damaging to women’s career development, primarily because the REF is a time-oriented mechanism (Yarrow, 2016). Whilst periods of maternity leave are accounted for in terms of a reduction of one paper per period of maternity leave, the longer-term caring responsibilities still hold the potential to detract from women’s longer-term career development, which becomes further problematic and typified when it is also considered that women are still engaged disproportionately in domestic labour in the home. Furthermore, women in academia take shorter periods of maternity leave than many other professions, which may be indicative of an awareness of the potential implications of taking maternity leave, and the associated anxiety (Bawden, 2014).

The importance of informal networks

Informal networks play an integral, yet somewhat indirect, role in submissions to, and management of the REF. The importance of informal networks was clear throughout the study, often expressed in terms of learning ‘the rules of the game’ as well as finding out about opportunities for promotion or development and having access to decision-makers, and having them ‘on-side’ which also plays a role in women’s submissions to the REF, particularly as it is often the head of school or research director who make the final decisions on inclusion or exclusion (Yarrow, 2016). It is also noteworthy that the head of school role is often disproportionately occupied by male academics, and this was so in the anonymous university in my case study.

Informal networks have also been found to play an integral role in the recruitment and selection of REF panel members who evaluated submissions, a finding that echoes earlier findings in the REF analysis of panel membership (REF, 2011). It is of note that the 2014 REF panels were disproportionately male, and that this in itself may import issues of gender bias, particularly when the reliance on informal networks for the recruitment and selection of panel members is considered. Informality is the invisible hand that holds the potential to affect equality and diversity, and particularly women’s experiences of research evaluation in the UK today, yet it is a factor largely ignored by the recent Stern Review (Stern, 2016).

The pervasive role of unconscious bias

A growing body of literature surrounds the notion of unconscious bias specifically in higher education. Unconscious bias may be defined as “the associations that we hold which, despite being outside our conscious awareness, can have a significant influence on our attitudes and behaviour” (ECU, 2013, p. 1). Implicit bias may be defined as “when we have attitudes towards people or associate stereotypes with them without our conscious knowledge. A fairly commonplace example of this is seen in studies that show that white people will frequently associate criminality with black people without even realising they’re doing it” (Perception Institute, 2016).

The tensions between unconscious and implicit bias are being increasingly questioned in academia, though this is still an area which requires further awareness and training in order to better understand its potential implications. The main differences and tensions between unconscious bias and implicit bias, though the terms are often used interchangeably in the current discourse, centre on the notion that individuals are unaware of their biases (unconscious), but that increasingly, bodies such as the Equality Challenge Unit argue that the notion of implicit bias must be questioned as to how unconscious it may actually be, as individuals are being made more actively aware of biases and stereotypes for example through equality and diversity training and that: “Once we know that biases are not always explicit, we are responsible for them” (ECU, 2013, p. 1).

The importance of the potential role of unconscious bias in higher education has been made clear by the Equality Challenge Unit in that they acknowledge that “bias is likely to be relevant to many areas of an institution’s work, for example appraisals and grievances, Research Excellence Framework submissions, student admissions and course evaluations” (ECU, 2013, p. 4), thereby demonstrating the wide range of aspects of academic life that unconscious bias may affect.

Moreover, it is important to note that there appears to be a disparity between institutional promises and policies surrounding equality and diversity in general, with the ECU highlighting that institutional strategies and promises are simply not equal to an institution actively practising its commitment to equality and diversity (ECU, 2013, p. 4). It is argued that in the gaps between organisational practice, the equality and diversity discourse and some universities’ policies, opportunities for inequalities persist through the presence of unconscious bias, as well as informal subversion of policy and practice. It is these grey areas or lacunae, which appear to actively contribute to gender inequality in academia. The policies and processes are indeed in place, partly due to compliance with legislation such as the Equality Act, 2010, but in some instances these serve to be merely tick-box exercises, and in reality decisions are made quickly and informally, but may still be portrayed to be in line with organisational protocol.

The notion of unconscious or implicit bias is currently being increasingly explored theoretically, and the recent research of Milkman et al. whilst focusing on doctoral students applying to universities in the US, demonstrates that multiple decisions are made before formal entry into organisations (Milkman et al., 2015). This also contributes to the notion that unconscious bias is notoriously difficult to identify, measure, and correct.

However, unconscious bias is an issue which appears to be encompassed in several aspects of the processes, contributing to REF submissions, peer review processes in journals and the readers of materials for REF submissions in some institutions. This becomes increasingly problematic when the issue of the lack of clarity surrounding the recruitment and selection of REF panel members is deliberated, and that journal editorial boards are still dominated by (often well-networked) men (Özbilgin, 2009). Additionally, the recruitment and selection of panel members, as well as a lack of female and BME (black and minority ethnic) panel members, has again been identified. Although there is an acknowledgement from REF that there are issues with REF panels (REF, 2011), there is little indication as to what the concrete actions may be to remedy this in order to improve the current situation.

Van den Brink & Benschop further find that:

Academic excellence cannot be treated as an objective and measurable attribute, but that it is a social construction that is always embedded within a social context and is thus subject to multiple cultural and political influences. (Van den Brink & Benschop, 2011, p. 50)

With specific regard to unconscious bias in higher education, the main research focuses on its role in recruitment and selection, and decision-making. However, Roos (2008) argues that workplace interactions are permeated with gendered and institutionalised status beliefs, as well as organisational policies and decision-making processes that are also marred by institutional gender stereotypes (2008, p. 186), thus demonstrating the linkages between institutional policies, decision-making and how this may interact with gender inequality.

Of further note is the work of Easterly & Ricard (2011) who reviewed women’s departure from academia, and although based on a US STEM context, valuable insight can be drawn regarding the role that unconscious bias plays in women leaving academia. One of their main assertions is that:

Not only do these lenses shape how people perceive, conceive, and discuss social reality, but because they are embedded in social institutions, they also shape the more material things – like unequal pay and inadequate day care – that constitute social reality itself. (Easterly & Ricard, 2011, p. 62)

This demonstrates again the role that not only unconscious bias, but gender assumptions can play in the construction of social reality for women in higher education organisations. Whilst the effects are evident through the under-representation of women in the upper echelons of the academy, but how change can be sought and implemented, and unconscious bias tackled, remain to an extent theoretical. However, the following section outlines some of the recommendations from participants in the study, which may be helpful or insightful for others, as well as providing some insight into what may be anecdotal, but constructive advice to departments to avoid potential unfairness.

An integral aspect where the role of unconscious bias was pertinent was with regard to the recruitment and selection of REF panel members, internal university readers of outputs for submission, and peer review. There is a body of literature which suggests that unconscious gendered bias plays a role in the conceptualisation of excellence and suitability for certain types of work. See, for example, Leslie et al. (2015).

Whilst focusing on the gender biases of faculty favouring male students in the sciences in the US, Moss-Racusin et al. (2012) found that female students were less likely to be hired because they were perceived (notably by both male and female faculty members) to be less competent, and that ultimately this may well undermine academic meritocracy. Budden et al. (2008), found that even in processes of double-blind peer review, this is often not adhered to in practice; when double-blind review is used, where neither the author’s nor the reviewer’s identity is known, the number of women who are subsequently published increases. This has linkages to my PhD research upon which this article draws, in that it may be argued that where panels are disproportionately male with an under-representation of female academics, the scope for the impact of unconscious as well as implicit bias may potentially be increased (Yarrow, 2016). This holds the potential to further contribute to gender bias in research evaluation processes, thereby being further potentially damaging to the careers of female academics. Although these linkages may appear tenuous on the surface, my empirical findings suggest the contrary – women’s lived experiences in the academy still appear to be shaped, in some instances, by the permeation of unconscious bias into university practices, such as recruitment and selection, and inclusion in research evaluation, both of which play an important role in academic careers.

Helpful pointers from participants

  • The following section outlines some of the pointers from the PhD study findings that participants gave as to what has helped them in their careers, and might be useful for others:
  • Building up a positive relationship with the head of school.
  • Learning to say ‘no’.
  • Building up your own network and alliances within your university and further afield.
  • Not letting others take credit for your work such as by being first author on a paper.
  • Having a senior mentor who is also willing to introduce you to their contacts.

Department-level advice to help avoid unfairness

My findings indicated that departmental culture and leadership within a department plays an important role in women’s experiences of REF. The following points are some of the strategies that some key respondents in my study outlined as being useful to help avoid unfairness:

  • Have open meetings to discuss the workload model if there is one in place, and hold the meeting in hours when people with children and caring responsibilities can attend.
  • Actively engage with and know about research from the Equality Challenge Unit and the University and College Union; there may be issues at play that you’re simply not aware of.
  • Reassure academics who are not included that this will not affect their internal promotions.
  • Discuss the myths around REF and REF submissions that may be present in the department and actively involve everyone in better understanding and then dispelling them.
  • Ongoing emotional support, reassurance and kindness.


Women are still significantly under-represented in the upper echelons of the academy nationally and research evaluation evidently plays a role in this. The REF may hold the potential to provide positive opportunity for change, as well as equality of opportunity through a more transparent framework that makes use of gender-balanced panels, for example, though currently it appears that women are still at a disadvantage, particularly in the context of increasing individualisation of academic work and in an academy where gentlemen’s agreements still hold weight and networks play an extremely important role in many aspects of academic life.

If the inline PDF is not rendering correctly, you can download the PDF file here.