All posts by Jake Anders

Associate Professor of Educational and Social Statistics at UCL Institute of Education, University College London.

Young people’s physical health during the COVID-19 pandemic

Jake Anders is Principal Investigator of the COVID Social Mobility & Opportunities study (COSMO), and an Associate Professor and Deputy Director at UCL Centre for Education Policy & Equalising Opportunities.

Although young people were among those least likely to be directly affected by severe effects of COVID-19, they were not immune from its immediate effects on health. We are better able to understand implications of this using data from the COVID Social Mobility & Opportunities study (COSMO). The study includes a representative sample of over 13,000 young people across England, who were aged 14–15 at the onset of the pandemic, and 16–17 during the academic year 2020/21 when our first data were collected.

COSMO’s purpose is also wider than the direct health impacts of COVID-19. As such, this blog post — drawing on our latest COSMO briefing published today — also takes a wider look at young people’s health behaviours during this period.

Young people’s experiences of COVID-19

When first taking part in COSMO (between October 2021 and March 2022) almost half of the cohort reported having had COVID-19 (of whom 28% had had this confirmed by a test, likely with many having been affected before widespread testing was available).

Many of these young people will have experienced relative mild — or at least, transient — symptoms. But this was not the case for all. 1 in 5 of young people who reported having had COVID-19 (just under 10% of the cohort as a whole) reported that they continued to experience symptoms more than 4 weeks after first catching the virus: the accepted definition of Long COVID. Among this group, a quarter described their case of Long COVID as limiting their ability to carry out daily activities a lot.

Figure 1: Proportion reporting a case of severe long COVID by deprivation quintile group

Furthermore, there were inequalities in young people’s chances of experiencing long COVID. Among those who reported having caught COVID-19, those from less socio-economically advantaged backgrounds were more likely to report that this had persisted into a case of long COVID. This is despite no difference in reporting having contracted COVID-19 between these groups. What, then, explains differences in this persistence? Reasons for these inequalities are rather uncertain. One hypothesis is that getting sufficient rest is important for COVID-19 recovery, and this may be easier to achieve for some groups than others. Another is that pre-existing differences in overall health levels associated with socio-economic status pre-infection explain risk of prolonged disease.

Educational consequences of physical health during the pandemic

Young people who reported having experienced severe Long COVID received lower teacher-assessed GCSEs than their peers who had never had COVID-19. Given the differences discussed above between young people who experienced severe Long COVID, we wanted to ensure that this was not simply an artefact of these differences. After adjusting statistically for differences in demographics, socio-economic status, and prior attainment, it was still the case that young people who had experienced severe long COVID performed worse than comparable peers. The size of this difference is very roughly equivalent to two months of learning.

Figure 2: Differences in GCSE teacher assessed grades by shielding status and severe long COVID

Notes: Differences reported have been standardised. Severe long COVID analysis is compared to those who reported not having had COVID at all.

Those who were asked to shield during the pandemic — because pre-existing medical conditions meant that they were at the highest risk of severe illness if they caught COVID-19 — also received lower grades in their teacher assessed GCSEs than their peers. 8% of the COSMO cohort reported being asked to shield and — after adjusting for demographics, socio-economic status, and prior attainment — had lower GCSE grades very roughly equivalent to four months of learning.

Young people’s wider health

COSMO also allows us to shine a light more broadly on the health of this generation of young people.

Continuing a trend seen across successive cohorts, this cohort of 16–17 year olds are less likely to drink alcohol than those who came before them. 63% say they have ever done so, compared to 85% of a comparable group of 16–17 year olds in 2007. Across various measures, pupils from advantaged backgrounds (such as those attending private schools, or in state schools with more advantaged intakes) are more likely to report ever having drunk alcohol, and reported drinking more frequently.

This group of young people are more likely to have ever used an e-cigarette (33%) than to have ever smoked a cigarette (23%). However, regular e-cigarette use is much lower than this might imply. Just 6% report using e-cigarettes once a day or more. In contrast to alcohol use, those from less advantaged backgrounds are more likely to use e-cigarettes than their more advantaged peers.

Just under 1 in 6 of this cohort of young people report having ever tried any illegal drugs. Among those who have, almost all had tried cannabis. Just 4% of the sample reported having tried any other illegal substance. And similar to the patterning seen with alcohol, those in more advantaged circumstances were more likely to have taken an illegal drug. Those identifying as White or Mixed ethnic background are far more likely to report having tried illegal drugs than any other ethnic groups.

Conclusions

With attention on issues around COVID-19 fading, we should not forget its continuing effect on those for whom it was a particularly debilitating illness to experience. Some continue to suffer from long COVID. Even those who have recovered have seen implications for their wider lives and life chances, such as lower academic attainment scores for those who experienced severe long COVID. These impacts also seem to have reinforced existing health and socioeconomic inequalities.

COSMO also helps us shine a wider light on the experiences of this cohort of young people, and quite how much this has changed compared to earlier generations. Substantially reduced drinking of alcohol compared to earlier generations, low rates of drug use, and the emergence of e-cigarette use all raise questions about implications for future health, too.

There is more detail on these findings, as well as a wider range of health impacts and behaviour of the COSMO cohort, in our latest briefing note. You can also find our earlier briefing notes on the study website.

Advertisement

How has Covid-19 affected inequalities between state and private schools?

Covid-19 caused significant and widespread disruption to young people’s education. But the effects were not the same for everyone. Many pre-existing inequalities have been exacerbated, including those between state schools and independent schools.

The pandemic has had a major effect on the education and wider lives of children and young people, ranging from access to early years provision, learning losses among school age pupils, and broader effects on mental health and wellbeing (Farmer, 2020). But the impact has not been the same for everyone.

It has become clear that school closures and online learning due to the pandemic have exacerbated existing inequalities in education. Those from disadvantaged backgrounds and pupils with additional needs (including special educational needs and disabilities) have been especially badly affected by the disruption. In this article, we focus particularly on inequalities that emerged through differences in the responses of state and private schools.

Less than 10% of young people in England attend a private school during their education. Nevertheless, these institutions have an outsized influence on the persistence of educational advantage. They are highly socially selective, with the vast majority of those attending private schools coming from families in the top 10% of the income distribution (Anders et al., 2020; Henseke et al., 2021).

Further, the gap in resources between state and private schools is large (Green et al, 2017). It has grown larger over the past decade: from £3,100 per pupil per year in 2010/11 to £6,500 by 2020/21 (Sibieta, 2021).

Based on a combination of these factors, people who have attended private schools are disproportionately likely to be employed in high-status jobs later in life (Sullivan et al., 2016). Privately educated graduates are a third more likely to enter a high-status occupation than state-educated peers from similar families and neighbourhoods (Macmillan et al., 2015).

Early on in the pandemic, there were many anecdotes about differences in school responses, emphasising some schools’ rapid move to live online lessons. But much of the data available to test these narratives only covered state schools (a notable exception is Andrew et al., 2020).

Things have changed with new data from the COVID Social Mobility & Opportunities study (COSMO). This cohort study of young people in Year 10 at the pandemic’s onset provides high quality new data to explore the issue. Here, we consider some of the differences that emerged between state and private schools due to the large resource differences and why they matter.

Schooling in lockdown

When the first national lockdown began in the spring of 2020, schools played a vital role in responding to myriad challenges facing their pupils. With concerns about welfare and wellbeing, it was understandable that tasks such as free school meal voucher distribution, checking on how pupils’ families were coping in terms of mental health, welfare and food, and providing information on where they could obtain support distracted from efforts to organise online learning for pupils. This was particularly acute for schools with many pupils from disadvantaged backgrounds (Moss et al., 2020).

With more resources and fewer issues to solve, private schools were able to move quickly to set up alternatives to in-person classes. For example, 94% of private school participants who responded to the COSMO study received live online lessons during the first national lockdown.

This was 30 percentage points higher than the rate reported among those in comprehensive schools. The vast majority (84%) also reported receiving more than three online classes per school day during this period, compared with 41% in state grammar schools and 33% in state comprehensive schools.

Figure 1: Provision of live online lessons, by school characteristics, lockdown one (March-June 2020) and lockdown three (January-March 2021)

Source: COVID Social Mobility & Opportunities study (COSMO).
Notes: N=12,505, including 11,317 in state comprehensive group; analysis weighted to account for study design and young person non-response.

There were not national school closures during the second lockdown (in the autumn of 2020). By the third national lockdown in early 2021, state schools with the most advantaged intakes had largely caught up with private schools’ remote learning provision.

Pupils in these schools were now similarly likely to be offered remote lessons (around 95% in state grammar and state comprehensive schools with the least disadvantaged intakes). But they were still not providing as many per day, with 82% of those in state grammars and 71% in the least deprived state comprehensives reporting receiving more than three online classes per day, compared with 93% in private schools.

On the other hand, state schools with more disadvantaged intakes had made less progress on both fronts – 80% of pupils in these schools reported receiving any online lessons and 53% more than three per day – remaining behind in terms of online lesson provision.

Pupils at independent schools were also more likely to have regular contact with a teacher outside class during lockdowns. Here, state schools caught up slightly in the third lockdown.

But state comprehensive pupils were more than twice as likely to report difficulties in accessing support from teachers than pupils at private schools (17% compared with 8% in the third lockdown).

Pupils at private schools also reported spending more time on schoolwork during lockdowns than their state school peers. This added up to more than one extra working day per week of time spent on schoolwork in both the first and third lockdowns.

Many of these differences are not down to the schools directly. Rather, they are due to challenges faced depending on pupils’ circumstances.

For example, almost a quarter of pupils attending state comprehensives with the most deprived intakes did not have access to a suitable device for joining remote classes in the first lockdown. This figure was under 2% for pupils at independent schools.

Similarly, 13% of those at state comprehensive schools reported issues due to having to share devices needed to take part in online learning. Only 4% at independent schools faced the same issue.

Catch up

Given these inequalities, it is unsurprising that young people at state schools were more likely to think that their progress has suffered because of the Covid-19 disruption. In state schools, 81% of pupils reported this, compared with 72% at private schools. To address this, a much larger effort to support young people in state schools to catch up is needed.

This is especially important as the gap in disruption to learning did not stop with the resumption of in-person schooling. For example, teacher absence has taken a larger toll on state schools in the post-restrictions period of the pandemic.

Staff absence rates due to Covid-19 infections in state schools have been higher than private schools. In a snapshot survey in early 2022, 20% of state school teachers reported that at least one in ten staff in their school were absent. Only 12% of private school staff reported the same (Sutton Trust, 2022).

What have we learned about differences in catch-up activities between state and private schools? Evidence from the COSMO study finds that 53% of young people took part in at least one type of catch-up activity.

This figure was 54% for young people in state comprehensive schools, and slightly lower at 51% for those in independent schools. So, there is some evidence of differential catch up weighted towards the state sector, but it is not large. The definition of catch-up activities is wide, from extra online classes to one-to-one tuition. The latter of these is most likely to have had significant catch-up benefits (Burgess, 2020).

Focusing on the offer of tutoring, this was more likely for young people at independent schools (52% compared with 41% in state comprehensive schools). This is despite the government’s flagship National Tutoring Programme (NTP).

The NTP was meant to target this sort of provision to pupils who needed it most. Unfortunately, its reach was narrower than hoped, with an official evaluation report highlighting that ‘only around a half of school leads and staff felt that all or most of the pupils selected were disadvantaged’ (Lord et al., 2022).

That said, pupils in independent schools were less likely to have taken part in tutoring than those in state schools. Among independent school pupils, 23% said they had taken up this offer, compared with 27% in comprehensive schools. This seems to reflect pupils in private schools being less likely to feel that they needed this help, even if it was being offered. An equally important aspect of pupils’ ability to re-engage fully in their education is their mental health. Pupils attending private schools were more than twice as likely to report that their school mental health support was very good (26%) than state school pupils (10%) (Holt-White et al., 2022)

Figure 2: Whether participant agreed they had caught up with lost learning during the pandemic, by gender and school type

Source: COSMO Notes: N=12,149; analysis is weighted for survey design and young person non-response.

Despite catch-up efforts, pupils at state comprehensive schools were the least likely to think they had been able to catch up (34%). This compared with 58% of pupils at independent schools who felt this way. Almost half (46%) of pupils at comprehensive schools said they had not been able to catch up, while just over a quarter (27%) at independent schools felt the same.

Assessment during Covid-19

Because of the disruption to schooling, the government decided to replace GCSE and A-level exams with teacher assessed grades in 2020 (and centre assessed grades in 2021). Some adaptation was unavoidable, given the Covid-19 restrictions. But many highlighted the importance of retaining externally set and marked assessment (Anders et al., 2021).

Indeed, there is evidence of biases in teacher assessed grades that affect ethnic minority groups, those from lower income backgrounds, boys and those with special educational needs (Burgess and Greaves, 2013; Campbell, 2015). Top grades awarded in the two years in which teacher or centre assessed grades were used jumped up compared with the years before.

But the increase was most prevalent in private schools (Anders and Macmillan, 2022). The proportion of A grades awarded at A-level rose from 44.7% in 2019 to 70.4% in private schools in 2021 (compared with a rise from 24.1% to 42.1% in state academy schools).

Some of this could have been due to the differences in provision highlighted above. Yet, the particularly dramatic fall in grades for private schools in 2022 – a cohort that had more of their education disrupted – casts doubt on this.

We can also use findings from the COSMO study to understand differences in assessment practices between the sectors. Pupils in this cohort received teacher assessed grades for their GCSEs in 2021.

Private school pupils were more likely to say that their teacher assessed grades were better than expected than state school pupils. Likewise, they were much less likely to report that they did worse than expected. As with the changes in grades over time, this suggests a more generous approach to teacher assessed grades in private schools.

Whatever the exact mechanisms, these results represent a significant widening of the qualifications gap between state and private school pupils. This will have had – and will continue to have – significant implications for pupils’ future educational and employment trajectories.

Conclusion

Taken together, we can see that there are a range of ways in which disruption as a result of the pandemic has laid the groundwork for increased inequalities between pupils who attended state and private schools throughout their lives.

Existing gaps have been reinforced by differences in schools’ capacity to implement quick and effective responses at the outbreak of the pandemic.

It seems unlikely that these inequalities have been fully addressed with the fairly small differences in catch-up offerings – pupils themselves reinforce this view. And the use of teacher assessment in the place of usual exams appears to have baked in the results.

We will not know the full extent of the implications of this disruption for years to come. The COSMO study is designed to try to help us to do exactly that.

Some of the differences may unwind. For example, if some pupils’ grades were flattered more by teacher assessment than others, we would not expect this in itself to result in long-term productivity gains and hence to affect individuals’ wages (Wyness et al., 2021). Nevertheless, the knock-on effects (for example, access to university) may still reinforce the educational gap.

Ultimately, the early signs are not promising for inequalities in life chances between state and privately educated pupils as a result of the pandemic. We must not forget about the young people who have been disproportionately affected during this time.

As the pandemic subsides, it is easy to focus on ‘getting back to normal’, but in reality, extraordinary efforts are still needed to try to address these inequalities.

While important in itself, ensuring everyone is given the opportunity to achieve their potential in life is vital to the country’s productivity and, hence, everyone’s future economic prosperity.

This article was first published on the Economics Observatory on 5 December 2022.

We must continue to address the unequal impacts of the pandemic

COSMO’s Principal Investigator, Jake Anders, outlines how COSMO will help us tackle the unequal effects of COVID-19.

As colleges and schools have returned to some kind of normality as the COVID-19 pandemic subsides, it is easy to think of its disruption as over. This is particularly the case as wider challenges around funding and staffing become ever more pressing. But we must not forget the effects of the pandemic on young people’s education and well-being cast a long shadow. The continued disruption last academic year, including due to higher than usual staff absence causing short-notice class cancellations, has continued to reinforce immediate effects. Worse, early signs that impacts have widened existing inequalities show no sign of changing.

The first step to addressing these unequal effects is understanding who is most affected and how much. Findings from the first wave of the COVID Social Mobility & Opportunities (COSMO) study help us to do exactly this. COSMO is a representative study of young people currently in Year 13 (or equivalent) across England. Over 13,000 participants and their parents (to whom we are extremely grateful) reported on their experiences during the pandemic and beyond. This helps us to paint a vivid picture of their lives during this momentum time, including as a baseline for understanding their transitions in years to come. Acting on their messages is vital to continue to inform work across the education sector.

Young people’s educational experiences during COVID-19 lockdowns varied a lot. To take one example, we looked at the provision of live online lessons. In the early pandemic, the most dramatic differences were between the state and private sectors. State schools with well-off intakes caught up with private schools in the early 2021 lockdown. But schools with poorer intakes continued to lag behind, likely due to tackling important welfare needs.

And the differences were not only in what schools could offer. Less well-off young people were more likely to report barriers to learning at home. They were less likely to have a quiet space to focus on learning. And they were more likely to use a mobile device or have to share devices to carry out online activities. We also confirmed that those affected by these issues did, indeed, report spending less time on schoolwork during lockdowns.

Due to this disruption and its variability, concerns about being behind in their learning are widespread. Four in five young people told us that their progress in education had suffered due to the pandemic. And almost half disagreed that they had managed to catch up with the learning they lost. Over a third felt they had fallen behind their classmates. This rises to almost half for those who attended schools with the least well-off intakes.

On top of this, efforts to help students catch up have not reached as many as we would hope. Almost half of young people said they had received no specific catch-up learning at all. Despite the efforts of the National Tutoring Programme, only 27% reported receiving tutoring. And less than a third took up the most available option of extra online classes.

Bringing this together, young people were evenly split on whether they felt prepared for their next step in education or beyond. As a result, it is likely to be even harder than usual to plan courses’ curricula to meet pupils where they are starting. But it is vital to do so in order not to leave behind students such as these.

Dr Jake Anders is COSMO’s Principal Investigator.

This post first appeared on the COSMO blog.

Understanding young people’s unequal experience of the pandemic: now and into the future

Jake Anders is the Principal Investigator of the COVID Social Mobility & Opportunities study (COSMO), and Associate Professor and Deputy Director of the UCL Centre for Education Policy & Equalising Opportunities (CEPEO)

Today marks an important landmark in the COVID Social Mobility & Opportunities study (COSMO) and, with it, our understanding of the unequal impacts of the COVID-19 pandemic on young people’s life chances. After more than 18 months of development and fieldwork, we are releasing initial findings and data from the first wave of this planned cohort study.

Unlike much other research from the pandemic, COSMO comes as close as possible to recruiting a representative group of young people from across the country. This is because it is based on invitations to a random sample of all young people in schools across England, rather than a convenient sample collected for some other purpose who are not, therefore, necessarily similar to the population of young people as a whole. Why does this matter? Having a representative sample gives us increased confidence that our findings paint an accurate picture of the extent and variation in the experiences of this group.

As a result, we are extremely grateful to the more than 13,000 young people across England who responded to our invitation to take part in this study, providing us with details of their experiences to help us build up a picture of the lives of this generation and how it has been disrupted. Through this, COSMO is providing vital new evidence on the effects of the pandemic on the lives of young people, with strong signs that it has severely widened existing educational inequalities: 80% of participants told us their academic progress suffered because of the pandemic’s disruption — a figure that is even higher among those from less advantaged backgrounds. Moreover, there are worrying signs that initial impacts have not been fully addressed by the policy response in this country: only about a third of young people told us they have been able to catch up with their lost learning.

Participants’ views on whether their academic progress has suffered

We are documenting aspects of young people’s experiences during and beyond the pandemic in a series of briefing notes authored by members of the COSMO research team, with the first three of these on ‘lockdown learning’, ‘education recovery and catch up’, and ‘future plans and aspirations’ released today and more to come. These shine a light on important details of young people’s experiences.

Changes to educational plans by COVID-19 status

They highlight, for example, that well over half of young people report some change to their education and career plans because of the pandemic, and that this is particularly the case among those who report more acute direct experiences of COVID-19 itself, raising important questions about the consequences of such shifts if they are realised in the years to come.

They also increase our understanding of barriers to learning in lockdown, helping us to break down the factors that seemed to limit home learning, notably the lack of suitable electronic devices to engage in remote lessons or other online educational activities — this is particularly concerning given that over half of those who lacked such a suitable device told us this need had still not been met at the end of the second period of national school closures.

The extremely rich data from COSMO covering the circumstances and experiences of this representative sample of young people, are now available for other researchers to use to address other vital research questions. COSMO is not just a standalone effort to set out the findings as we see them, but also a resource to support others carrying out research to understand this generation.

But, importantly, COSMO’s ability to tell us about these short-term effects is just the start. COSMO is designed as a cohort study, which means that we aim to continue following the lives of its members over the coming years. Just as we are publishing findings from our first survey, we are just getting started in the process of inviting members of the cohort to take part in the first follow-up. Establishing this link through time will be vital to understanding whether conditions during the pandemic have lasting consequences for decisions that young people make about their wellbeing, education, careers and more. Cohort studies for previous generations, such as the British Cohort Study that continues to follow a group of people born in 1970, have been vital to illuminating our understanding of social mobility in society – extending such understanding to the current generation with their unique experiences completing their education will only become more vital.

Whether or not we think of the pandemic as over, its effects will continue to cast a long shadow, and COSMO will continue to help us to understand this in the years to come.

This post first appeared on the COSMO blog.

Turbulence on the glide path: A-level results 2022

Jake Anders and Lindsey Macmillan

This year sees A-level (or equivalent) results for the cohort of pupils who were on the cusp of sitting their GCSEs in 2020 when exams were first cancelled due to the Covid-19 pandemic. Now exams are back, much to the dismay of many who believe that teacher-assessed grades should become the new norm. Here at CEPEO, we’ve regularly made the case for externally set and externally marked exams, based on academic evidence that teacher assessment leads to biases against certain groups. But what has the return of exams meant for results?

The implementation of a “glide path” has been successful – generally speaking, the results are pretty much halfway between those from 2019 and 2021. While the headlines are all focused on grades being down on last year, they are, if anything, slightly more generous than might have been predicted given the intention to deal with half of the grade inflation from teacher assessment seen in 2020 and 2021. This will be driven by the fact that these weren’t strictly exams in the 2019 sense – leniency in marking was encouraged and pupils had extra information in advance of exams, which may also have had other consequences that we’ll come back to shortly.

They don’t mean that learning loss for these students is no longer a problem – since the overall level of grades was decided by that “glide path”. As such, we do not know if this group of students would have achieved as good grades as their peers in 2019 despite the disruption. This is particularly the case given some remaining incomparability with 2019 around leniency in marking and advance notice of exam content. Ongoing concerns about learning loss, its consequences, and support for catching up need to be informed by other evidence, not allayed by these results.

Regional differences present a worrying picture for inequalities – the gap in the proportion of pupils achieving A or above at A level has increased between regions, from 7.3ppts in 2019 to 8.2ppts in 2022. So, London and the South East are continuing to pull away from other areas while the North East and Yorkshire and the Humber are particularly being left behind. This trend of Southern regions pulling ahead was also evident in the Key Stage 2 data released last month, and will provide a big headache for a government that has put ‘levelling up’ at the heart of what they plan to achieve.

Distribution of grades in 2019 and 2022 by English region

Source: JCQ

A mixed picture across centre type – FE colleges and independent schools have seen the biggest absolute declines in the proportion of A-levels awarded A or above. For independent schools, this is not altogether surprising given their huge grade inflation from 2019 to 2021, which saw a massive 25.7ppt increase in the proportion of pupils achieving A or above over the period, and over 70% of all private school pupils awarded an A or above in 2021. This number has returned to a more ‘reasonable’ 58% for 2022. Two points to make here: First, this highlights the extent of grade inflation in this type of centre during the period of teacher assessment. Second, this is still an extremely high proportion of pupils achieving a grade A or above, which could be indicative of private school pupils being better placed to make use of the additional information on exam content made available in advance this year, as well as the advantages that private school pupils had in terms of relative learning loss during the pandemic.

FE colleges are far more puzzling – they have seen a similar absolute decline in the proportion achieving A or above, but from a much lower base. They saw a 16.5ppt rise in pupils achieving A or above from 2019 to 2021, only 29.2% reached this level of achievement by 2021. This has fallen back down to 16.5% in 2022. Somewhat speculatively, it seems plausible that students in FE colleges were, in contrast to those at independent schools, particularly badly affected by the disruption wrought by COVID-19.

Percentage of A-levels awarded A*/A by centre type in 2019, 2021 and 2022

Source: https://www.gov.uk/government/news/guide-to-as-and-a-level-results-in-england-summer-2022

Grammar schools, on the other hand, have found themselves performing at a level that is almost comparable with that seen in 2021, despite the different type of assessment, and 14.1ppts higher than 2019. One explanation for this could be, like private schools, that grammar school teachers and pupils were better placed to use the additional information released in advance of exams this year, focusing on a far narrower set of curricula. Additionally, this may be the first indication (through exam results at least) that pupils from more advantaged backgrounds were less constrained by learning loss during the pandemic.

In many ways, these results have not told us much we did not already know – it was already pretty clear that independent school grades, especially, were artificially inflated by teacher assessment, although the percentage coming back down now challenges those who told a tale of Covid learning loss advantages last year. We already knew that the impacts of COVID-19 disruption were likely to be unequal. And we already knew that this was going to be an extremely tough transition for those young people in the middle of it, rightly proud of what they have achieved through difficult circumstances, but unsure of what it all means compared to their peers one year above or below them.

This post first appeared on the UCL CEPEO blog.

Learning About Culture: The importance of arts-based learning, the limits of what we know about it, and the challenges of evaluating it

Jake Anders, Kim Bohling, Nikki Shure and Alex Sutherland

There is little doubt about the importance of arts and culture to the education and upbringing of young people. Arts-based education gives young people an important means of creative expression and “arts for arts’ sake” is the best argument for having arts-based education in schools. However, far less is known about the link specifically between arts-based learning activities and pupils’ educational outcomes – partially due to a lack of robust studies on this topic. Yet this is a link that is often invoked as part of the overall importance of these programmes, partly in response to a perception that an increased focus on “core educational outcomes” is squeezing arts-based education out of schooling.

Over the past four years, a team from UCL and the Behavioural Insights Team has been working with the Education Endowment Foundation (EEF), the Royal Society for the Arts (RSA) and five arts-based education organisations on a project called Learning About Culture (see Table 1 below for programme detail). At the heart of this project are five randomised controlled trials (RCTs) involving around 8,500 children in 400 state schools across England. These evaluations were designed to look at the impact of five specific arts-based learning interventions on literacy outcomes. To our knowledge, these trials represent the largest collection of RCTs testing arts-based approaches on attainment outcomes. This body of research represents a significant step forward in understanding how to assess the relationship between creative activities and pupil outcomes, which is in itself important.

Each of the programme reports is linked to below and an overarching report that synthesises the findings, lessons, and recommendations can be found here. What you’ll immediately notice is the diversity of approaches we looked at – including music, storytelling, and journalism – reflecting the richness and diversity of the sector.

Table 1. Learning about Culture programmes

Each programme name is hyperlinked to the EEF project page.

Programme name (Developer):Description:
First Thing Music (Tees Valley Music Service)Programme to train teachers in the Kodály method of music instruction in order to deliver daily a structured, sequential music curriculum of increasing progression (Key Stage 1)
Speech Bubbles (London Bubble)Weekly drama and storytelling intervention aimed at supporting children’s communication skills, confidence, and wellbeing. (Key Stage 1)
The Craft of Writing (Arvon, University of Exeter, Open University)Programme to develop teachers as writers combined with explicit focus on pedagogical implications for the classroom. (Key Stage 2)
The Power of Pictures (Centre for Literacy in Primary Education)Specialist training from published author-illustrators and expert teachers helps primary teachers to develop their understanding of the craft of picture book creation. (Key Stage 2)
Young Journalist Academy (Paradigm Arts)The project aims to develop pupils’ writing by involving them in journalism. In doing so, it aims to provide pupils with a meaningful purpose for writing and teach specific writing techniques. (Key Stage 2)

What did we find?

When compared to ‘business as usual,’ we were unable to find improvements in pupil attainment in any of the five trials that we could reliably say weren’t due to chance. However, it’s important to emphasise that this is an extremely challenging barrier to clear and the fact of the matter is that most of the trials that the EEF funds don’t find impacts of interventions on pupil learning outcomes.

While it is easy to focus on the lack of a positive impact in the outcome measures, we also want to emphasise the trials found no evidence of detrimental effects from introducing such programmes. That is actually really good news, because it means that including arts-based programmes alongside ‘core curriculum’ subjects isn’t a zero-sum game where increasing time on arts means lower grades elsewhere.

And, as we pointed out above, improving pupil academic attainment is not the best or only reason for schools to implement arts-based interventions in schools. Although they did not improve literacy test scores, in interviews with participating teachers and pupils, we found that the programmes generated a great deal of enthusiasm among the teachers and pupils who took part in them. Perceived improved pupil engagement was a theme that emerged from the implementation and process evaluations across the five programmes.

In the overarching report, we also stress that these results should absolutely not be seen as the last word in whether arts-based learning is effective in improving outcomes for pupils. Necessarily, in this kind of research, we focused on one set of outcomes, which could be quantified and measured over a fairly short time horizon. But benefits could accrue in many other ways that we just couldn’t capture. For one, having these initiatives available to pupils may have long term consequences for the subjects these pupils choose at GCSE or A level or the career paths they choose to follow. We don’t know that there are these benefits, either, but our evidence shouldn’t be used to discount such possibilities.

Our reflections as evaluators

The overarching report contains thoughts and lessons for multiple audiences: researchers, funders, and arts organisations. For brevity, we’ve only selected a few takeaways to highlight here.

Evaluators and funders

There is a line of argument against our efforts here that what we can measure in trials (and research more broadly) is not always what ‘matters’, or what we ‘should’ measure. Equally, some will point to challenges in measuring what we did use as outcomes, as well. We know that the measures used are imperfect, but given the choice between imperfect measurement of something versus perfect measurement of nothing – or something further removed from the intervention – then we stand by our decision to do what we can in an imperfect world. This isn’t an abstract research issue: in order to be able to ascertain whether something is effective (or not) we need to be clear what we expect to change and measure that as best we can.

In line with EEF’s policy, reflecting their primary aim as an organisation, our impact evaluations focused on measuring pupil attainment outcomes. While this approach has many strengths given the undoubted importance of such outcomes, these projects – where we see positive signs of engagement based on the implementation and process evaluation but ultimately no impacts on our measured outcomes – highlight one of its key limitations: a null finding leaves a lot of unanswered questions. An alternative approach – with similarities to the increased emphasis on ‘mechanism experiments’ in economics and particularly important where there is a limited evidence base about how interventions work – would focus first on establishing whether the interventions do indeed affect the intermediate steps via which they are thought to improve attainment. This would help us first to establish whether the programme is working as we think it does or if there is more to be done to understand this crucial first stage to achieving impact on pupils’ academic attainment.

Arts organisations

We really appreciate the courage and commitment from the arts-based education organisations who put themselves forward to participate in a multi-year evaluation process. The EEF’s support for both an individual and overarching approach to the evaluation meant that we were able to observe themes across the programmes that could be useful to other arts organisations. From these themes, we offer some recommendations for consideration.

Ensure buy-in and engagement from school staff at multiple levels.

High teacher buy-in was crucial for the day-to-day delivery of the programme, and senior leadership team (SLT) buy-in was important for supporting the teacher in high-quality delivery. For example, SLT members were able to ensure teachers had access to necessary resources and space, as well as ensure there was time in the timetable for the programme.

Carefully consider programme resource requirements and test assumptions about what’s available in schools

The interventions placed different demands on schools in terms of the resources needed to take part, and even where required resources were considered ‘standard’, challenges were still reported. In some cases, schools did not have resources, such as arts supplies, that were assumed to be available in most schools. In other cases, the schools had the required resources, such as technological equipment, but they were difficult to access. Organisations may want to consider how to surface these challenges early in set-up and whether they can provide any support to schools in overcoming them.

On a more personal note

As independent evaluators, we have a responsibility to be as objective as possible, recognise our biases, and do our best to minimise their influence on our work. We are also all researchers who care deeply about improving outcomes for pupils and furthering our understanding of ‘what works’ to support pupil development. When we are able to take the ‘evaluator hat’ off, this team also broadly supports the inclusion of arts in the school day, and some of us have direct experience of delivering arts-based learning opportunities either in the school day or extended learning space. We would have been thrilled to report that the programmes had a significant impact on attainment outcomes – not only to further enhance the toolkit for improving pupil outcomes, but also to secure further protection for the arts in the school day. Ultimately, we are not able to report those outcomes, and we stand by the findings of the six reports produced. We are still supporters of arts in education and we also enthusiastically support further research in this space, as there is certainly more to learn.

This post originally appeared on the UCL CEPEO blog.

A-levelling up: the thorny path back from teacher assessed grades

By Jake Anders, Claire Crawford, and Gill Wyness
This piece first appeared on theGuardian.com.

This week’s GCSE and A level results confirmed the expectations of many who study education policy: the proportion of students achieving top grades in these qualifications has increased substantially compared to 2019, especially at A level. Students themselves should be extremely proud of their results, which were achieved under very difficult circumstances. Likewise, teachers have worked extremely hard to make the best assessment they can of their pupils’ performance. But there is no getting around the fact that these results are different – and not directly comparable with – pre-Covid results.

It is right to allow for the fact that students taking GCSEs and A levels this year and last are at a disadvantage compared to previous cohorts. In-person exams would have been next to impossible in 2020, and those assessed this year have missed significant amounts of schooling.

To deal with this, the government chose an entirely different means of measuring performance: teacher assessments. (We advocated a different approach, based on more flexible exams, in 2021.) This year’s approach has been rather more orderly than last year’s chaos, but the wide range of measures that teachers could consider – such as mock exams, in-class tests and coursework – inevitably led to variation in how schools assessed their pupils.

This year’s grades may also be capturing average or ‘best’ performance across a range of pieces of work, rather than a snapshot from one or two exams. This seems to have been particularly true at A level, where grades have immediate consequences for university entry decisions. In short, it is unsurprising that grades based on teacher assessment are higher than those based on exams alone: while some have called this grade inflation we think it’s more accurate to say that they are capturing different information.

A level grade distribution in 2019, 2020 and 2021

But given they have been presented on the same scale, the stark increase in grades compared to pre-Covid times present significant challenges for current and future cohorts.

Even making comparisons between pupils within the 2021 cohort may be challenging. Using teacher assessment is likely to have disadvantaged some students relative to others. Previous research has shown that Black Caribbean pupils are more likely than white pupils to receive a grade from their teacher below their score in an externally marked test taken at the same time. Similarly, girls have also been found to perform better at coursework, while boys do better at exams on average. Differences by gender have been particularly apparent this year, with girls seeing larger improvements in performance than boys compared to pre-pandemic.

This year’s record high scores raise challenging questions. The much larger proportion of pupils getting As and A*s at A level, for example, may lead to universities relying more heavily on alternative methods of distinguishing between applicants – such as personal statements – which have been shown to entrench (dis)advantage.

There is also the all-important question of what to do next year: are this year’s grade distributions the right starting point, or should we be looking to return to something closer to the 2019 distribution? Is it possible to go back? And would we want to?

Assuming in-person exams are feasible next year, one possibility would be to return to 2019’s system as if nothing had happened. This would probably see substantial reductions in the proportion of students getting top grades, especially at A level. One can only imagine the political challenge of trying to do this.

Even more important is that the next cohorts of GCSE and A level students (and indeed the ones that follows – we are tracking the experiences of those taking GCSEs this year as part of a new UKRI-funded cohort study, COSMO) have also been affected by the pandemic, arguably to a greater degree than this year’s. They are therefore likely to underperform their potential and get lower grades than cohorts who took their exams before the pandemic struck. That is clearly not desirable.

It is important to continue making allowances for the exceptional circumstances young people have faced during this crucial time in their education. During the period affected by pandemic learning loss, our suggestion would be to design exams with more flexibility, allowing candidates to choose which questions to answer based on their strengths, as is common in university exams. This would enable a return to the fairest way to assess students – exams – while still taking account of lost learning.

Either way, any return to exam-based grades is likely to result in an immediate pronounced drop in results compared to the last two years, especially at A level. Gavin Williamson has suggested that the government will aim instead for a “glide path back to a more normal state of affairs”. This would smooth out the unfairness of sharp discontinuities between cohorts. But it would mean moving away from grades being based on the same standard over time, instead setting quotas of students allowed to achieve each grade, gradually reducing the higher grades and increasing the lower ones. Even if that seems a good plan now, it would be very hard to stick to: the fall-out from the small reduction in pass rates seen in Scotland this week would be a taste of things to come for years.

A more radical possibility would be to reset the grading system entirely. This would get around the political issue of there being very large or deliberate small falls in grades for future cohorts, but one wonders whether this is the right time to undertake such a drastic overhaul. The pandemic will have repercussions on young people’s grades for years to come: is the best approach really a total reset right now?

The question of what to do next is one that policymakers will have to grapple with over the coming months and years. Of more fundamental importance and urgency, however, is that pupils have experienced widespread learning losses due to the pandemic – regardless of what their grades show – and are likely to be affected by these for years. Students require ongoing support throughout the rest of their educational careers, including catch up support throughout school, college and university.

We cannot simply award them GCSE and A level grades that try to look past the learning they have lost and move on – the learning loss remains and must be addressed.

Dr Gill Wyness & Dr Jake Anders are deputy directors of the UCL Centre for Education Policy & Equalising Opportunities (CEPEO). Dr Claire Crawford is an associate professor at CEPEO.

The ‘graduate parent’ advantage in teacher assessed grades

By Jake Anders, Lindsey Macmillan, Patrick Sturgis, and Gill Wyness

Following a disastrous attempt to assign pupil grades using a controversial algorithm, last year’s GCSE and A level grades were eventually determined using Centre Assessed Grades (CAGs) following public outcry. Now, new evidence from a survey carried out by the UCL Centre for Education Policy and Equalising Opportunity (CEPEO) and the London School of Economics finds that some pupils appear to have benefited from an unfair advantage from this approach – particularly pupils with graduate parents. As teachers will again be deciding exam grades this year, this finding serves as an important warning of the challenges involved in ensuring that a system using teacher assessments is fair.

The decision to cancel formal exams in 2020 was taken at a late stage in the school year, meaning that there was little time for the government to develop a robust approach to assessment. After a short consultation, the Department for Education (DfE) decided that pupils’ exam grades would be determined by the teacher’s assessment of pupils’ grades, including their ranking. However, to prevent grade inflation due to teachers’ overpredicting their pupils, Ofqual then applied an algorithm to the rankings to calculate final grades, based on the historical results of the school.

A level pupils received their calculated grades on results days 2020, and although Ofqual reporting showed that the calculated grades were slightly higher than 2019 across the grade range, many pupils were devastated to find their teacher assessed grades had been lowered by the algorithm. More than a third of pupils received lower calculated grades than their original teacher assessed grades. Following widespread public outcry, the calculated grades were abandoned, and pupils were awarded the grades initially assessed by teachers. This inevitably led to significant grade inflation compared to previous cohorts.

This also created a unique situation where pupils received two sets of grades for their A levels – the calculated grades from the algorithm and the teacher allocated “centre assessed grades” or “CAGs”.

While it is now well established that CAGs were, on average, higher than the algorithm-calculated grades, less is known about the disparities between the two sets of grades for pupils from different backgrounds. Understanding these differences is important since it sheds light on whether some pupils received a larger boost from the move to teacher predicted CAGs, and hence to their future education and employment prospects. It is also, of course, relevant to this year’s grading process, as grades will again be allocated by teachers.

Administrative data on the differences between calculated grades and CAGs is not currently publicly available. However, findings from a new UKRI-funded survey of young people by the UCL Centre for Education Policy and Equalising Opportunity (CEPEO) and the London School of Economics (LSE) can help us to understand the issue. The survey provides representative data on over 4000 young people in England aged between 13 and 20, with interviews carried out online between November 2020 and January 2021.

Respondents affected by the A level exam cancellations (300 respondents) were asked whether their CAGs were higher or lower than their calculated grades. The resulting data reveal stark differences in the extent to which pupils were given a boost by the decision to revert to CAGs. As shown in Figure 1, pupils with graduate parents were 17 percentage points more likely to report that their CAGs were higher than their Ofqual calculated grades.  The survey data are linked to administrative data on prior attainment at Key Stages 2 and 4, as well as demographic and background characteristics such as, free school meals status, ethnicity, SEN and English as an additional language). Even after accounting for differences between pupils across these characteristics, those with graduate parents were still 15 percentage points more likely to report having higher CAGs than calculated grades.

Figure 1. The proportion of young people reporting their CAGs were better than their calculated grades by whether or not they report that one of their parents has a university degree (left panel: raw difference; right panel: adjusted for demographic characteristics and prior attainment)

There are a number of possible explanations for these differences. First, it could be that pupils with graduate parents are more likely to attend particular types of schools which have a greater tendency to ‘over-assess’ grades. While not directly relevant to this sample, an extreme version of this are the documented cases of independent schools deliberately over-assessing their pupils, but this could also happen in less dramatic and more unconscious ways. It could, for example, be more likely among schools that are used to predicting grades as part of the process for pupils applying to highly competitive university courses, where over-prediction may help more than it hurts.

A second possibility is that graduate parents are more likely to lobby their child’s school to ensure they receive favourable assessments. Such practices are reportedly becoming more common this year, with reports of “pointy elbowed” parents in affluent areas emailing teachers to attempt to influence their children’s GCSE and A level grades ahead of teacher assessed grades replacing exams this summer.

A third possibility is that the relatively high assessments enjoyed by those with graduate parents is a result of unconscious bias by teachers. A recent review by Ofqual found evidence of teacher biases in assessment, particularly against those from SEN and disadvantaged backgrounds, while a new study from Russia showed that teachers gave higher grades to pupils with more agreeable personalities. Interestingly, we found no differences between FSM and non-FSM pupils, perhaps suggesting teachers were careful not to treat FSM pupils differently. But they may nonetheless exhibit an unconscious positive bias towards pupils from backgrounds that tend to be associated with higher educational achievement.

Our results do not afford any leverage on which of these explanations, if any, is correct. Regardless of what is behind this systematic difference, our findings show that pupils with more educated parents received an unfair advantage in their A level results last year, with potential repercussions for equality and social mobility. They also highlight this is a substantial risk for this year’s process – perhaps even more so without the expectation of algorithmic moderation: grading pupils fairly in the absence of externally set and marked assessments is setting teachers an almost impossible task.

The working paper ‘Inequalities in young peoples’ education experiences and wellbeing during the Covid-19 pandemic’ is available here.

Learn more about our project on the impact of the pandemic on young people here.

Notes
The UKRI Covid-19 funded UCL CEPEO / LSE survey records information from a sample of 4,255 respondents, a subset of the 6,409 respondents who consented to recontact as part of the Wellcome Trust Science Education Tracker (SET) 2019 survey. The SET study was commissioned by Wellcome with additional funding from the Department for Education (DfE), UKRI, and the Royal Society. The original sample was a random sample of state school pupils in England, drawn from the National Pupil Database (NPD) and Individualised Learner Record (ILR). To correct for potentially systematic patterns of respondent attrition, non-response weights were calculated and applied to all analyses, aligning the sample profile with that of the original survey and the profile of young people in England.

This work is funded as part of the UKRI Covid-19 project ES/V013017/1 “Assessing the impact of Covid-19 on young peoples’ learning, motivation, wellbeing, and aspirations using a representative probability panel”.

This work was produced using statistical data hosted by ONS. The use of the ONS statistical data in this work does not imply the endorsement of the ONS in relation to the interpretation or analysis of the statistical data. This work uses research datasets which may not exactly reproduce National Statistics aggregates.

This post originally appeared on the UCL CEPEO blog.

The challenges of COVID-19 for young people need a new cohort study: introducing COSMO 

Jake Anders and Carl Cullinane 

The COVID-19 pandemic and its impact is a generation-defining challenge. One of its most concerning aspects, particularly in the long term, is the already profound effect it has had on young people’s lives. Disruption to their development, wellbeing and education could have substantial, long-lasting effects on later life chances, particularly for those from lower-income homes. Evidence is already showing disadvantaged pupils lagging 5 months behind their peers. This poses a unique challenge for educational policy and practice, with the scale of the disruption requiring solutions to match that scale.  

In order to address these impacts, it is vital that we fully understand these effects, and in particular, the disproportionate burden falling on those from certain groups, including those from lower socio-economic backgrounds and minority ethnic groups. This needs high-quality data. Recovering from the effects of the past 12 months will be a long–term project, and to reflect this we need research of similar ambition. 

The COVID Social Mobility and Opportunity Study (COSMO for short), launched today, seeks to play this role, harnessing the power of longitudinal research to capture the experiences of a cohort of young people for whom the pandemic has had an acute impact, and its effects on their educational and career trajectories. 

This country has a grand tradition of cohort studies, including the pioneering 1958 National Child Development Study and the 1970 British Cohort Study. Such studies are a key tool in understanding life trajectories and the complex factors that shape them. And they are particularly vital when it comes to measuring the impact of events that are likely to last through someone’s life course. The existing longitudinal studies, including those run by our colleagues in the UCL Centre for Longitudinal Studies, have played a huge role in understanding the impacts of the pandemic on society in the last year. 

But there is a key gap in the current portfolio of cohort studies: and that is the generation of young people at the sharp end of their school education, who would have taken GCSEs this summer, and within a matter of months will be moving onto new pathways at sixth form, further education, traineeships and apprenticeships. The impacts on this group are likely to be profound and long-lasting, and understanding the complex elements that have aggravated or mitigated these impacts is crucial. 

A variety of studies have already collected some such data, providing emerging evidence of inequalities in pupils’ outcomes and experiences of remote schooling. This has highlighted alarming challenges for pupils’ learning and wellbeing. However, to develop a full understanding we require the combination of rich, representative, survey data on topics such as learning loss experiences, wellbeing, and aspirations, linked with administrative data on educational outcomes, and concurrent interventions. We also need to follow up those young people over the next few years as they pass through key stages of education and their early career, to understand what has happened next, ideally long into their working lives. 

Such evidence will be key in shaping policies that can help to alleviate the long–term impacts on young people. Which groups have suffered most and how, how long will these impacts persist, and how can we reduce their effect. These will be fundamental questions for national policymakers, education providers, employers and third sector organisations in the coming years, both in the UK and internationally. 

That’s why we’re extremely excited to be launching COSMO with funding from UK Research and Innovation (UKRI)’s Ideas to Address COVID-19 response fund. Our study will deliver exactly that data over the coming years, helping to inform future policy interventions that will be required, given that the huge effects of the pandemic are only just beginning. As the British Academy pointed out on the anniversary of the first COVID lockdown – this is not going to go away quickly. 

Beginning this autumn, the study will recruit a representative sample of 12,000 current Year 11 pupils across England, with sample boosts for disadvantaged and ethnic minority groups plus targeting of other hard-to-reach groups. Young person, parent, and school questionnaires – enhanced with administrative data from DfE– will collect rich data on young people’s experiences of education and wellbeing during the past challenging 12 months, along with information on their transitions into post-16 pathways via this summer’s unusual GCSE assessment process. 

The study is a collaboration between the UCL Centre for Education Policy & Equalising Opportunities (CEPEO), the UCL Centre for Longitudinal Studies (CLS) and the Sutton Trust. The study will harness CEPEO’s cutting-edge research focused on equalising opportunities across the life course, seeking to improve education policy and wider practices to achieve this goal. The Sutton Trust also brings 25 years of experience using research to inform the public and achieve policy change in the area of social mobility.  

COSMO will also be part of the family of cohort studies housed in the UCL Centre for Longitudinal Studies, whose expertise in life course research is world-renowned. We are also working closely with Kantar Public, who will lead on delivering the fieldwork for this large study, alongside NatCen Social Research. More broadly still, all our work will be co-produced with project stakeholders including the Department for Education and the Office for Students. We are also working with partners in Scotland and Wales to maximise comparability across the nations. 

We are excited for COSMO to make a big contribution both to the landscape of educational research and to the post-pandemic policy environment, and we are delighted to be getting to work delivering on this promise over the coming years.

This post originally appeared on the UCL CEPEO blog.

Vaccine hesitancy in children and young adults in England

By Patrick Sturgis, Lindsey Macmillan, Jake Anders, Gill Wyness

Children and young people are, mercifully, at extremely low risk of death or serious illness from the coronavirus and, for this reason, they are likely to be the last demographic in the queue to be vaccinated, if they are vaccinated at all. Yet, there are good reasons to think that a programme of child vaccination against covid-19 will eventually be necessary in order to free ourselves from the grip of the pandemic. In anticipation of this future need, clinical trials assessing the safety and efficacy of existing covid-19 vaccines on young people have recently commenced in the UK.

While children and young people experience much milder symptoms of covid-19 than older adults, there is currently a lack of understanding of the long-term consequences of covid-19 infection across all age groups and there have been indications that some children may be susceptible to potentially severe and dangerous complications. Scientists also believe that immunisation against covid-19 in childhood may confer lifetime protection (£), reducing the need for large-scale population immunisation in the future.

Most importantly, perhaps, vaccination of children may be required to minimise the risk of future outbreaks in the years ahead. If substantial numbers of adults refuse immunisation and the vaccines are, as seems likely, less than 100% effective against infection, vaccination of children will be necessary if we are to achieve ‘herd immunity’.

We now know a great deal about covid-19 vaccine hesitancy in general populations around the world from a large and growing body of survey and polling data and, increasingly, from actual vaccine uptake. Much less is known, however, about vaccine hesitancy amongst children and younger adults. Here, we report preliminary findings from a new UKRI funded survey of young people carried out by Kantar Public for the UCL Centre for Education Policy and Equalising Opportunity (CEPEO) and the London School of Economics. The survey provides high quality, representative data on over 4000 young people in England aged between 13 and 20, with interviews carried out online between November 2020 and January 2021. Methodological details of the survey are provided at the end of this blog.

Respondents were asked, “If a coronavirus vaccine became available and was offered to you, how likely or unlikely would you personally be to get the vaccine?”. While the majority (70%) of young people say they are likely or certain to get the vaccine, this includes 25% who are only ‘fairly’ likely. Worryingly, nearly a third express some degree of vaccine hesitancy, saying that they either definitely won’t get the vaccine (9%) or are that they are not likely to do so (22%).

Although there are differences in question wording and response alternatives, this represents a substantially higher level of vaccine hesitancy than a recent Office for National Statistics (ONS) survey of UK adults, which found just 6% expressing vaccine hesitancy, although this rose to 15% amongst 16 to 29 year olds.

Differences in vaccine hesitancy across groups

We found little variation in hesitancy between male and female respondents (32% female and 29% male), or between age groups. However, as can be seen in the chart below, there were substantial differences in vaccine hesitancy between ethnic groups. Black young people are considerably more hesitant to consider getting the vaccine than other ethnic groups, with nearly two thirds (64%) expressing hesitancy compared to just a quarter (25%) of those who self-identified as White.  Young people who identified as mixed race or Asian[1] expressed levels of hesitancy between these extremes, with a third (33%) of mixed race and 39% of Asian young people expressing vaccine hesitancy. This ordering matches the findings for ethnic group differences in the ONS survey, where 44% of Black adults expressed vaccine hesitancy compared to just 8% of White adults.

To explore potential sources of differences in vaccine hesitancy, respondents were asked to state their level of trust in the information provided by a range of different actors in the coronavirus pandemic. The chart below shows wide variability in expressed levels of trust across different sources between ethnic groups, but most notably between Black young people and those from other ethnic groups. Young people self-identifying as Black were considerably less likely to trust information from doctors, scientists, the WHO and politicians and more likely to trust information from friends and family than those from other groups. Although in terms of overall levels, doctors, scientists and the WHO are most trusted across all groups. Encouragingly, only 5% of young people say they trust information from social media, a figure which was consistently low across ethnic groups.

We also find evidence of a small social class gradient in vaccine hesitancy, with a quarter (25%) of young people from families with at least one parent with a university degree[2] expressing vaccine hesitancy compared to a third (33%) of young people with no graduate parent.

We can also compare levels of vaccine hesitancy according to how young people scored on a short test of factual knowledge about science. [3]  Vaccine hesitancy was notably higher amongst respondents who were categorised as ‘low’[4] in scientific knowledge (36%) compared to those with ‘average’ (28%), and ‘high’ (22%) scientific knowledge. This suggests that vaccine hesitancy may be related, in part, to the extent to which young people are able to understand the underlying science of viral infection and inoculation and to reject pseudoscientific claims and conspiracy theories.

How much are differences in vaccine hesitancy just picking up underlying variation between ethnic groups in scientific knowledge and broader levels of trust? In the chart below, we compare raw differences in vaccine hesitancy for young people from the same ethnic group, sex, and graduate parent status (blue plots) with differences after taking account of differences in scientific knowledge and levels of trust in different sources of information about coronavirus. The inclusion of these potential drivers vaccine hesitancy do account for all of the differences between ethnic and social class groups. While Black young people are around 40 percentage points more likely to express vaccine hesitancy than their White counterparts, this is reduced to 33 ppts when comparing Black and White young people with similar levels of scientific knowledge and (in particular) levels of trust in sources of coronavirus information.

Our survey shows high levels of vaccine hesitancy amongst young people in England, which should be a cause for concern, given the likely need to vaccinate this group later in the year. We also find substantial differences in hesitancy between ethnic groups, mirroring those found in the adult population, with ethnic minorities – and Black young people in particular – saying they are unlikely or certain not to be vaccinated. These differences seem to be related to the levels of trust young people have in different sources of information about coronavirus, with young Black people more likely to trust information from friends and family and less likely to trust health professionals and politicians.

There are reasons to think that actual vaccine take up may be higher than these findings suggest. First, Professor Ben Ansell and colleagues have found a decrease in hesitancy amongst adults between October and February, a trend which was also evident in the recent ONS survey.  It seems that hesitancy is declining amongst adults as the vaccine programme is successfully rolled out with no signs of adverse effects and this trend may also be evident amongst young people. Given that parental consent will be required for vaccination for under 18s, it may be the case that parental hesitancy is as important for take up.

There may also have been some uncertainty in our respondent’s minds about what is meant by ‘being offered’ the vaccine, given there were no vaccines authorised for young people at the time the survey was conducted and no official timetable for immunisation of this group. Nonetheless, this uncertainty cannot explain the large differences we see across groups, particularly those between White young people and those from ethnic minority groups.

If the vaccine roll out is to be extended to younger age groups in the months ahead, we will face a considerable challenge in tackling these high levels of and disparities in vaccine hesitancy.

Methodology

The UKRI Covid-19 funded UCL CEPEO / LSE survey records information from a sample of 4,255 respondents, a subset of the 6,409 respondents who consented to recontact as part of the Wellcome Trust Science Education Tracker (SET) 2019 survey. The SET study was commissioned by Wellcome with additional funding from the Department for Education (DfE), UKRI, and the Royal Society. The original sample was a random sample of state school pupils in England, drawn from the National Pupil Database (NPD) and Individualised Learner Record (ILR). To correct for potentially systematic patterns of respondent attrition, non-response weights were calculated and applied to all analyses, aligning the sample profile with that of the original survey and the profile of young people in England. Our final sample consists of 2,873 (76%) White, 208 (6%) Black, 452 (12%) Asian, 196 (5%) Mixed, and 50 (1%) Other ethnic groups.  The Asian group contains respondents who self-identified as Asian British, Indian, Pakistani, Bangladeshi, Chinese or ‘other Asian’.

[1] Respondents in the Asian category are a combination of Indian, Pakistani, Bangladeshi, Chinese or ‘other Asian’ origin.

[2] We have not yet liked the survey data to the National Pupil Database and Individualised Learner Records which will enable us to use an indicator of eligibility for free school meals and IDACI. Currently we use parent graduate status as a proxy for socio-economic status.

[3] Once the survey is linked to the National Pupil Database we will be able to look across a wider range of measures of school achievement.

[4] There were ten items in the quiz, ‘low’ knowledge equated to a score of 5 or less, ‘average’ knowledge to a score of 6 to 8, and ‘high’ knowledge to a score of 9 or 10. Note that this test was administered in the previous (2019) wave of the survey.

This work is funded as part of the UKRI Covid-19 project ES/V013017/1 “Assessing the impact of Covid-19 on young peoples’ learning, motivation, wellbeing, and aspirations using a representative probability panel”.

This originally appeared on the UCL CEPEO blog.