All posts by Jake Anders

Senior Research Associate at UCL Institute of Education.

GCSE and A-level results: it’s not just the grades that matter

This blog post was co-authored with Catherine Dilnot and was commissioned and first published by The Conversation UK. It also appears on the IOE London Blog.The Conversation

A-level results will soon be out, with more than 300,000 students eagerly waiting to find out if they’ve made the grade. Then come GCSE results, with even more students keen to find out how they’ve done.

Whether students are heading to university, into an apprenticeship or straight into employment, chances are they will all be wishing and hoping and dreaming and praying of a set of grades that will reflect their level of academic accomplishment.

For would-be university applicants, there is often a requirement that students take a particular set of subjects at A-level – and achieve a certain grade – to be in with a chance of getting a place on a degree course. To study medicine, for example it’s often required that an applicant has taken chemistry and biology at A-level.

In this way, the subjects a student chooses to study at school can have long term consequences. In England, young people start making decisions on subject choice at the age of 14 when they pick GCSE options. For many pupils this may seem far too early to be thinking about what they want to do with the rest of their life. So given the fact that many students may not have decided what career path they want to take, are there subjects that are “better” to study than others?

The current advice

The Russell Group – which is made up of 24 leading UK universities – publishes an annual guide to A-level subject choice for 16-year-olds known as “informed choices”. This suggests A-levels in science, maths, languages, history and geography are good choices for students to take if they want to keep their options open.

This is also in part why the English Baccalaureate (EBacc) – which aims to give students a wide background in a variety of subjects at GCSE level – was introduced in 2010. According to the schools minister, Nick Gibb, it includes subjects the Russell Group identifies as “key for university study”. To count towards the EBacc, a pupil must achieve GCSE grade C or above in English, maths, history or geography, two sciences and a language.

With this in mind, our research set out to understand the implications of subject choice and if these choices then play a part in whether students go to university – and where they end up studying.

We looked at the subjects chosen by young people at the age of 14 and 16 and found that pupils who study the full set of EBacc subjects are slightly more likely to go to university than those who don’t.

Our research also revealed that studying certain A-level subjects often leads to a place at a better ranked university. So a student who studies some combination of science, maths, languages, history and geography is more likely to attend a higher ranked university, than a student who chooses A-levels outside of these subjects.

Vocational vs traditional

Our research also revealed that studying more vocational subjects at both GCSE and A-level may be less helpful in terms of getting into a higher ranked university. We found that those who studied applied GCSE subjects (which are more vocational) were less likely to attend university.

These vocational style GCSEs were introduced in 2002 and include subjects such as applied business and applied home economics. But their introduction has since been criticised, as many of the qualifications have been downgraded in performance tables.

There was found to be a similar picture at A-level. Students who studied the more vocational study subjects – such as accounting or business – were more likely to go to a lower ranked university.

The most striking results were in law. Consistent with anecdotal evidence that higher ranking universities “don’t like” law A-level, our research shows that studying law at A-level is associated with attending a lower ranked university. So although a 16-year-old who aspires to have a career in law, accounting or business might think that an A-level directly related to the profession would help them take their chosen path, this may not actually be the case. But whether this is because law A-level is perceived by universities to be an easier A-level, or because those with law A-level are applying to lower ranked universities is unclear.

Either way, what all this shows is that while the subjects young people study in school are important for next steps in education, there are some subjects that can be more important than others in helping to further horizons.

Although that said, it’s important to emphasise that the differences are not large. Ultimately, it’s far more important to perform well in whatever subject studied. But still, when it comes to students deciding what subjects to choose at A-level or GCSE, it might be worth them trying to keep their options open, where possible.

The findings discussed in this post draw on work published in UCL Centre for Longitudinal Studies Working Papers by Jake Anders, Morag Henderson, Vanessa Moulton, Alice Sullivan and Catherine Dilnot.

Advertisements

How are school to work transitions changing in England?

This post was commissioned and first published by the Jacob’s Foundation Blog on Learning and Development (BOLD).

In recent years, there has been growing concern about the number of young people failing to make a successful transition from education into employment. Increasingly, this appears to be a structural, rather than cyclical, issue. We see evidence of this from the fact that although youth unemployment in the UK was falling in the late 1990s and early 2000s, it started rising again as early as 2004, long before the general downturn in the economy.

This is an important issue, not least because making a successful school to work transition is a good predictor for young people’s long-term economic success. Multiple studies have shown that periods of unemployment during these early years have long-term negative effects on later employment and earnings prospects.

As such, my colleague Richard Dorsett and I set out to explore how these transitions have changed for successive generations of young English people born in roughly 1960, 1970, 1980 and 1990. A lot has changed about labour markets over these thirty years. Many of these changes apply not just to England but also other developed economies. As such, our findings are likely to be relevant in other parts of the world too.

We analyse data on young people’s main activity each month between age 16 and 19, using statistical techniques known as “sequence analysis” and “cluster analysis” designed to group together individuals who experience similar types of transitions. We place them into broad categories of “Entering the Labour Market”, “Getting More Education” and “Experiencing a Difficult Transition”. Broadly speaking, those who are Entering the Labour Market move into stable jobs before they reach age 19; those who remain in full-time education throughout our period of analysis are defined as Getting More Education; and those who leave education but move in and out of work are defined as Experiencing a Difficult Transition.

We first looked at how the size of these groups has changed over these successive generations. We found that the Entering the Labour Market group had declined in size over these cohorts from around 90% for those born in 1960, to under 40% among those born in 1990. This dramatic change is mainly explained by a growth in those Getting More Education, with this group increasing from roughly 5% to 50% over the thirty years. However, this is not the whole story. In addition, there has been an increase in the proportion of individuals who Experience a Difficult Transition, especially between those born in 1980 and those born in 1990.

We also explored how the composition of this group of individuals has changed between cohorts. Most strikingly, we find that both females and ethnic minorities go from being more likely to Experience a Difficult Transition to being less likely to do so.

There will be many reasons for these changes. It is probable that one is a decline in the proportion of young women who move quickly from education into an extended period of inactivity associated with starting a family. Another is that across this period the ethnic minority population of England has grown significantly in size, has diversified and has become more established; it seems plausible that these will have contributed to the relative improvement in their chances of experiencing transitions likely to be precursors of future economic prosperity.

However, some things don’t change. Family disadvantage is a strong predictor of young people experiencing a difficult transition regardless of the year they were born. This is unsurprising, but underlines that it is among those from disadvantaged backgrounds where there has been the greatest increase in difficult transitions.

This blog post draws on findings from “What young English people do once they reach school-leaving age: A cross-cohort comparison for the last 30 years” by Jake Anders and Richard Dorsett. The full research article is published in Longitudinal and Life Course Studies International Journal.

How have young people’s routes from school to work changed over the past 30 years?

This blog post was co-authored with Richard Dorsett and first appeared on the NIESR blog.

In recent years there has been growing concern about the number of young people failing to make a successful transition from education into employment. Increasingly, this appears to be a structural, rather than cyclical, problem. We see evidence of this from that fact that although youth unemployment in the UK was falling in the late 1990s and early 2000s, it started rising again as early as 2004, long before the general downturn in the economy.

This is an important issue, not least because making a successful transition from education into the labour market is important for young people’s long-term economic success; periods of unemployment during these early years may have long-term scarring effects on later employment on earnings prospects.

Today we publish research, building on work funded by the Department of Business Innovation and Skills, that considers this important question of how young people’s transitions out of compulsory education have changed as the labour market around them has evolved. Our report analyses the transitions of four cohorts of young people, those born in 1958, 1970, 1980, and 1990.

The most obvious change over this period is hardly surprisingly. The dominant type of transition has gone from being a direct transition between school and work to a transition involving at least two additional years of education beyond that which is compulsory. The former type of transition was followed by over 90% in our earliest cohort, but the size of the group has declined to under 4 out of 10 in the most recent; meanwhile those following a transition including additional education have grown from 4% to over half. This reflects the changing nature of the labour market, in which there are now fewer jobs available that do not require additional education.

However, another more worrying trend has accompanied this change. An increasing proportion of young people experience a transition that we characterise as “potential cause for concern”; the proportion seeing a transition of this type has risen from 4% in the earliest cohort to 12% in the most recent one. These transitions are marked by extended periods out of the labour market or moving in and out of work. As noted above, previous evidence warns that experiences of this type have worrying long term consequences.

Young women and those who are from non-white ethnic backgrounds have gone from being more likely to experience a transition likely to affect their long-term economic outcomes negatively than young men and those from non-white ethnic backgrounds, to being less likely over the period we analyse. By contrast, coming from an advantaged background has remained a strong predictor of avoiding a transition of this type across all four cohorts. It seems likely that the changes described stem from the higher likelihood of staying in education for young women and those from non-white ethnic backgrounds, leaving them better placed, on average, to avoid a difficult transition into the labour market.

Our research has highlighted important changes that have occurred in the labour market over the past 30 years. These are long-term trends that economic recovery will not necessarily alter. There remains a clear role for labour market policies to help smooth this transition, such as discussed in previous NIESR research on this issue.

Evaluating impact in the social sector: a practical perspective

Co-authored with Lucy Stokes (National Institute of Economic and Social Research) & Tamara Baleanu (The Access Project). This post has also been published on the NIESR, TAPNesta, & TSIP blogs.

Charities are increasingly required to demonstrate that they are achieving impact, based on robust evaluation of their work. While it is important to ensure resources are being used effectively, undertaking evaluation often poses challenges for charities, which may well have limited resources or experience with which to do this.

Funded by Nesta’s Centre for Social Action Innovation Fund, the National Institute of Economic and Social Research (NIESR) has recently been working with The Access Project (TAP), with the aim of strengthening TAP’s capacity to evaluate its own effectiveness. TAP supports young people from disadvantaged backgrounds to progress to top universities through one-to-one tutorials, aimed primarily at helping students to achieve better grades, along with multi-faceted university application support, enabling students to make informed choices. NIESR delivered guidance and tools for evaluating both of these elements. In this blog post, NIESR and TAP reflect on the project, with the aim of sharing their experiences with others embarking on similar work.

1. Why do you think impact evaluation is important in the social sector?

NIESR: Charities need to know about the impact of their activities so that they can ensure their efforts are having their intended effect. That’s not to say that impact evaluation will necessarily be able to capture all of the valuable outcomes produced as a result of a charity’s work – some outcomes may be extremely difficult or impossible to measure, and it doesn’t mean those outcomes are not important. However, just because it may be difficult to measure the impact of something, it doesn’t mean we shouldn’t try!

TAP: We really think that, as a charity, social impact is our bottom line. We exist to make a difference in the lives of the young people we work with and need to be able to evidence that difference. More generally, there’s been an increased push across the sector for evidence-based interventions. In education particularly, as a data-rich field, organisations are now expected to be able to show the effects of their work based on credible methodologies. Working with research institutions such as NIESR then becomes key; they have the expertise that small charities lack and can offer support not just with one-off evaluations but, more importantly for us, with building our own evaluation capacity.

2. What were some of the challenges you encountered during this project?

NIESR:  A key challenge in this project was thinking about what form of evaluation would be feasible for TAP to carry out themselves and sustainable long after the project finished. This had to take into account data availability, selecting methods that were relatively straightforward to implement without requiring specialist software, and not placing undue burden on TAP’s staff. Meeting this challenge required close collaboration to fully understand TAP’s needs and operation. Given that TAP is continuing to evolve, we also needed to ensure our work would be relatively adaptable to changes that TAP faces in the future.

TAP: While we’ve invested considerable resources internally on monitoring and evaluation before, this has been the first time we looked externally for expertise. A couple of potential challenges we were particularly aware of, mainly speaking a different ‘language’ to the researchers and balancing robustness with the messy reality of delivering an intervention. In practice, these challenges were seamlessly ironed out in the course of our collaboration. NIESR were very sensitive to our working realities and our ability to implement the tools they were developing. In fact, most of our conversations ended up being around finding ways to marry science with the real world in terms of everything from data availability and statistics expertise to the challenges of surveying young people in a school context.

3. What was different/special about this project?

NIESR: TAP were great to work with as they are clearly very passionate about their extremely worthwhile work. It was relatively unusual to work on a project that was all about supporting another organisation to develop their skills and capacity to do their own evaluation, rather than conducting a one-off evaluation ourselves. However, TAP were already interested in and understanding of the issues involved in evaluation, which made our task much easier.

TAP: We patently did not seek a one off evaluation. What we wanted was a set of tools we could take on and integrate in our yearly operations, both in terms of impact measurement but also programme development and learning. In the end, what we discovered was that impact measurement can also be a great tool for impact management. Instead of only looking at our results once a year, many of the conclusions NIESR’s work allowed us to draw were put to use in tweaking and improving our model.

4. What were you hoping to get out of taking part in the project? How did what you actually got out of it differ, if it did?

NIESR: As well as university access being a particular area of interest for us, a strong attraction for taking on the project was the opportunity to have a direct and immediate impact on TAP’s ability to understand the effect of their work. TAP have already told us about work they’ve done or changes they’ve made that have helped them to understand and target their work better, allowing them to be as effective as they can possibly be. That has been extremely rewarding for us.

TAP:  I think we definitely got what we sought – a robust set of tools for analysing our impact that we could realistically employ ourselves on an annual basis. What we also got (and did not seek) was a sounding board for our programme, our ideas around measurement and impact and our standards for evaluation.

5. What advice would you give to other charities/research organisation embarking on a similar journey?

NIESR: From the researchers’ perspective, it’s important to “embed” yourself in the work that the organisation is doing at an early stage. Three particularly important elements to understand are: what information and resources the organisation has (and what it might be able to obtain), what it’s able to do in order to achieve its goals, and, above all, what it cares about achieving. Without a strong understanding of these elements your work won’t be useful to the organisation you’re supporting. If resources had allowed, it would have been great to build in a longer formal mentoring component to the project to provide more hands-off support to TAP as they undertake the first couple of evaluations.

TAP: We’ve learned a lot about cross-sector collaborations in this process and we’ve definitely become believers in reaching out for where the expertise is. Being very clear on what your aims are, both in terms of the project itself but also the partnership as a whole. This involves clarity around deliverables, deadlines etc. but, equally important, around communication lines. Finally, I think one of the opportunities in working with external partners is that they bring a different perspective to the project and your work more generally. That is where the learning comes from!

Next steps

TAP: We’re in the full swing of putting NIESR’s tools to work, gathering the data and running the analyses. So far it’s been insightful, useful and, for the evaluation geeks at TAP, pretty fun as well! It would also be great to find ways to continue our collaboration with NIESR in the future; we feel we still have plenty to learn on the impact measurement front.

Jake Anders and Lucy Stokes are both researchers at NIESR. Jake’s research seeks to understand the causes and consequences of inequality in education, and to evaluate policies and programmes aiming to reduce it. Lucy’s research interests within the field of education include issues relating to inequalities in educational outcomes, the school workforce, and school effectiveness.

Tamara Balenu is the Programme Development Manager at The Access Project, working to improve the model and delivery of TAP’s university access programme. For the last two years, Tamara has been wearing two different hats at TAP, combining frontline delivery in a London school with developing Monitoring and Evaluation systems across the organisation.

How and why do young people change their expectations of going to university?

This article has also been published on the Economics of Higher Education and Institute of Education blogs.

It is now taken as a given that an ‘aspirations deficit’ is not the reason so many fewer young people from lower socio-economic backgrounds go to university. This assumption is based on the fact that at age 14 the proportion of young people from less advantaged backgrounds who think they are likely to get into university is much higher than the proportion who will ultimately end up going.

However, just as important is what happens over the next few years. Many young people’s expectations of applying to university change during their adolescent years. There are large falls in the proportion who think they are likely to do so and pre-existing inequalities in expectations grow further.

At the same time, we observe a ‘hardening’ in expectations, as individuals tend to move from saying they are ‘fairly likely’ to apply to university, to ‘very likely’; or – on the flip side – from saying they are ‘not very likely’ to reporting that they are ‘not at all likely’ to apply. Being on the wrong side of this hardening makes it far less likely that potentially qualified young people will apply to university.

Understanding what is associated with these changes may be key to sustaining high expectations, encouraging a wider cross section of university applicants, and hence widening access to university. So, what seems to explain the large changes we see?

Unsurprisingly, some change is associated with the young people’s prior attainment, as individuals with lower performance in national tests at age 11 revise their expectations downwards during their adolescent years. Interestingly, while individuals’ GCSE performance five years later, at the end of compulsory education, is also correlated with changes, there is not evidence that it suddenly causes young people to update their future plans. This is likely because GCSE results do not, for most people, come as a surprise: they are partly anticipated and so changes in expectations are smoothed over time.

This transition at the end of compulsory education is also important for another reason. Almost all the decline in expectations we observe is among those who leave education at this point, or, to a lesser extent, one year later. Among those still in education at age 18, there is no decline in average levels of expectation over the teenage years. This suggests that post-16 education is largely seen as a gateway to higher education, rather than as an aspiration in its own right.

Also of interest is whether other individual characteristics have an effect above and beyond differences in attainment. We find that parental socioeconomic status does indeed seem to have an additional impact, although it is heartening to note that it explains less of the difference in expectations than does attainment at age 11 (however, we should remember that this too is socially graded).

We also find that schools are very important for sustaining high expectations and raising low ones during this period. A tenth of the variation in maintaining expectations is explained by schools, and this rises to as much as a quarter when it comes to raising expectations.

Obviously there are many reasons why schools matter so much, not all of which will be within a school’s control (such as any association between family background and school admissions). Nevertheless, our findings suggest that teachers have a key role to play in keeping bright teenagers from less advantaged households on the track towards university.

Young people’s hopes of applying to university start out high. However, sustaining them, where appropriate, is still too heavily associated with their family background. Without breaking this link it is difficult to see how we can go on also to break the link between socioeconomic status and university attendance.


“Teenagers’ expectations of applying to university: how do they change?” by Jake Anders and John Micklewright is published as a Department of Quantitative Social Science Working Paper available from the Institute of Education, University of London website at: http://repec.ioe.ac.uk/REPEc/pdf/qsswp1313.pdf

Is the university admissions system fair?

This article also appears in the July 2012 edition of ESRC Society Now magazine.

Reforming the university application system gets a lot of attention from politicians. Initiatives such as the Office of Fair Access and suggestions that more use is made of information about young people’s family backgrounds as part of the selection process both hint at the idea that the process is fundamentally unfair and in need of changes.

Certainly, a look at the socioeconomic gap in university attendance reveals young people with higher household incomes are much more likely to go to university. New research using the Longitudinal Study of Young People in England shows that teenagers in the top fifth of the income distribution are 2.7 times more likely to attend university than those in the bottom fifth.

However, a closer look reveals problems with this analysis. If, like universities, we can look just at individuals who have applied to university, the observed gap in those who get a place shrinks significantly. Among teenagers who apply, those in the top fifth of the income distribution are only 1.2 times more likely to get a place than their peers from the bottom fifth. Importantly, if we compare applicants with similar exam results from as early as age 11 this difference becomes so small it is no long clear that it exists at all.

This is not to deny that the socioeconomic gap in university attendance does exist, but to suggest it needs to be tackled a different way. The evidence shows the gap develops earlier. As such, policies like those in the opening paragraph can only hope to work on an already small group. They are not the key to making big improvements.

Although harder to implement, bigger changes can only come from policies which work at an earlier age. Much of the observed gap in university attendance is explained by earlier exam results, for example GCSEs. This partly reflects teenagers from richer families doing better in these earlier tests. As such, an important part of the process has to be ensuring that young people from poorer backgrounds can reach their potential in such exams.

Having said that changes to the admissions process are not the key, there are, however, a couple of exceptions to this. Changes to the admissions procedures, to shift young people’s perceptions of how they’ll be treated by universities, might make pupils from poorer backgrounds more confident of a fair hearing. This might in itself help close the applications gap, and hopefully, in turn, the participation gap.

Likewise, moving to a post-results application system may also have benefits. Research by the Department of Business Innovation and Skills has shown that teenagers from worse-off families are more likely to have their A-levels grades poorly predicted. When those predicted grades play such an important part of the application process how can they then expect a fair hearing from universities? Getting lower predicted grades than they ultimately achieve may even put people off applying to university in the first place.

While there is clearly a gap between the richest and poorest families in university attendance, we must look earlier in young people’s lives to understand it fully. Some changes to the admissions process could make a difference. However, to make bigger strides towards levelling the playing field, and thereby improving one important aspect of social mobility, young people from poorer background must achieve their potential throughout their school career. It is too late to try and tackle it during the university admissions process alone.

EMA and Couples Transferrable Tax Allowance

The main argument I’ve heard deployed against EMA is that it is a waste of money because so much is paid to individuals who, when asked, say they would have stayed on anyway. In economics this is usually referred to as deadweight loss.

Actually, I don’t think increased staying on rates are the only benefit of EMA. I think it also helps improve attainment among those who receive it by, amongst other things, providing them with money to buy useful educational resources and making them less likely to feel the need to undertake more part time work than is perhaps a good idea while studying. However, that’s not the main point I wish to pursue.

Instead, the original line of argument got me thinking about how such blunt analysis would fare against other policies. In particular it made me wonder how much of the proposed Couples Transferrable Tax Allowance, planned to be worth at most £150 a year in reduced tax to couples in certain circumstances, will actually prevent couples from splitting up. My guess? Not many.