Brenig Davies argues improvement in PISA will come when education improvement is coterminous with improvements in the social and economic conditions across Wales
…So what? Bottom of the UK and middle of the OECD PISA ranking. Are our children of Wales less happy than other parts of the UK? Are they less creative. Yes, they should be higher up the English, mathematics and science PISA leagues. In the affluent areas the results, are without doubt, better than those in the poor areas of Wales. If potential could be more reliably measured, I suspect, the Wales profile would reveal more about the socio-economic conditions bearing down on pupils achievements than the pupils innate abilities. Though aspirations and the expectations in the home and those of friends are amongst the critical influences on post school realistic choices. It is worth remembering that the usefulness of PISA is its assessment of a country’s education system, albeit based on the achievement of pupils in participating schools. It is not a measurement of a pupil’s future achievement per se. Though, of course, it is reasonable to draw certain tentative conclusions of a pupil’s school attainment apropos future career prospects, whatever they may be. Though PISA results suggest a link between ranking and the needs of a modern economy in world wide competitive markets.
The recent Assembly ‘hwb’ initiative designed to prepare high achieving children across Wales with a more realistic chance of being successful in the entrance process of Oxford and Cambridge, provides one model for overcoming or compensating for a ‘shortage’ of an educational achievement culture of a community, economy or home, and hence future career prospects. As it is entering higher education is much more than Oxbridge.
Improvements in the quality and scope of apprenticeship programmes offer great opportunities for pupils who choose this route to higher education and well paid and personally rewarding careers. Apprenticeships at level 3 and 4 within the framework of indentured apprenticeships will lead to an increased status of vocational education and training. There is a caveat though: the apprenticeship should be serious and lead to high competence and mastery in the respective occupation. Anything less is an apprenticeship in name alone, and certainly is absent from an indentured apprenticeship (an indentured apprenticeship is one that has been designed by a respected industry body or company, and involves an employer, apprentice and industry body in a shared contract for the apprentice to be successfully inducted into an occupation ending with the mastery of relevant skills and knowledge).
It is not complacent to acknowledge that there is little travel-work mobility in some of our communities. This is understandable if the achievement norm of the community is limited; or expressing the desirability of high achievement, and being clever, leads to being ostracised by one’s friends.
Many Welsh Assembly departments, working together to help pupils gain apprenticeships, outside their immediate community, but still within a reasonable travel-to-work area, might gradually create a growing critical mass of young people with the confidence to challenge and avail themselves to opportunities that will surprise themselves, their peers and parents, and even many of their former teachers and careers advisors.
PISA is a useful measure of where Welsh pupils are in relation to pupils in other OECD countries. The cultural expectations, styles of learning and societal expectations are those that cannot and arguably should not be emulated in the UK. from high ranking PISA countries. The concern for Wales is its poor performance in relation to the three other countries in the UK.
Seeking ways of improving Wales’ PISA ranking will not come from expecting the heavy lifting to be done solely by the Assembly’s education department (It is interesting to note that on the day the PISA results were published the Assembly pushed out the Secretary of State for Education to answer questions put to her by the media). Neither will it come from too much reliance on education systems closer to home, such as the oft quoted Finnish system, or the Swedish model(!) apropos the Foundation programme. Much is expected of lessons learned in Scotland through The Donaldson Report and subsequent curriculum change.
It is ironic that the greatest qualification weakness in ‘modern’ times is the home grown Welsh Baccalaureate; a weakness that must not be mentioned in polite society. Only recently, in a newly built secondary school, there was a designated room with a notice on the door: ‘Baccalaureate room’! So much for an all embracing integrated curriculum framework. Providing incentives to encourage excellent teachers to move to the poor areas of Wales, is likely to produce higher results than moving teaching and curriculum practices from another country to the poor parts of Wales.
Simple solutions and political targets help little. Who remembers the name of the Welsh education minister who said four years ago that the next set of PISA results should see Wales in the top quartile?
An appropriate curriculum, mediated (not copied) and taught by excellent teachers, in good surroundings with modern teaching technologies, offers a realistic chance of change and improvement in pupils’ achievements. Teacher training institutes providing new teachers with the necessary skills to raise pupils’ achievement across the board, and a rewarding pay structure to attract those who will become excellent teachers, seems a reasonable place to add to reviews, when planning for the future.
An improvement in PISA will come when education improvement is coterminous with improvements in the social and economic conditions across Wales. Pupils in terraced houses should expect the same quality of education to those living in detached houses.
While there is little new in this essay, it is always timely to review and discuss issues that may improve pupils achievement in Wales, following PISA results.
I think we may be attaching too much importance to the PISA system, as Dr Victoria Winckler has put so succinctly lately. They have become a political bashing excuse, because of the (erroneous) comparison question. They can be critisised on technical grounds, including the lack of transparency on statistical methods, and they don’t take into account different cultures and society imperatives. They don’t measure the attributes which make a balances student and future member of society – creative abilities, happiness, well-being – citizenship – things which apparently even the Singaporean Ministry of Education recognises and is seeking to rectify. So, yes, we need to do better in core subjects – and stop up to 50 wishing to enter further education first needing further dedicated teaching in these to be able to even contribute and understand! – and we need to give vocational studies as much worth in society as the traditional academic and professional studies. But, please, let’s celebrate what is good about our education system and the excellence they produce in research, and not reduce the whole argument to a scoring game between politicians who often don’t know or don’t care enough about what is involved, and the future of our Country beyond their next election….
I read that the Welsh performance was particularly and relatively weak amoung the most able and least blighted by social economic disadvantage. Have I been misinformed?
Oh dear. Here we go; someone who hasn’t read the Wales PISA analysis report. Our economically poorer pupils actually did relatively well at PISA
Not eligible for FSM Eligible for FSM
Science 489 446
Mathematics 483 440
Reading 482 441
Our better off pupils did disastrously.
Figure 6.4 Average PISA 2015 science scores in Wales by national quartiles of the Economic, Social and Cultural status (ESCS) index
England Wales OECD
Poorest 25% 475 464 452
Second Quartile 492 474 481
Third Quartile 529 498 505
Richest 25% 561 517 540
In Wales our top 10%, our potential “Oxbridge entrants”, scored way below the top 10% of most other countries. Our Welsh medium schools, 85% of which have fewer than 15% of pupils eligible for free school meals, did worst of all and 15 year old pupils who took the reading test through the medium of Welsh were almost a year behind those pupils who took the test through the medium of English despite the fact that pupils in WM schools are far more likely to come from an economically advantaged home and far more likely to have a parent educated to degree level (48% against 36% for pupils in EM schools).
We can decide whether we want a very powerful “Welsh in Education department” or a powerful School improvement policy but we can’t have both.
PISA is for 15 year olds when they are just startting GCSEs and measures what education has gone before so your points on apprenticeships and post 16 education per se are a a bit of a red herring.
In this article http://www.bbc.co.uk/news/uk-wales-38208738 you will read
“The [PISA] report also notes that only 6% of the variation in how Welsh students are performing can be put down to their socio-economic background – in the rest of the UK, 11% of the variation can be explained by it.”
so banging the the disadvantage drum too hard is not appropriate.
One of the key points from the report, also made here http://www.bbc.co.uk/news/uk-wales-politics-38225951, was that those at the bottom of the heap are not served any worse in Wales than elsewhere in the UK. What we are failing to do is stretch enough of the more able.
Huw Lewis…….we will remember them.
Thank you Brian. Always better to read a report before you write an article on it…dont you think.
Jon Owen Jones
Thank you for your comments. They are helpful and allow me to think more clearly about the essay. I hope it has not led you to feeling that you have been misinformed, too much. I make the following comments that are intended to add to the discussion.
1) To quote from the essay: ‘If potential could be more reliably measured, I suspect, the Wales profile would reveal more about the socio-economic conditions bearing down on pupils achievements than the pupils innate abilities.’ The emphasis is on ‘… could be more reliably measured… ‘ In the absence of reliable measures, several proxy measures are used to form a judgement on the influence of the local environment on pupils’ performance. One significant measure is that used to attract EU Structural Funds in West, North Wales and the Valleys, and East Wales. These funds will, amongst other projects, be used for ‘Support people into work and training, youth employment… i.e. Post PISA young people.
2) And while … ‘performance… (of pupils) were least blighted by social economic disadvantage’… the geographic spread of EU Structural Funds indicates that there are more young people in the Structural Funds local authorities than there are ‘among the most able’, outside EU Structural Fund local authorities So while the PISA conclusions (you imply) do offer useful guidance for planning education, the proxy indicator of Structural Funds reinforces the view that many more young people will continue to benefit from ESF project funding. I offer the view, ( though without hard evidence) that one reason why pupils from socio-econimically deprived areas (largly Structural Fund areas) have performed comparatively well in PISA, is because of the success of several previous rounds of ESF, in addressing socio-economic needs of communities, especially those with problematical travel to work areas (a point I mention in the fourth paragraph).This assertion is in the heading of the essay ‘… argues improvement in PISA will come when education improvement is coterminous with improvements in the social and economic conditions across Wales.’
There was an interesting experiment done in America some time ago Brenig. It started from the well evidenced position that people low on the scale of socio-economic status had children who performed poorly at school. The experiment was simplistic; they selected a number of disadvantaged families and dramatically increased their finances. The result was much as you imagine…flash car…more drinking…super holiday…
Altogether the stupidest experiment ever and probably designed by someone who wanted it to fail spectacularly.
One oddity that is rarely picked up on is what the performance of NON-EFSM pupils is in schools if they are grouped according to the 3 year average EFSM percentage of those schools. So, in 2016 I looked at the year 6 higher (level 5) achievement of non-EFSM pupils from non Welsh speaking homes in Welsh medium schools and compared it to the same groups (non-EFSM) at varying school EFSM percentages.
The WM pupils underperformed in every comparison except where the sample was tiny (there are few WM schools with high EFSM percentages). But even the non-EFSM pupils in the EM schools had progressively worse scores as the SCHOOL EFSM percentage increased.
In an ideal world every school would have the same percentage of pupils entitled to free school meals…this would raise performance across the board so, perversely, its the ability to choose schools…pupil mobility, that is the enemy of school improvement.
Reply to J Jones
Thank you for drawing my attention to relevant statistics in commentary (1), which I fully accept. and the research summary in (2). It is all helpful to planning the future of education in Wales. You are quite right that I drafted the essay on media reports and not on the OECD report. It was the headline PISA results that prompted me to write the essay.
You will note that most of the essay is concerned with opportunities and/or inhibiting factors for post PISA young people. The heading of the essay indicates its main assertion; that being the assumed relationship between improvements in the quality of an educational system with improvements in the economy, with increasing career prospects.
Working on published statistics, and deducing the number of pupils in the PISA cohort in Wales, there are approximately
52, 000 pupils, of which 350 were sampled. Taking the 6% who were influenced by poor economic circumstances ( as mentioned in the PISA report) gives a figure of 3,120 pupils, who might be inhibited from doing their best in school. It is this significant number of pupils ( which I calculated in considering your comments) and perhaps more – not to mention others in poor households – which attempts to underpin my argument about the post PISA socio-economic factors bearing down on some pupils. I assume that such factors strongly influence the Welsh government for giving priority to poor communities, along with addressing travel to work issues, and supporting initiatives for increased employment.
Thank you for contributing to the debate on education and related careers prospects of those young people in Wales.
And above all we in Wales must avoid this:-
Page 120 of the UCL Welsh Pisa analysis:-
37. The key message to be taken from Figure 6.7 is that differences in these
background characteristics cannot explain the achievement gap between pupils who
completed the test in English versus Welsh. In fact, differences in average PISA
science and reading scores between pupils who completed the test in English and
Welsh actually increase somewhat (and reach statistical significance at the five per
cent level) after such background characteristics have been taken into account. For
instance, the difference between pupils who took the PISA test in English versus
Welsh stands at 25 test points in science (significant at the five per cent level), 30
points in reading (significant at the five per cent level) and 8 points in mathematics
(insignificant at the five per cent level) after these characteristics have been taken
into account. This further strengthens the evidence that pupils who took the PISA
test in Welsh achieve lower average scores than those who completed the test in
English (at least in the science and reading domains).
So, to put that in perspective, pupils taking the test through Welsh, were nearly one year behind those taking the test in English in Wales in Science and a whole year behind in reading. And those taking the test in English were far behind England let alone the leading European nations.
J. Jones; that is a very interesting quotation. I asked you some time ago if there was any evidence of the relative performance of pupils taking science through the medium of Welsh and English. You replied to the effect that there was insufficient data. I ask since when I attended Welsh Medium education, science and maths were taught in English and i believe this remains the case in one or two schools. As a former science teacher and a father of three children who were all taught science in welsh I have pondered this question for some time. Although we may describe the language medium as english it is in effect a very specialised, technical and largely international language mostly based on latinised terms. Pupils taking science at A level have to acquire this vocabulary and if they study in Welsh they really need to know the terminology in both languages. I cannot see how this is not a disadvantage.
By now the memory of our, once again, disappointing PISA results will be fading and several apologists, notably those academics and supporters of curriculum change in Wales and Scotland, will be publishing articles which undermine the PISA results.
Before we settle in to wait another three years for the changes to our school system to be validated by improved PISA scores it is worth asking one more question; “Have we been fooling ourselves?”.
If you asked this question of Michael Gove 5 years ago he would have said unequivocally “YES”. He initiated major change to GCSE grading, most notably by removing “softer” equivalent options and removing grade inflation as far as possible. He also incentivised the taking of more academic subjects and dis-incentivised early entry for GCSE.
In Wales we did not follow this lead and, despite the fact that for at least two years we could no longer compare ourselves with England on GCSE results, we have nevertheless congratulated ourselves on closing the gap with our neighbours.
What I would ask is this; if our PISA results have worsened since 2006 why have our GCSE results improved so markedly?
If we take reading for instance and use English or Welsh GCSE A*-C pass percentage as a proxy for comparison purposes what can we tell?
In 2009 we scored 476 on the PISA scale and our GCSE A*-C pass percentage was 61% in English and 75% in Welsh L1
In 2015 we scored 477 on the PISA scale and GCSE pass percentage in 2015 was 68.6% in English and 75% in Welsh L1
We also know that pupils who took the reading test in English in PISA 2015 scored a higher 480 and those taking it in Welsh, a lower 455.
So PISA remains unchanged for all pupils (this was a massive sample and therefore reliable) whilst GCSE improves by nearly 8% in English and remains 6% higher in Welsh that the equivalent GCSE in English.
So pupils who took the reading test in Welsh, and 75% of whom scored A*-C at GCSE, were in fact 25 points (nearly a year) behind pupils taking the test in English.
This will come as no surprise to senior managers in English medium schools (nor pupils who take both English and Welsh L1 GCSE exams). Welsh GCSE is a “doddle” compared to English GCSE and has been consistently dumbed down to accommodate the increasing percentage of pupils from English only homes who are in WM schools.
Even more striking than reading is the GCSE/PISA comparison for Science.
Since 2009 GCSE A*-C pass percentage has gone from 61.8% to 84%. On paper an improvement of 22%. However, since I do scatter graphs for all school’s attainment every year what I can tell you is that GCSE Science is atypical in pattern…whereas free school meals entitlement is very predictive for Maths and English GCSE pass percentage, the same does not hold true for science. In Science the schools which do best in Maths and Science such as Cardiff High, significantly under-perform some schools with 30% efsm, in Science. The reason is BTEC, which is “assessed” mostly within the school itself, as opposed to single sciences which are an external GCSE examination. The more able pupils in high achieving schools take single Sciences for the most part.
In Science at PISA of course Wales’ trajectory has been wholly the other way; 496 in 2009 down to 485 in 2015 and now a full 20 points (or 9 school months) between the performance of pupils in 2006 and 2015.
Never mind. Our short sighted, short term-ist, ideologically obsessed Welsh Assembly is culturally incapable of taking hard decisions. They cannot and will not look outside their “Bay comfort zone”, at the REAL world of PISA and, shamefully our tame academics can’t do it either. Politicians are what they are but academics (and the press and media) in the education field in Wales are an inexcusable disgrace.
To J Jones
I must correct an error revealed in reviewing my previous reply; the number is closer to 3500 (3461) The error was dropping an ‘0’ off 350, when rounding up. There is not a significant change on my 6%. comment.
As a governor of a Welsh medium primary school, it helps to be aware of your comment, that ‘… pupils taking the test through Welsh, were nearly one year behind those taking the test in English in Wales in Science and a whole year behind in reading.’
Brenig; the 2015 cohort of 15 year old pupils in Wales was 34,004 and so a 10% sample was a huge, and therefore reliable sample. It is still a representative sample though and a look at the actual cohort that it represents shows that the percentage of pupils who were eligible for free school meals was 17.4%. That is overall but 9.8% amongst Welsh medium pupils.
One thing that is noticeable about PISA is that it isn’t as sensitive to socio-economic status as GCSE results and you will have seen this from the analysis.
As I have said, improvement in the economic wellbeing of areas in Wales (an obvious good in itself) is not an answer to educational under achievement. That requires a change in the approach to teaching. Teachers have to take FULL responsibility for schooling and the three obvious shortcomings highlighted by PISA need to be dealt with; teacher absenteeism (both missing from school and employed elsewhere in school rather than in class), teachers failing to prepare adequate lessons and lax classroom discipline.
Jon Owen Jones; yes I remember you asking about the language of instruction in Science and at the time I couldn’t answer. Since then there has been some in depth research done into the language medium of education and there is some data :-
http://gov.wales/docs/statistics/2016/160301-welsh-language-use-in-wales-2013-15-en.pdf
Page 65. There were some good tables in the National Survey of Wales but unfortunately I can’t locate them at the moment.
Hope that helps answer your question.
J. Jones sorry to say the tables you refer me to do not answer my question. They tell me how many and what percentage and where pupils study Science through the medium of welsh. They are silent on whether this has a neutral effect on the pupils understanding of the subject or an advantage or disadvantage.Are there no statistics which compare the outcomes in terms of examination grades or uptake at A level which compare similar Welsh and non Welsh language schools? Better than that would be comparison of results in Welsh medium schools which continued to teach Science in English.
No Jon but I would suggest that you make the logical association that pupils who actually took PISA through the medium of Welsh were taught Science through the medium of Welsh and then, by age 15, were nearly a year behind those pupils who were taught through the medium of English.
I read an article today which said that only 80 students enrolled as science teachers in Wales last year and, when you consider how few will actually be Welsh medium trained, you can see chickens coming home to roost. You can see that in 2008 our weakness in Science teaching was already being recognised:-
http://www.walesonline.co.uk/news/local-news/damning-verdict-school-science-teaching-2167726
So poor outcomes in Science in 2008…a dearth of science teachers in 2016. Part of the blame lies with the pressure put on schools to achieve the CSI (Core Subject Indicator) which is the percentage of GCSE pupils who gain an A*-C in English/Welsh, Maths and Science. Small schools and particularly rural WM schools enter pupils for BTEC because it is largely assessed within the school, but it is a “soft option” in comparison with separate Sciences.
In the three years to 2015 I saw 5 schools score 100% in GCSE A*-C science. Of those 5, three were in Gwynedd.
The problem is that BTEC does not give the basis for A level separate sciences and we now reap the rewards in poor PISA Science scores and an absence of qualified teachers.
In 2008 they were pointing to good outcomes in Science at Key Stage 2, the end of primary schooling. This was wilful blindness. Primary school assessments had never been adequately moderated in Wales once SATs were dropped. The results looked good but they were often fictional.