Stevie Upton, author of the IWA’s report Making a Difference at Key Stage 3, responds to David Reynolds
Professor Reynolds and I are in agreement on a number of points:
- All the evidence indicates that Wales is seriously underperforming at a vital stage in pupils’ education, Key Stage 3.
- The lack of Wales-specific research on this 11-14 age group is a cause for serious concern. The IWA study is a crucial first step in building our knowledge base on what works in Wales.
- Given the “depressed and depressing” tone of the debate in the wake of Wales’ poor performance in the PISA tests, there is a need to acknowledge those instances where we are getting it right.
The report set out to provide, through in-depth case studies coupled with synthesis of the key features, a new level of information about good practice in Welsh schools. And Professor Reynolds concludes that it does just that:
“Practitioners can feel confident that they have in this study useful designs of effective practice.”
Whilst there is much on which we agree, Professor Reynolds presents a number of caveats about the research which are themselves open to question.
He first queries the fact that “the study is based upon school level data … rather than upon individual pupil data”. It is true that, by looking only at the whole-school averages, we lack the ability to determine the range of performance found within the school. Thus although we know that the school adds value to the student body as a whole, we cannot be certain how consistently that value is added. Is this a school whose pupils consistently perform close to the whole-school mean, or does the mean conceal a wide variation in attainment levels?
This notwithstanding, the data are pupil-matched, which means that the aggregated data do reflect the extent of each individual pupil’s progress. For the school as a whole to show value added, a substantial proportion of those individuals must have seen significant progression. Since the IWA has only limited resources, we had no choice but to make the compromise of working with school-level data, rather than the much higher volume of pupil-level.
Professor Reynolds’ second concern is that we cannot be certain that the five schools profiled are “Wales’ most successful schools”, not least because two of the schools did not exhibit success at Key Stage 3. To address the first point, the report does not claim to profile the “most successful” schools in the country – indeed, it’s doubtful that one could objectively define such a thing. Instead, it seeks to identify schools at different stages of development that exhibit good practice at Key Stage 3.
In this context, it is too simplistic to say that simply because a school is not adding statistically significant contextual value added at KS3 it is not ‘successful’. On a statistical note, for instance, the Welsh Government itself recognises that traditionally high-performing schools can find it hard to add value – the best they can hope is to perform as expected. And to take the Cwmtawe Community School example, the school produces excellent results at Key Stage 4, the culmination of a child’s compulsory education. In this case Key Stage 3 can be seen as part of a progression towards the achievement of GCSE success.
Professor Reynolds thirdly questions the use of Estyn judgements as the basis for analysis of the key elements of good practice. Crucially, however, these judgements were used only to identify general themes as a framework for discussions with the case study schools.
Findings from the inspection reports of thirty schools were amalgamated in the creation of an interview schedule, and thus the Estyn reports for the five case study schools were not directly used to inform assessment of the factors considered key to their success.
The fourth concern raised by Professor Reynolds is that the report “emphasises how in fact the schools are not really all that similar”. Certainly differences between the schools are acknowledged, but the report does not “emphasise” them. In fact, Professor Reynolds himself goes on to note that similarities “are repeatedly mentioned”. It is uncovering these similarities that prompted the research, and they therefore form the structure for the analysis of the schools’ approaches.
Nevertheless, it is important to note differences in how the key factors are implemented. This is simply to acknowledge that teachers do not (and should not be expected to) implement initiatives without regard to context. It should not detract from the importance of the common factors identified across the schools.
The significance of context is crucial. Professor Reynolds reproaches me for falling victim to the “Welsh curse” of putting teacher freedom above pupil outcomes. Yet we must equally avoid another very British curse, that of de-professionalising teaching through the imposition of one strategy after another (however well-meaning).
That said, there is undoubtedly a need in some Welsh schools for substantial external input as a means to improving both standards and consistency. As Making a Difference at Key Stage 3 concludes:
“Some schools are already operating to a high standard. They undertake monitoring and evaluation that is suitable to their particular circumstances, and should be given the freedom to continue doing so. Others will need to earn that degree of autonomy. Schools which are not yet deploying and acting upon self-evaluation procedures with sufficient rigour will require external input. This does not mean that they should be required to renounce all control over the monitoring of school and pupil progression. The emphasis must be on building internal capacity, to enable ongoing development and allow the school to earn autonomy in the medium to long term.”
Professor Reynolds and I share, therefore, the view that local freedom to innovate should be introduced only once schools have demonstrated the capacity to use it effectively. Where we differ is over whether the same strictures should be applied concurrently across the whole of the Welsh educational system. The gap in performance between our best and worst performing schools suggests not.
For those schools which require additional support, one critical question remains that only time will answer. Having intervened, will the government or its agents be willing and able to release control?
2 thoughts on “Combating Welsh and British ‘curses’ in education”
The lack of Wales-specific research is not unique to education, it is an issue across numerous sectors and needs addressing urgently. So many bodies and organisations who operate on a pan-UK level, love creating England-and-Wales stats, which tell us the next thing to nothing and should be a thing of the past in today’s devolved Wales.
There should be a warning issued to anybody instigating educational research – you do so at your peril. You are very likely to end up with egg on your face. There should be a greater warning for the schools upon which the research is based.
To hold up schools to be emulated is a very risky thing to do and you have to be sure of your facts. The recently published data on school’s value added performance ranks the IWA’s chosen five schools as 51st, 92nd, 95th, 104th and 133rd out of 222 schools. This wouldn’t appear to be evidence of the most appropriate schools to be copying.
It can be rightly argued that this ranking is too simplistic. However, the same argument could be levelled at the IWA’s criteria for choosing its star schools.
However the whole premise of the research looks to be on even shakier ground. Recent research by the BBC into GCSE results found that the use at home of computers had an effect of boosting results by a grade. Teacher influence was described as making ‘no significant difference’.
Report writers, policy inventors and politicians need to realise that education is a very complicated field. There is no quick fix and merely telling teachers to do things differently will not have the desired effect. A realisation that schooling does not have the monopoly on educational attainment would be a start. From there you could start to think of other areas that will have an educational impact and try to focus research on them.
Comments are closed.