Why claims of ‘transformational’ school reading improvement are premature

The government has made some bold claims for its school reading policies – including that early results have been “transformational”. But we should be careful about rushing to judgement this early.

Following the release of a Ministry of Education report on phonics checks over the first three terms of this school year, Education Minister Erica Stanford said the results showed “a significant boost in reading success” and that “in less than a year we are reversing the decades of decline in student achievement”.

The claims are based on results from an assessment of children’s ability to decode a series of real and made-up (pseudo) words that look and sound like words but aren’t (“blork”, for example). According to the report, the results “show significant increases in student achievement”.

To understand what the data is really telling us, however, we first need to take a step back and consider whether the ability to read individual words (or pseudo words) can be considered “reading” at all.

The ‘simple view of reading’

This question is particularly significant, as it is connected to the rationale for implementing structured approaches to literacy, based on what is known as the “simple view of reading”.

The “simple view” argues that reading is a combination of decoding and language comprehension. It argues that if children are taught to lift words off the page, and they have good oral language skills, they will be able to understand and read well with practice.

The simple view of reading has itself been accused of being too simple. And we know large amounts of the variation in reading ability are not explained by the model. Yet it is still supported by those who favour the kinds of structured literacy approaches now being mandated in New Zealand schools.

Also, it has been shown that the parts of the brain that are active when people are reading individual (often pseudo) words are different to those parts activated by meaningful reading (such as an interesting story).

So, when looking at the phonics check data it would be a mistake to equate increased achievement in that specific measurement with increased achievement in actual reading.

In fact, a study in the United Kingdom showed greatly increased achievement on a similar phonics test did not result in significant improvements in later reading ability.

This is not to question the importance of decoding ability as a necessary skill for reading. But it should not be equated with reading itself. While we can celebrate an improvement in phonics results if there is one, we should be careful not to overreach when discussing its significance.

Inconclusive data

The minister highlighted the number of schools and thousands of children included in the data, and how representative of the population they are.

However, closer analysis of the report shows these are mostly not the same children; they are different children from different schools at each time point. This makes extrapolating evidence of progress difficult.

Within the thousands of children measured, there are only 516 for whom we have data at both the six-month and one-year points of their progress. We also don’t know their cultural and socioeconomic backgrounds.

We can look at the results of those children to consider change over time and whether progress is being made. But we are limited in how much we can say generally with such a small sample.

With those caveats, then, the data actually show that after six months of schooling there were 21.7% of children “exceeding expectations”. After one year this had fallen to 16.7%.

There were more children considered “proficient” in reading pseudo words after a year (22.7% compared with 18.6% after six months). But many of them had been exceeding expectations six months earlier.

The number of children meeting or exceeding expectations went from 40.3% after six months to 39.4% after one year. While this looks like a slight drop, we can’t really say that because of the small sample. We can say there was no real change.

After one year there are fewer “needing support”, which is good news. Again, with such a small sample of children from unknown backgrounds, that provides reason for cautious optimism at best.

There is also quite a bit of movement between the bands, both up and down, and not all students saw accelerated achievement in decoding ability. For the 516 children for whom we have clear data, “transformational” is perhaps not the right word.