Test scores aren't good quality indicators for schools or students

In a new research paper, prominent education researcher John Hattie suggests current education policies aren’t improving our place in world education rankings because we are appealing to what parents want rather than doing what we know works in education.

He identifies five “distractions” we tend to focus on that have little or no effect on improving education outcomes: appeasing the parents; fixing the infrastructure; fixing the students; fixing the schools; and fixing the teachers.

Rather than label these quick-fixes as distractions, it may be better to reframe the debate and examine why, and indeed if, parents are concerned about Australia’s place in the world ranking of education systems.

Education rankings: do people care about them?

Comparing education systems across the world is a complex endeavour that is currently reduced to one indicator: the performance of 15-year-old students as measured by the Programme for International Student Assessment (PISA).

The value of standardised testing such as PISA is contestable. However, given the difficulties of attracting funding for long-term evaluations of other measures of success, the relatively quick and easy test score indicators are filling the gap.

Within any educational system, linking test scores to teacher effectiveness will ultimately result in teaching to the test rather than educating for life or, as Hattie points out, focusing on “surface” learning rather than “deep” learning.

Parents expect their children will develop a wide variety of skills during their school years, conscious that the ability to regurgitate information is now the work of search engines. In this way, standardised tests where students are simply asked to recall information are relics of the last century.

The skills we require of people are changing along with technological advances, so the way we assess students should change too.

How do you measure school quality?

“School quality” has become a buzz phrase in recent years. If it is to dominate the national debate, Australians need to carefully consider how “quality” is measured.

Are NAPLAN results or ATARs true indicators of school quality? Or is it higher fees? In a market system, price is generally viewed as an indicator of quality. However, in education, paying more guarantees more status and prestige but does it guarantee better quality?

Quality depends on what you value. If test scores are all that is valued then this is an easy measure to access via the MySchool website. However, apart from the average test scores for each school, the MySchool website reports on a range of other indicators that parents may value.

The ICSEA (Index of Cultural, Socio-Educational Advantage) mean is a measure of the average level of advantage of the students attending the school. The higher the ICSEA mean, the higher the level of advantage. The ICSEA quartiles provide an indication of the concentration of students with high, or low, levels of socio-educational advantage.

Parents wanting to increase their children’s awareness of differences in levels of socio-economic advantage may select a school with a relatively equal proportion of students in each quartile.

The percentages of students from non-English-speaking backgrounds (NESB) and Indigenous students are indicators of multiculturalism within the school. Therefore, parents wanting to increase their children’s cultural awareness may select schools with relatively high proportions of NESB and/or Indigenous students.

The MySchool website also provides information on the number of students and the number of teachers, allowing parents to estimate the student-to-teacher ratio. The number of non-teaching staff is also reported. On the finance page, parents are provided with information about the amount and sources of income and the level of funding per student.

If we are to engage in a national debate about the quality of Australian schools, we should at least be clear about how we measure quality and why we need to measure it. If Australians are truly concerned with our “world ranking”, then perhaps it is timely to consider how this ranking is formulated and whether it is relevant to what we want to gain from education.

Jennifer Chesters is affiliated with The Australian Sociological Association.