Are there ‘male’ and ‘female’ brains? Computers can see a distinction, but they rely strongly on differences in head size

How useful are the well-known and hotly contested categories of “male brain” and “female brain”?

Among experts, nobody really questions that anatomical sex differences in the brain exist. But since the advent of brain science, the scientific community has been divided over how many differences there are, which ones have been definitively proven, how large or small they are, and what they actually mean.

And, over the past several years, a new debate has been brewing among experts. Do anatomical differences in the brain “add up” to two clearly recognisable (sex-specific) brain types? Or do they rather “mix up” and form idiosyncratic combinations or “mosaics”, independent of sex?

Read more: Brain scientists haven't been able to find major differences between women's and men's brains, despite over a century of searching

A mosaic of male and female features

The mosaic hypothesis was supported by the results of a ground-breaking study published in 2015 by Daphna Joel and her collaborators at Tel-Aviv University.

Using brain scans of more than 1,400 participants, Joel and company identified the 10 regions showing the largest differences in size between men and women. Next, they classified each region of each brain as “male-typical”, “female-typical” or “intermediate”.

Most of the brains turned out to be “mosaics” of male-typical and female-typical features, rather than being consistently male-typical (“male brains”) or female-typical (“female brains”). Joel concluded that brains “cannot be categorised into two distinct classes: male brain/female brain”.

Read more: Medicine's gender revolution: how women stopped being treated as 'small men'

Algorithms can ‘predict’ sex from brain data

Critics of the mosaic brain theory, however, point to machine-learning algorithms that can use a brain scan to “predict” an individual’s sex with 80 to 90 percent accuracy.

If an algorithm can classify brains into sexes so easily, the argument goes, it must be recognising some underlying difference.

To some extent, this is a disagreement about what the terms “male brains” and “female brains” should entail. For Joel, using these categories would only be justified if, for example, knowing somebody had a “female” or “male” brain allowed you to predict other things about their brain’s features.

But for Joel’s critics, the important thing is predicting the individual’s sex. It doesn’t matter whether or not slotting somebody’s brain into a sex category gives you more information about its structure.

Most machine-learning classification algorithms are “black boxes”, which means they don’t reveal anything about how they combine brain features to define “male” and “female” brains. Despite the accuracy of the algorithms, their definitions may not even be consistent: some evidence suggests the algorithms use different brain features when classifying different subpopulations of females and males.

Algorithms’ sex prediction may depend on head size

And now even this classification accuracy is under challenge. A research team led by one of us (Carla Sanchis Segura) published a new study that considers a neglected complication. On average, women have smaller bodies, heads and brains than men.

In the early days of brain science, these differences in body and brain were mistakenly taken as evidence of (white) men’s intellectual superiority. But in recent years, it has been recognised that head size variation poses a problem for neuroscientists interested in sex differences.

When you see a female/male difference in the size of a brain region, how do you know if you are seeing a specific effect of sex? It might simply be a difference between larger brains (more of which belong to males) and smaller brains (more of which belong to females), or a combination of the two.

Neuroscientists try to solve this problem by statistically “controlling” for head size. But exactly how is this done?

There are several different statistical methods in use. The current “gold standard” for assessing their validity is comparing the sex differences in the brain they find with those obtained in selected groups of females and males matched to have similar head sizes.

Read more: How we inherit masculine and feminine behaviours: a new idea about environment and genes

Using this “gold standard”, the Sanchis-Segura research team found, in an earlier study, that not all currently used methods are effective and valid. They also found that the method used has a major impact on the number, the size and even the direction of the estimated sex differences.

Having worked out which statistical control techniques are the most valid, Sanchis-Segura and her team were able to investigate an important question: to what extent does the high accuracy of “brain sex” classification depend on head size variation?

The researchers tested 12 different sex-predicting machine-learning algorithms with data that had been properly adjusted for head size variation, data that had been poorly adjusted, and data that had not been adjusted at all.

The algorithms delivered highly accurate results when using both raw data and poorly adjusted data. But when the same 12 algorithms were fed with properly adjusted data, classification accuracy dropped to 10% above ‘chance’, at about 60% accuracy.

One particularly deflationary finding of the study was that the algorithms achieved high accuracy if they were given just one piece of information – namely, head size!

These new findings continue to challenge the usefulfness of the categories “male brain” and “female brain”. Sex certainly affects the brain, and sex effects are important to study. But current attempts to classify brains into the categories “male brain” or “female brain” using machine-learning algorithm seem to add little beyond what has been known since the inception of modern science – that men, on average, have larger heads.