how has it come to this in Western democracies?

The Liberal leadership spill and Malcolm Turnbull’s downfall is but the latest instalment in a game of musical chairs that has dominated Australian politics for the best part of a decade.

For many, it has been enough to portray Tony Abbott as the villain of the story. Others have pointed to Peter Dutton and his allies as willing, though not-so-clever, accomplices. There’s also been a highlighting of the herd instinct: once self-serving mutiny gathers steam, others will want to follow.

But this barely scratches the surface. And the trend is not confined to Australia.

Read more: Dutton v Turnbull is the latest manifestation of the splintering of the centre-right in Australian politics

We need only think of Donald Trump’s America, Britain’s Brexit saga or the rise of far-right populist movements in Europe. Politics in the West seem uneasily suspended between farce and tragedy, as deception, accusations of “fake news” and infighting have become commonplace.

In Australia, the revolving prime ministerial door has had much to do with deep tensions surrounding climate change and energy policy more generally.

In Britain, a longstanding ambivalence towards European integration has deeply divided mainstream parties and plunged the country into “Brexit chaos”, a protracted crisis greatly exacerbated by government incompetence and political expediency.

In Italy, the steady erosion of support for the establishment parties has paved the way for a governing coalition that includes a far-right party committed to cracking down on “illegal”, specifically Muslim, immigration.

Yet, beyond these differences are certain common, cross-cultural threads which help explain the present Western malaise.

Simply put, we now have a glaring and widening gap between the enormity of the challenges facing Western societies and the capacity of their political institutions to address them.

Neoliberalism at work

The political class in Australia, as in Europe and North America, is operating within an institutional framework that is compromised by two powerful forces: the dominance of the neoliberal order and relentless globalisation.

The interplay of these two forces goes a long way towards explaining the failure of political elites. They offer neither a compelling national narrative nor a coherent program for the future. Instead, the public is treated to a series of sideshows and constant rivalries over the spoils of office.

Read more: Partially right: rejecting neoliberalism shouldn't mean giving up on social liberalism

How does the neoliberal creed underpin the state of current political discourse and practice? The shorthand answer is by setting economic growth as the overriding national objective . Such growth, we are told, requires the public sector to be squeezed and the private sector to be given free reign.

And when economic performance falls short of the mark, pressing social and environmental needs are unmet, or a global financial crisis exposes large-scale financial crimes and shoddy lending practices, these are simply dismissed as inconvenient truths.

Compounding the impact of this highly restrictive economic agenda is globalisation or, to be more accurate, the phenomenal growth of cross-border flows of goods and services, capital, money, carbon emissions, technical know-how, arms, information, images and people. The sheer scale, speed and intensity of these flows make them impervious to national control.

Read more: It's not just the economy, stupid; it's whether the economy is fair

But governments and political parties want to maintain the pretence they can stem the tide. To admit they cannot is to run the risk of appearing incompetent or irrelevant. Importantly, they risk losing the financial or political support of powerful interests that benefit from globalisation, such as the coal lobby.

And so, deception and self-deception become the only viable option. So it is that several US presidents, including Trump, and large segments of the US Congress have flagrantly contradicted climate science or downplayed its implications.

Much the same can be said of Australia. When confronted with climate sceptics in the Liberal ranks, the Turnbull government chose to prioritise lowering electricity prices while minimising its commitment to carbon emission reductions.

The erosion of truth and trust

In the face of such evasion and disinformation, large segments of the population, especially those who are experiencing hard times or feel alienated, provide fertile ground for populist slogans and the personalities willing to mouth them.

Each country has its distinctive history and political culture. But everywhere we see the same refusal to face up to harsh realities. Some will deny the science of climate change. Others will want to roll back the unprecedented movements of people seeking refuge from war, discrimination or abject poverty.

Others still will pretend the state can regulate the accelerating use of information technology, even though the technology is already being used to threaten people’s privacy and reduce control over personal data. Both the state and corporate sector are subjecting citizens to unprecedented levels of surveillance.

Read more: The Turnbull government is all but finished, and the Liberals will now need to work out who they are

Lies, “fake news” and cover-ups are not, of course, the preserve of politicians. They have become commonplace in so many of our institutions.

The extraordinary revelations from the Banking Royal Commission make clear that Australia’s largest banks and other financial enterprises have massively defrauded customers, given short shrift to both the law and regulators and consistently disregarded the truth.

And now, as a result of another Royal Commission, we have a belated appreciation of the rampant sexual abuse of children in the Catholic Church, which has been consistently covered up by religious officials.

These various public and private arenas, where truth is regularly concealed, denied or obscured, have had a profoundly corrosive effect on the fabric of society, and inevitably on the public sphere. They have severely diminished the social trust on which the viability of democratic processes vitally depends.

There is no simple remedy to the current political disarray. The powerful forces driving financial flows and production and communication technologies are reshaping culture, the global economy and policy-making processes in deeply troubling ways.

Truth and trust are now in short supply. Yet, they are indispensable to democratic processes and institutions.

A sustained national and international conversation on ways to redeem truth and trust has become one of the defining imperatives of our time.

Joseph Camilleri will speak more on this topic in three interactive public lectures entitled Brave New World at St Michael’s on Collins in Melbourne on Sept. 11, 18 and 25.