How to move energy policy models beyond bias and vested interests

The Turnbull government’s flagship energy plan, the National Energy Guarantee, was intended to end a decade-long stalemate on energy and climate policy in Australia.

Ironically, since its unveiling in October 2017, the debate has heightened considerably, with the result that the government has now walked away from the emissions-reduction component of the policy.

Read more: Malcolm Turnbull shelves emissions reduction target as leadership speculation mounts

Plenty of attention has gone to the high political drama – and fundamental conflicts over the importance of emissions reductions. But another key issue is the lack of trust in government models predicting the results of their policies.

For instance, the government claimed this month that the NEG will reduce household bills by A$150 per year. Independent analysts, as well as Labor and Greens politicians, have questioned this figure. They point out that other models suggest different results – especially one announced by federal energy minister Josh Frydenberg in October 2017, which predicted a A$100 reduction. All these groups have called for the full release of the government’s modelling work.

Read more: The National Energy Guarantee is a flagship policy. So why hasn't the modelling been made public?

But if modelling is a form of scientific analysis, why do different models give such different results?

What is a model?

A model is a simplified representation of reality, but that “reality” is defined by the modeller. We give a model a set of inputs and it produces a set of outputs.

The modelling process involves a sequence of “choices” which the modeller makes about the methods to use, the input data to feed in, and the relationships between these data (that is, what impacts what).

By giving some factors more weight – whether deliberately or unintentionally – the modeller can make one outcome look more appealing, likely or important than the others.

Read more: Explainer: why we shouldn't be so quick to trust energy modelling

Imagine asking 100 cooks from different countries to make the best noodle soup in the world. They would all chose different ingredients, types of noodles, and ways of cooking.

These choices would reflect the recipes they already know, the tastes they personally like or dislike, and the ingredients with which they are familiar. These form their biases with regard to what a good noodle soup should be. You would not be surprised if you see 100 very different noodle soups at the end of this competition!

Like noodle soup, policy models are also made with a variety of ingredients, which are shaped by the choices and biases of their modellers and stakeholders. The cumulative effect of these choices creates different models, and therefore different results.

That’s why some modellers and analysts now argue that no model is “the” right model, just as no noodle soup is the right noodle soup, and that no single model would result in an “established truthfulness”.

So, how can we design policies using models that are filled with biases and vested interests?

Exploration, not prediction

Here is our answer: we should not consider models tools for “prediction”, but rather for “exploration”. We should not expect models to give us “the” answer to our policy questions. We need models to explore a range of scenarios to inform policy discussions.

Let’s use the example of reducing greenhouse gas emissions. There are many ways to do this. We can transform our electricity generation systems to boost the amount of renewables; we can improve building efficiencies; we can use cleaner means of transport.

Each pathway has its opponents and proponents. They might argue over their benefits, their consequences, and how much investment each deserves from a finite pool of money.

Read more: Inaccurate energy forecasts are costing us the Earth: here's why

In the conventional predictive approach, we would model each policy option (or a combination of options) and assess its impact on emissions. (And probably each side would undertake their own modelling, with their own implicit assumptions.)

But in an exploratory approach, we treat the model as something to play with, to “test” policy options. We change assumptions underlying the model and see how the results change. We change future scenarios and run numerous scenarios and see how policy options perform under different scenarios. And in the end of this playful exercise, there is no single answer! Each result depends on the assumptions and the scenarios from which it was produced, and – crucially – these assumptions are all documented and made transparent.

We used this approach to investigate India’s transition to clean energy. They, like Australia, are dealing with highly complicated political and social issues that do not fold neatly into conventional ways of modelling, which strive to give a single answer.

We certainly do not suggest that exploratory modelling is a silver bullet for resolving political differences on complex policy issues.

It can, however, transform our understanding of models from a “blackbox” process to a transparent process open to scrutiny. It can turn implicit assumptions into explicit scenarios which can be tested and debated. In this way, we may have more policies which will deliver what they promise – and a commonly agreed foundation of information over which to argue.