Today we have learned that the big six energy providers have been overcharging customers to the tune of over £1 billion per year.

Obviously your first thought on this story is “And what will we learn next week? Which religion the Pope follows? Or perhaps what do bears do in the woods?” But I think it’s worth taking a moment to think about why the energy companies have got away with this, and what might be done about it.

Energy companies were privatised by the Thatcher government back in the late 1980s and early 1990s, based on the ideological belief that competition would make the market more efficient. I’m not sure I’d call overcharging consumers by over £1 billion efficient.

It’s as if Thatcher had read the first few pages of an economics textbook that talks about the advantages of competition and the free market, and then gave up on the book without reading the rest of it to find out what can go wrong with free markets in practice.

Many things can go wrong with free markets, but the big one here is information asymmetry. It’s an important assumption of free market competition that buyers and sellers have perfect information. If buyers do not know how much something is costing them, how can they choose the cheapest supplier?

It is extraordinarily difficult to compare prices among energy suppliers. When I last switched my energy supplier, I spent well over an hour constructing a spreadsheet to figure out which supplier would be cheapest for me. And I’m a professional statistician, so I’m probably better equipped to do that task than most.

Even finding out the prices is a struggle. Here is what I was presented with after I looked on NPower’s website to try to find the prices of their energy:

It seems that they want to know everything about me before they’ll reveal their prices. And I’d already had to give them my postcode before I even got that far. Not exactly transparent, is it?

It was similarly impossible to find out Eon’s prices without giving them my entire life history. EDF and SSE were a bit more transparent, though both of them needed to know my postcode before they’d reveal their prices.

Here are EDF’s rates:

And here are SSE’s rates:

Which of those is cheaper? Without going through that spreadsheet exercise, I have no idea. And that’s just the electricity prices. Obviously I have to do the same calculations for gas, and given that they all give dual fuel discounts, I then have to calculate a total as well as figuring out whether I would be better off going with separate suppliers for gas and electricity to take the cheapest deal on each and whether that would compensate for the dual fuel discount.

And then of course I also have to take into account how long prices are fixed for, what the exit charges are, etc etc.

Seriously, if I as a professional statistician find this impossibly difficult, how is anyone else supposed to figure it out? There are price comparison websites that are supposed to help people compare prices, but of course they have to make a living, and have their own problems.

It’s no wonder that competition is not working for the benefit of consumers.

So what is to be done about it?

I think there is a simple solution here. All suppliers should be required to charge in a simple and transparent way. The standing charge should go. Suppliers should be required simply to quote a price per unit, and should also be required to publish those prices prominently on their website without consumers having to give their inside leg measurements first. If different rates are given for day and night use, a common ratio of day rate to night rate should be required (the ratio used could be reviewed annually in response to market conditions).

Suppliers will no doubt argue that a flat price per unit is inefficient, as there are costs involved in simply having a customer even before any energy is used, and a customer who uses twice as much energy as another does not cost them twice as much.

Tough. The energy companies have had over 20 years to sort out their act, and have failed. While I’m not a fan of governments intervening in markets as a general principle, there are times when it is useful, and this is one of them. I don’t see how anyone can argue that an industry that overcharges consumers by over £1 billion per year is efficient. No one energy company would be at  a disadvantage, as all their competitors would be in the same position.

There would be a further benefit to this idea, in that it would add an element of progressiveness to energy pricing. At the moment, poor people who don’t use much energy pay more per unit than rich people. That doesn’t really seem fair, does it?

This is such a simple and workable idea it is hard to understand why it hasn’t already been implemented. Unless, of course, recent governments were somehow on the side of big business and cared far less about ordinary consumers.

But that can’t be true, can it?