David Shearer, the leader of the New Zealand Labour Party didn’t lose his job after an electoral defeat, but a mere twenty months into his term as Leader of the Opposition. He went out with a whimper, lamenting how the job was beneath him, as if contemptuous of where ambition had led him. All that politicking. He found it ‘boring’. The man who made his name working as a humanitarian coordinator for the United Nations in Rwanda, Somalia, Lebanon, Afghanistan, could not adjust to the warfare at home.

Under his leadership, the party did nothing but drift, barely recovering from its worst-ever result at the 2011 election. Elected by a split caucus whose main preoccupation seemed to be stifling the ambition of another pretender, and counting among his supporters the relics of the party that engineered the radical neoliberal reforms of the 1980s, Shearer seemed uninterested in establishing a strong political identity, waiting nearly three months before giving his first major speech, then toying with different narratives and making blunders that were magnified by his vagueness. When he finally resigned, he barely attracted the insincere, pro-forma eulogies that are usually reserved to the recently dethroned.

So the party had to replace him. And I’ll admit that, in spite of new rules stating that an election had to be held in which ordinary members, MPs and the affiliated unions would all have their say (to the tune of 40, 40 and 20 per cent of the vote respectively), I thought they’d find a way to avoid it. Pundits, for their part, chimed in almost to a man before the ink on the resignation was dry: the Labour Party couldn’t afford to let things descend into democracy. Such a process was bound to get ‘ugly’. The outcome, uncertain. And what if the membership and unions returned a candidate disliked by the caucus? What then? No, the only solution would be for the two main candidates to form a unity ticket that the base would have no choice but to endorse.

This view wasn’t shared only by people who, like Michael Egan in Australia, are opposed in principle to the parliamentary party relinquishing its power of selection. Even Chris Trotter – long-term political commentator and most vocal supporter of the candidate with whom the caucus wanted nothing to do – was against going through with the election, favouring compromise:

The party organisation, though champing at the bit to have, for the first time, some say in the selection of a Labour leader, would be wise to accept the uncontested succession represented by a Cunliffe-Robertson Unity Ticket. Elections are unpredictable things and do not always go the way people expect. Things can be done that cannot be undone; words spoken that cannot be unsaid. Better by far to accept the clear ideological and programmatic advance represented by Cunliffe’s topping of the ticket. That way, when their man walks into the November Conference as Labour’s new leader, they’ll be able to raise the roof in a tumultuous tribute not merely to his victory – but their own.

Less than a month later, one can derive the customary satisfaction from contemplating how utterly wrong about everything so many of the nation’s pundits were. The election went ahead and the sky didn’t fall. In fact, besides enjoying an unprecedented three weeks of free media for such a traditionally drab affair as a leadership selection, the party reportedly registered a 30 percent increase in membership after the cut-off date for qualifying to take part in the vote. Nobody did or said things that the most tribal loyalist might wish could be undone or unsaid. And the members got their man, as they seemingly would have in both scenarios. Which begs the question why so many people around the party – not to mention within it – were so afraid of the process, so anxious about its outcomes.

Most recently, the caucus of the New Zealand Labour Party elected two utterly disastrous leaders in succession. Yet variations on Egan’s dictum hung in the air for days: that it could be catastrophic. As opposed to what? What is it that is so frightening about the exercise of democracy within a modern political organisation, and a leftist one at that? And why is it seemingly so difficult for so many people close to the political process to view and portray democratic participation as a good in itself?

I don’t think there’s a terribly straightforward answer. There is a tension between the managerialism and growing appetite for technocratic solutions of organisations such as the NZLP, and the shift – which is part of a broader international trend – towards the direct election of party leaders. Could it be that, post the neoliberal turn, members are allowed to vote simply because the choice is so narrow as to be meaningless? Possibly. But it’s also true that the reforms within New Zealand Labour were significantly driven by a membership that sought to broaden the range of available choices, to develop policies and elect representatives that represented it more fairly – above all, to have a say in the party’s renewal, and to ditch at long last elements of its past.

‘I would be surprised if the essential economic reforms of the Hawke and Keating years could have been achieved under the new system,’ writes Egan: precisely so.

Within the ever-narrowing horizon of retail politics, such modest aspirations are enough to strike fear.