Yesterday’s technology demonstration by SpinVox at its Marlow HQ reminded everyone just how hard it is to do voice to text machine translation, and how far away anyone is from automating the bulk of the voicemail translation in the real world.

All of the messages supplied by our small group of visitors tripped through to a human operator. The event was unnecessary and humiliating for all concerned. SpinVox shouldn’t have had to lift its skirts; we didn’t need to be there.

It’s hard to see else what CIO Rob Wheatley more could have done to explain its Voice Mail Conversion System, or VMCS, other than acknowledge the real human:machine ratio – which no one expected him to do.

As far as we can conclude, yesterday’s demos were no fake. We saw that the SpinVox technical architecture is sound, and the experts – in particular, Tony Robinson – are first rate, smart and practical.

But CEO Christina Domecq wants to show it’s a different kind of business to the one it really is – a variable cost business heavily dependent on human labour. The demo put paid to the notion that SpinVox has cracked this problem.

The server side automation works like this.

It checks an audio recording of the voicemail for hang ups or duff calls (such as accidents). The server side does indeed contain speech recognition – “how we train it is a secret”, said Wheatley. DSP is applied to clean up the audio, if possible.

The server side recognition can – unless what we were watching was all an elaborate hoax – recognize quite a few simple words in a clean room environment with no shouting, slurring, background speech or music. If it fails the machine recognition, it’s sent to a QC agent in a call centre. All this has never been in dispute – what is contended is the proportion of messages that require human intervention.

“We’ve made a hybrid of two qualities,” said Wheatley, referring to the machine portion and the human portion. “Human QC agents that are consulted if the confidence score is not sufficiently high.”

The QC agent uses a type-ahead program called Tenzing, which is filled with the output of the machine translation attempt. (Tenzing is also called the LQT). It uses a “lattice”, or decision matrix, and some knowledge is incorporated. This is the “machine learning” part.

SpinVox remembers “call pairs” so names are remembered on a per-user basis. This raised the privacy issue; SpinVox didn’t know how long it kept the audio files, but like Google, said it wants to keep them as long as possible for learning purposes.

How well did Tenzing work? The program is zippy and certainly speeds up typing. As I noted above, all but the first examples tripped through. One short, clear message contained the word Sainsbury’s while the rest of the phrase had been pretty accurately translated by the machine.

But in a very rapid message left by one visitor, Milo Yiannopoulos, several passes had to be made – I’d estimate about six or seven passes. I have no doubt from first-hand experience of SpinVox that the message would have been flagged untranslatable – you would have been told to call an number to hear the audio yourself – or (and this happened increasingly frequently) been inserted with so many guesses(?) or ___ blanks that you would have had to call and listen to it anyway.

So we weren’t actually looking at technology per se – we were looking at an operational policy that attempts to minimise the time an agent spends trying to decipher the audio. Because time is money. And that’s the difference between a good service and a poor one – human skill.

“We’re the only people delivering this service at this scale,” said the CIO.

I mentioned that Jott was also exploring human/machine hybrids, while PhoneTag (formerly Simulscribe) claimed to be the biggest in the US. Last month Nuance acquired Jott for an undisclosed sum.

Towards the end, after prolonged badgering on the machine:human question, an exasperated Wheatley asked, “Does it matter? What we do is effective, and we’ve got carriers.”

Indeed, if SpinVox had taken this approach from the start – “we use humans, so what?” – we wouldn’t all be where we are now