Google is trying to make machines sound more human and that’s freaking people out.

Earlier this week Google demonstrated a cool new technology it’s working on called Duplex that is essentially an AI-powered automated voice system designed to enable more ‘natural’ conversations between machines and people. You can see the live demo below and click here for a bunch of audio clips showing how far along it is.

While there is clearly still a fair bit of fine tuning to be done, the inclusion of conversational furniture such as umms and ahs has unsettled some commentators, mainly on the grounds that it’s becoming hard to know if you’re speaking to a real person or not. While the whole point seems to be to make interacting with machines more smooth and intuitive, it seems we’ve hit a cognitive wall.

‘Google Grapples With ‘Horrifying’ Reaction to Uncanny AI Tech’, blurted Bloomberg. ‘Could Google’s creepy new AI push us to a tipping point?’ wailed the Washington Post. ‘Google Assistant’s new ability to call people creates some serious ethical issues’, moaned Mashable. ‘Google Should Have Thought About Duplex’s Ethical Issues Before Showing It Off’, fulminated Fortune. And then there’s this Twitter thread from a New York Times writer.

Hyperbolic headline-writing aside these are all good points. Google’s grand unveiling coincides with the broadcast of the second season of Westworld, a drama in which androids indistinguishable from humans rebel and decide to start calling the shots. And, of course, talk of AI (at least from this correspondent) is only ever one step away from references to The Terminator and The Matrix.

The above reactions to the demonstration of Duplex have forced Google to state that such interactions will always make it clear when you’re talking to a machine but it’s not yet clear exactly how. More significant, however, has been this timely reminder that not everyone embraces technological advancement as unconditionally as Silicon Valley and that AI seems to have already reached a level of sophistication that is ringing alarm bells.

And it’s not like Duplex is an isolated example. The NYT reports on findings that it’s possible to embed suggestions into recordings of music or spoken word such that smart speakers receive them as commands. The extra scary bit is that it’s possible to make these commands undetectable to regular punters.

Meanwhile Spotify has announced a new ‘Hate Content and Hateful Conduct Public Policy’, that is enforced by an automated monitoring tool called Spotify AudioWatch. This bit of AI is able to sift the lyrics of songs on the platform for stuff that goes against Spotify’s new policy, which you can read here.

On one hand we can all agree that horridness is bad and something needs to be done about it, on the other this is yet another example of algorithmic censorship. According to Billboard this facility is also being used to erase from history any artists that may have sung or rapped something horrid in the past too.

These various examples of how AI is being used to automate, manipulate and censor audio are quite rightly ringing alarm bells. Greater automation seems to be inevitable but it’s perfectly reasonable to question whether or not you want to live in a world where machines increasingly decide what’s in your best interests.