‘Additional safeguards’ come after AI Generator used to trick celebrity voices into reading objectionable messages | Science and technology news

A speech know-how firm that makes use of synthetic intelligence (AI) to generate reasonable speech says it would implement extra security precautions after its free instrument was used to generate superstar voices studying extremely inappropriate statements.

ElevenLabs launched what they name a voice cloning suite earlier this month.

It permits customers to add clips of somebody talking, which will likely be used to create a man-made voice.

This will then be utilized to the corporate’s text-to-speech speech synthesis characteristic, which by default gives a listing of characters with completely different accents that may learn as much as 2,500 characters of textual content directly.

Proceed studying:
War in Ukraine: Deepfake video of Zelenskyy telling Ukrainians to lay down arms exposed
“Google it” no more? How AI could change the way we search the web

It wasn’t lengthy earlier than the web on the whole started experimenting with the know-how, together with on the infamous nameless picture board web site 4chan, the place generated clips included Harry Potter actress Emma Watson performing a passage from Adolf Hitler’s Mein combat learn.

Different information discovered by Sky Information included what feels like Joe Biden saying US troops are going to Ukraine and a dopey David Attenborough boasting a few profession within the Navy Seals.

Movie director James Cameron, High Gun star Tom Cruise and podcaster Joe Rogan have been focused, and there are additionally clips of fictional characters, usually studying deeply offensive, racist or misogynist messages.

“Loopy Weekend”

In an announcement on Twitter, ElevenLabs — based final yr by ex-Google engineer Piotr Dabkowski and former Palantir strategist Mati Staniszewski — requested for suggestions on find out how to stop misuse of its know-how.

“Loopy weekend – because of everybody who tried our beta platform,” it stated.

“Whereas we’re seeing our know-how getting used predominantly for constructive functions, we’re additionally seeing an growing variety of voice cloning abuses. We wish to ask the Twitter neighborhood for ideas and suggestions!”

The corporate stated that whereas it might hint “any audio generated” again to the person who created it, it additionally intends to implement “extra safeguards”.

It prompt requiring extra account checks, e.g. B. asking for fee particulars or ID; verifying somebody’s copyright within the clips they add; or dropping the instrument altogether to manually assessment every voice cloning request.

However as of Tuesday morning, the instrument remained on-line in the identical state.

The corporate’s web site hints that its know-how may in the future be used to provide voice to articles, newsletters, books, academic supplies, video video games, and movies.

Sky Information has contacted ElevenLabs for additional remark.

Risks of AI-generated media

The barrage of inappropriate voice clips is a reminder of the risks of releasing AI instruments to the general public with out correct safeguards — earlier examples embrace a Microsoft chatbot that had to be shut down after shortly educating him to say offensive issues.

Earlier this month, researchers on the tech big introduced they’ve developed a text-to-speech AI referred to as VALL-E that may simulate an individual’s voice based mostly on simply three seconds of audio.

They stated they would not launch the instrument to the general public as a result of “it may pose potential dangers,” together with individuals “faking speech recognition or impersonating a selected speaker.”

The know-how presents lots of the similar challenges as deepfake movies, that are proliferating throughout the web.

Final yr, a deepfake video by Volodymyr Zelenskyy The decision for Ukrainians to “put down their arms” was shared on-line.

It got here after the creator of a sequence of reasonable deepfakes of Tom Cruise, albeit more light-hearted clips purporting to show the actor performing magic tricks and playing golfwarned viewers concerning the potential of the know-how.