AI is at war with itself

This article is part of an ongoing project we call “Defense Against the AI Arts.” In subsequent articles, podcasts, panels, and special issues we will investigate the impact of generative AI on security, society, and law.

The rapid adoption of generative AI technology has spawned a reactive market niche dedicated to curbing the adversarial and unethical use of AI tools. More than two dozen competitors arose in the past 6 months in this niche, and more are on the way. It almost seems that the AI market is an industry at war with itself.

It would be hard to say if there was any other technological development that achieved mass adoption as quickly as generative AI, led by massive investments from Microsoft and Alphabet. It is easier to say that no technology was adopted faster for adversarial and criminal enterprises. OpenAI officially opened public access to their large language model platform, ChatGPT, in late fall, and before Christmas hit, criminals were using it to write malware.

Plagiarism leads to revelation

However, in another first, tools to detect and defend against generative AI crime appeared almost simultaneously, many of them based on previous tools to detect plagiarism in college essays. Copyleaks was one of the first companies offering plagiarism tools to pivot to detecting AI-generated text, according to CEO Alon Yamin.

“A half a year ago when ChatGPT was released, we understood that it’s not going to be enough to just say that content is original or plagiarism free because it could be totally original but created by a language model,” Yamin explained. “We discovered that the models we worked on for years to deal with plagiarism are able to identify the unique voice of humans vs that of LLMs.”

Yamin said they have identified unique characteristics of GPT 3.5, GPT 4, and Bard and can fairly accurately differentiate if a text is written by a human. They released an enterprise-level solution for AI content detection shortly after the ChatGPT release in December.

Generated text is one issue, however. Deep fakes create an entirely separate problem.

Deep fakes were inevitable

According to Vijay Balasubramaniyan, CEO of Pindrop, a simple edited recording of a person’s voice a criminal can convince a stockbroker to sell a certain amount of stock and have it wired to a specified account. That’s known as a replay attack. Then there are synthetic attacks where a voice simulator mimics certain vocal patterns you started seeing a lot of the synthetic attacks, creating a voice that sounds like you. “And then there is the deep fake attack that mimics more than patterns.” (Listen to the interview below)

Rik Turner, senior principal cybersecurity analyst for Omdia said, “Firms are rushing to increase their dependence on AI for all sorts of applications, for example, customer service in call centers and chat boxes, healthcare, business insights generally, pure research, and of course cybersecurity itself.” That dependence makes companies that rely on voice and visual communication vulnerable in a variety of ways.

Balasubramaniyan said a defense needs to determine if the caller is a live voice. “Is this a real human on the other end, making these conversations or not? In each of these cases, there are telltale signs that it is not life.

Pindrop’s technology focuses on the more subtle idiosyncrasies of human speech, identifies when a communication is potentially fraudulent and either flags it or blocks it altogether.

Finding fakes is complicated

“When you’re human, you have certain physical limitations by which you speak. You have a vocal tract that sounds like a certain way. You have your nasal cavity that looks like a certain way. And then you have your lips that constantly move. So when you, for example, say, ‘Hello, Paul,” your mouth is wide open when you said “hello,” and then it shut down when you said, “Paul.” There is a certain speed with which you can do that. These deep fake systems can’t mimic all these physical limitations (without a massive dataset). They’re only creating systems that only fool human ears and eyes.”

Pindrop goes one level deeper into determining the authenticity of audio by creating an AI/ML engine to identify known sources of deep fakes. Even those sources have unique “tell” and the Pindrop technology can identify those traits.

Using ML platforms to defend against adversarial platforms brings yet another problematic layer: attacking an AI itself.

In their best-selling book, Not with a Bug but with a Sticker, data scientists Ram Kumar and Hyrum Anderson point out that AI databases are extremely fragile. They claim that as many as one in two AIs in use today have already been corrupted.

Studies show weaknesses

A recent study at the University of Waterloo was able to fool voice authentication software 99 percent of the time. and in other studies the effectiveness of the tools was varied. Pindrop and CopyLeak performed poorly in some, and quite well in others, though never 100 percent.

The Waterloo study used five minutes of recorded voice audio to create a deep fake capable of fooling the defenses by the team removing markers from the deep-fake audio that the tools used to identify a computer-generated voice.

“Input data for AI and the algorithms are obvious targets for tampering/poisoning by threat actors,” said Turner, so AI products defending a system against breaches need to be defended as well. Turner said the company HiddenLayer was established out of the Cylance breach four years ago when it was discovered the cybersecurity company’s AI was being manipulated by criminals. So HiddenLayer protects an AI designed to protect AI from adversarial AIs.

That, in itself, is not enough. Balasubramaniyan and Yamin recognize that vulnerability and not only update their database with newly found adversarial attacks, and actively go looking for them.

“As (adversarial) AI gets more sophisticated and smarter, we’re getting more sophisticated as well. We are aware of all these tests being done in different places,” added Yamin. “The problem with them is that it’s a very, very small scale. Even like, you know, the bigger ones where it’s like 10, 20 documents, it’s almost nothing.”

Yamin said for text they consider the type of documents, the subject areas, languages, etc. “We’re creating huge, huge data sets with millions of documents. We’re checking ourselves as well as we’re checking everything that is out there.”

In cybersecurity, everything is a moving target. But at least in AI/ML the good guys are catching up, if they are not yet ahead.

Lou Covey

Lou Covey is the Chief Editor for Cyber Protection Magazine. In 50 years as a journalist he covered American politics, education, religious history, women’s fashion, music, marketing technology, renewable energy, semiconductors, avionics. He is currently focused on cybersecurity and artificial intelligence. He published a book on renewable energy policy in 2020 and is writing a second one on technology aptitude. He hosts the Crucial Tech podcast.