Google AI Overviews: The Risks of AI Search

Google has made its newest experimental search tool available to hundreds of millions of people using Chrome, Firefox and the Google app browser.

“AI Overviews” avoids the need to click through links, drawing on generative AI – the technology also behind the competing product ChatGPT – to create summaries of search results. For instance, a search for “how to keep bananas fresh for longer” produces an AI-generated roundup of practical advice, including keeping them somewhere cool and dark, away from fruit such as apples.

Yet a more unusual query can produce results that are catastrophic, or potentially unsafe.

Google is now attempting to correct these faults individually, but this has become a public-relations disaster for the search company and an awkward game of whack-a-mole.

AI Overviews usefully explains that “Whack-A-Mole is a classic arcade game where players use a mallet to hit moles that pop up at random for points. The game was invented in Japan in 1975 by the amusement manufacturer TOGO and was originally called Mogura Taiji or Mogura Tataki.”

However, AI Overviews also claims that “astronauts have met cats on the Moon, played with them, and provided care”.

More alarmingly, it advises that “you should eat at least one small rock per day” because “rocks are a vital source of minerals and vitamins”, while also recommending adding glue to pizza toppings.

Why are AI Overviews producing these results?

A central issue is that generative AI systems cannot distinguish truth from popularity. There are, for example, relatively few web pages discussing rock-eating, because it is plainly such a terrible idea.

There is, though, a widely read satirical piece by The Onion about eating rocks. Google’s AI therefore drew its summary from material that was popular rather than material that was accurate.

Generative AI systems also lack human values. Their training data includes a substantial portion of the internet.

Although advanced methods – with exotic labels such as “reinforcement learning from human feedback”, or RLHF – are employed to remove the worst material, it is hardly surprising that these tools reproduce some of the biases, conspiracy theories and even worse content found online. Indeed, considering the data used to train them, I am continually struck by how courteous and well-behaved AI chatbots are.

Is AI Overviews the future of search?

If this truly represents the future of search, the journey ahead will be uneven. Google is, naturally, trying to catch up with OpenAI and Microsoft.

The commercial rewards for leading the AI race are enormous. As a result, Google is showing less caution than it once did when placing this technology in users’ hands.

In 2023, Google chief executive Sundar Pichai said:

We've been cautious. There are areas where we've chosen not to be the first to put a product out. We've set up good structures around responsible AI. You will continue to see us take our time.

That statement now seems less accurate, as Google pushes back against claims that it has become a large, sluggish rival.

Google’s risky move

For Google, this is a high-risk approach. It could damage the public’s faith in Google as somewhere to obtain correct answers to their questions.

It may also weaken Google’s own billion-dollar business model. If people stop following links and merely read an AI summary, how will Google keep earning money?

The danger extends beyond Google. I am concerned that this application of AI could harm society more widely. Truth is already a rather disputed and malleable concept, and AI-generated falsehoods are likely to worsen the problem.

Ten years from now, we might regard 2024 as the web’s golden era: a time when most online material was high-quality, human-created content, before bots took over and flooded the internet with synthetic, progressively poorer AI-generated material.

Has AI begun breathing its own exhaust?

Second-generation large language models are likely, inadvertently, being trained on some outputs produced by first-generation models. Numerous AI start-ups also promote the advantages of training on synthetic data generated by AI.

But learning from the exhaust of today’s AI models may magnify even minor errors and biases. In the same way that inhaling exhaust fumes harms people, it harms AI.

These worries form part of a far broader situation. Worldwide, more than US$400 million (A$600 million) is invested in AI every day. Given this torrent of investment, governments are only beginning to recognise the need for safeguards and regulation to ensure AI is deployed responsibly.

Pharmaceutical firms cannot release harmful medicines. The same is true for car manufacturers. To date, however, technology companies have largely been free to do as they please.

Toby Walsh, Professor of AI, Research Group Leader, UNSW Sydney

This article is republished from The Conversation under a Creative Commons licence. Read the original article.