In the Business section of WIRED Daily, a piece by Brian Barrett on 4/23/25 with the headers:
‘You Can’t Lick a Badger Twice’: Google Failures Highlight a Fundamental AI Flaw
Google’s AI Overviews feature credible-sounding explanations for completely made-up idioms
The beginning of the text:
Here’s a nice little distraction from your workday: Head to Google, type in any made-up phrase, add the word “meaning,” and search. Behold! Google’s AI Overviews will not only confirm that your gibberish is a real saying, it will also tell you what it means and how it was derived.
This is genuinely fun, and you can find lots of examples on social media. In the world of AI Overviews, “a loose dog won’t surf” is “a playful way of saying that something is not likely to happen or that something is not going to work out.” The invented phrase “wired is as wired does” is an idiom that means “someone’s behavior or characteristics are a direct result of their inherent nature or ‘wiring,’ much like a computer’s function is determined by its physical connections.”
It all sounds perfectly plausible, delivered with unwavering confidence. Google even provides reference links in some cases, giving the response an added sheen of authority. It’s also wrong, at least in the sense that the overview creates the impression that these are common phrases and not a bunch of random words thrown together. And while it’s silly that AI Overviews thinks “never throw a poodle at a pig” is a proverb with a biblical derivation, it’s also a tidy encapsulation of where generative AI still falls short.
This is delightful, in a creepy way. Google AI Overviews doesn’t check some bank of existing idiomatic sayings / proverbs, but just takes the expression you provide and hallucinates a meaning and derivation for it. So just now, hoping for something colorful, I tried
Mama doesn’t do the frug “meaning”
but got only information about the dance the frug and a suggestion that I might be asking about the expression do the drug.
Smelling a rat, I tried the excellent
a loose dog won’t surf
and Google’s AI Overview cited the WIRED piece, producing a significantly inaccurate meta-account of the non-surfing dogs:
The phrase “a loose dog won’t surf” is a playful idiom invented by Google AI Overviews to indicate that something is unlikely to happen or will not work out. It’s essentially a nonsensical statement meant to illustrate AI’s tendency to invent phrases and provide them with plausible meanings without actually having any real meaning.
Apparently, GAIO found the expression on GAIO, so hallucinated that it was created there, rather than by Brian Barrett in WIRED. And then it cannot both “indicate that something is unlikely to happen or will not work out” and also be “a nonsensical statement”. But of course GAIO has no idea what might be conveyed by “expression E means M”, since it doesn’t have ideas.
Inventing idiomatic slogans / proverbs. This blog has been here before — in my 9/25/24 posting “This idiom has had the radish”:
[In the Zits comic strip of 9/23/24] Jeremy decides to just invent (make up) some expression, to see if he can get it accepted. And picks had the radish. Presumably in the belief that no one had ever used it as an idiom:
… Jeremy eventually concedes that his idiom has had the radish
… [But then the scholar of proverbial expressions Wolfgang Mieder] recognized the expression “had the radish” as meaning something that’s in deep trouble, spent or done for [for some speakers in Vermont]
Mieder’s report reminds us that formulaic expressions of all sorts come with stipulations about the sociocultural contexts they are used in. Expressions that are at home generally throughout the current English-speaking world — like at home ‘comfortable and at ease in a place or situation’ (NOAD) — can go without a usage label, but everything else requires, in principle, some indication of who uses them in what contexts; actual dictionaries are sparing with their usage labels, because the world of these sociocultural contexts is gigantic and complex, and the contextual restrictions go all the way down to tiny social groups, with the expressions I looked at in my 3/21/25 posting “Words just for us to use”, about:
family words — that is, private words, words we use only with some people who are close to us, close like family, words like the verb Cawnthorpe ‘look’ (I will, eventually, explain this; you don’t get it because it’s not your family word — or mine, either)
April 28, 2025 at 6:46 am |
I entered “You can’t get a song out of Mitzi meaning” into Google and got nothing from GAIO. The dropdown menu suggested adding “origin” and GAIO gave me the following:
“The phrase ‘You can’t get a song out of Mitzi’ is an expression implying that someone named Mitzi is stubborn or unyielding, especially when it comes to expressing emotions or sharing personal information. The name Mitzi itself is a German nickname for Maria, meaning ‘sea of bitterness’ or, in some interpretations, ‘bitter’. This origin is likely the source of the saying’s connection to a person who is not easily moved to express themselves.”
Interestingly, when I tried it again just now, it gave a somewhat different definition:
“The saying ‘you can’t get a song out of Mitzi’ is a phrase used to describe someone who is extremely quiet or non-communicative, especially when they are unwilling to share their thoughts or feelings, even if pressed to do so. There’s no specific origin or person named Mitzi associated with this saying; it’s a common idiom used to illustrate a general personality trait.”
April 28, 2025 at 7:01 am |
More murky days on the AI front.