You must log in or register to comment.
You asked a stupid question and got a stupid response, seems fine to me.
Yes, nobody asking that question is wonderring about the “straw” part of the word. They’re asking, is the “berry” part one, or two "r"s
“strawbery” has 2 R’s in it while “strawberry” has 3.
Fucking AI can’t even count.
How many strawberries could a strawberry bury if a strawberry could bury strawberries 🍓
Skill issue
The terrifying thing is everyone criticising the LLM as being poor, however it excelled at the task.
The question asked was how many R in strawbery and it answered. 2.
It also detected the typo and offered the correct spelling.
What’s the issue I’m missing?
There’s also a “r” in the first half of the word, “straw”, so it was completely skipping over that r and just focusing on the r’s in the word “berry”
It doesn’t see “strawberry” or “straw” or “berry”. It’s closer to think of it as seeing 🍓, an abstract token representing the same concept that the training data associated with the word.
It wasn’t focusing on anything. It was generating text per its training data. There’s no logical thought process whatsoever.
Uh oh, you’ve blown your cover, robot sir.
The issue that you are missing is that the AI answered that there is 1 ‘r’ in ‘strawbery’ even though there are 2 'r’s in the misspelled word. And the AI corrected the user with the correct spelling of the word ‘strawberry’ only to tell the user that there are 2 'r’s in that word even though there are 3.
Sure, but for what purpose would you ever ask about the total number of a specific letter in a word? This isn’t the gotcha that so many think it is. The LLM answers like it does because it makes perfect sense for someone to ask if a word is spelled with a single or double “r”.
It makes perfect sense if you do mental acrobatics to explain why a wrong answer is actually correct.
Not mental acrobatics, just common sense.
Except many many experts have said this is not why it happens. It cannot count letters in the incoming words. It doesn’t even know what “words” are. It has abstracted tokens by the time it’s being run through the model.
It’s more like you don’t know the word strawberry, and instead you see: How many 'r’s in 🍓?
And you respond with nonsense, because the relation between ‘r’ and 🍓 is nonsensical.
It’s predictive text on speed. The LLMs currently in vogue hardly qualify as A.I. tbh…
Still, it’s kinda insane how two years ago we didn’t imagine we would be instructing programs like “be helpful but avoid sensitive topics”.
That was definitely a big step in AI.
What would have been different about this if it had impressed you? It answered the literal question and also the question the user was actually trying to ask.
But you realize that it’s wrong on both counts, right?
Strawberry has three Rs or two Rs in the wrong spelling.
It didn’t? StRawbeRy has 2 rs. StRawbeRRy has 3.
OHHHHHHH… my bad. I’m an idiot. Being an LLM it’s giving the answer it thinks a human such as myself would come up with.
It’s like someone who has no formal education but has a high level of confidence and eavesdrops on a lot of random conversations.
You rang?
deleted by creator
It is wrong. Strawberry has 3 r’s
deleted by creator
Uh, no, that is not common parlance. If any human tells you that strawberry has two r’s, they are also wrong.
there are two 'r’s in ‘strawbery’
I’ve already had more than one conversation where people quote AI as if it were a source, like quoting google as a source. When I showed them how it can sometimes lie and explain it’s not a primary source for anything I just get that blank stare like I have two heads.
Me too. More than once on a language learning subreddit for my first language: “I asked ChatGPT whether this was correct grammar in German, it said no, but I read this counterexample”, then everyone correctly responded “why the fuck are you asking ChatGPT about this”.
I use ai like that except im not using the same shit everyone else is on. I use a dolphin fine tuned model with tool use hooked up to an embedder and searxng. Every claim it makes is sourced.
Sure buddy
I think I have seen this exact post word for word fifty times in the last year.
And yet they apparently still can’t get an accurate result with such a basic query.
Meanwhile… https://futurism.com/openai-signs-deal-us-government-nuclear-weapon-security
Has the number of "r"s changed over that time?
y do you ask?
Just playing, friend.
Same, i was making a pun
Oh, I see! Apologies.
No apologies needed. Enjoy your day and keep the good vibes up!
Yes
I’ve been avoiding this question up until now, but here goes:
Hey Siri …
- how many r’s in strawberry? 0
- how many letter r’s in the word strawberry? 10
- count the letters in strawberry. How many are r’s? ChatGPT ……2
Doc: That’s an interesting name, Mr…
Fletch: Babar.
Doc: Is that with one B or two?
Fletch: One. B-A-B-A-R.
Doc: That’s two.
Fletch: Yeah, but not right next to each other, that’s what I thought you meant.
Doc: Isn’t there a children’s book about an elephant named Babar.
Fletch: Ha, ha, ha. I wouldn’t know. I don’t have any.
Doc: No children?
Fletch: No elephant books.
Yeah and you know I always hated this screwdrivers make really bad hammers.
I asked mistral/brave AI and got this response:
How Many Rs in Strawberry
The word “strawberry” contains three "r"s. This simple question has highlighted a limitation in large language models (LLMs), such as GPT-4 and Claude, which often incorrectly count the number of "r"s as two. The error stems from the way these models process text through a process called tokenization, where text is broken down into smaller units called tokens. These tokens do not always correspond directly to individual letters, leading to errors in counting specific letters within words.
Yes, at some point the meme becomes the training data and the LLM doesn’t need to answer because it sees the answer all over the damn place.
Here’s my guess, aside from highlighted token issues:
We all know LLMs train on human-generated data. And when we ask something like “how many R’s” or “how many L’s” is in a given word, we don’t mean to count them all - we normally mean something like “how many consecutive letters there are, so I could spell it right”.
Yes, the word “strawberry” has 3 R’s. But what most people are interested in is whether it is “strawberry” or “strawbery”, and their “how many R’s” refers to this exactly, not the entire word.
It doesn’t even see the word ‘strawberry’, it’s been tokenized in a way to no longer see the ‘text’ that was input.
It’s more like it sees a question like: How many 'r’s in 草莓?
And it spits out an answer not based on analysis of the input, but a model of what people might have said.
But to be fair, as people we would not ask “how many Rs does strawberry have”, but “with how many Rs do you spell strawberry” or “do you spell strawberry with 1 R or 2 Rs”
