Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't believe ChatGPT – we do not offer a "phone lookup" service

blog.opencagedata.com

Post reply on HN
Print view View original

241–250 of 284 posts

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#241

Post by coldtea »

barking_biscuit wrote: post #155
coldtea wrote: post #24

ChatGPT doesn't "recommended" anything. It just recombines text based on statistical inferences that appear like a recommendation. It could just as well state that humans have 3 legs depending on its training set and/or time of day. In fact it has said similar BS.

Does YouTube recommend you videos to watch? Does Amazon recommend you products to buy? Or do they just recombine text based on statistical inferences that appear like a recommendation?

Obviously they "just recombine text based on statistical inferences that appear like a recommendation".

And even that, they do badly.

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#242

Post by return_to_monke »

impalallama wrote: post #157
hayksaakian wrote: post #11

This marks the new age of "AI Optimization" where companies will strive to get their business featured into answers in ChatGPT. The OP's example is Unwanted demand, but it clearly shows that ChatGPT can funnel potential customers towards a product or service.

God I can just see a company using chatgpt to Astroterf huge amounts of data on the internet about their service to hopefully get that sludge feed back into their system and then become recommended. What a world.

kinda related: https://news.ycombinator.com/item?id=34889336.

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#243

Post by supriyo-biswas »

nl wrote: post #96
crazygringo wrote: post #54

I'm curious -- does anyone know of ML directions that could add any kind of factual confidence level to ChatGPT and similar? We all know now that ChatGPT is just autocomplete on steroids. It produces plausibly convincing patterns of speech. But from the way it's built and trained, it's not like there's even any kind of factual confidence level you could threshold, or anything. The concept of factuality doesn't exist…

> does anyone know of ML directions that could add any kind of factual confidence level to ChatGPT and similar? Yes. It's a very active area of research. For example: Discovering Latent Knowledge in Language Models Without Supervision ( https://arxiv.org/abs/2212.03827 ) shows an unsupervised approach for probing a LLM to discover things it thinks are facts Locating and Editing Factual Associations in GPT ( https://a…

Replying to this comment to find it later. (Is there a good way to bookmark comments on HN?)

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#244

Post by numeri »

crazygringo wrote: post #54

I'm curious -- does anyone know of ML directions that could add any kind of factual confidence level to ChatGPT and similar? We all know now that ChatGPT is just autocomplete on steroids. It produces plausibly convincing patterns of speech. But from the way it's built and trained, it's not like there's even any kind of factual confidence level you could threshold, or anything. The concept of factuality doesn't exist…

> The concept of factuality doesn't exist in the model at all.

This is an example of a whole range of beliefs about LLMs that are very common (even in the field itself), because they were obviously true for small models, but that might not necessarily hold for larger models. There's a lot that we don't know about LLMs, but we do know that they exhibit emergent behaviors as they scale. Smaller models don't really have world models, just language models, but these larger models have started developing clear world models once given the capacity and data to do so.

As for the existence of a concept of factuality, I found this paper[1] very interesting. It details an unsupervised method to identify which internal activations of the model correspond to factual statements, regardless of what the model ends up saying. Looking at those internal activations rather than just the model's output even reduces the model's susceptibility to prompts that lead it towards saying the wrong answer.

[1] https://arxiv.org/abs/2212.03827

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#245

Post by dizhn »

VectorLock wrote: post #88

This is the biggest problem I encounter when trying to use ChatGPT on a daily basis for computer programming tasks. It "hallucinates" plausible looking code that never existed or would never work, especially confusing whats in one module or API for something in another. This is where ChatGPT breaks when pushed a bit further than "make customized StackOverflow snippets." For example I asked ChatGPT to show me how to u…

It wrote me a python snippet while my question was about a go library. When prompted it's a go library it wrote similar looking code in go with the same function names that don't actually exist in the library. It's like google search past 2010. It's trying to please everybody too much rather than saying I can't do that. Though when asked to write a new original Koran verse, it does refuse to do that. :)

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#246

Post by massysett »

flangola7 wrote: post #229
massysett wrote: post #169

I'm an attorney. I've typed legal questions into ChatGPT and it has spit out answers that are grievously, 100%, libelously wrong. It has named individuals and said they committed crimes, when it is unquestionable they did no such thing. I'm waiting for people to start calling me to ask questions about something ChatGPT said, and I'll tell them it's wrong. Then they'll start arguing with me and saying if ChatGPT said…

You're trying to use a language model as an information reference. A translator can explain what a diplomat is saying but they can't perform their whole job.

? ChatGPT did not simply explain what someone else was saying. It created something completely new and completely false.

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#247

Post by quantiq »

ChatGTP wrote: post #164
CactusOnFire wrote: post #6

Because ChatGPT is so new, we are in this weird period where people haven't learned that is just as incorrect as the rest of us. I am hoping that in a year from now people will be more skeptical of what they hear from conversational AI. But perhaps that is optimistic of me.

I’m also worried there’s so potential money involved now that it’s never going away. Even if it’s wrong, dangerous, misleading, fundamentally flawed as a concept whatever. Big tech and money will find ways to keep putting it in front of us.

I see a lot of parallels here to crypto and NFTs where people start inventing use cases for technologies that fundamentally haven’t demonstrated business value, and pray that one day business value will show up out of nowhere.

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#248

Post by sdrinf »

crazygringo wrote: post #54

I'm curious -- does anyone know of ML directions that could add any kind of factual confidence level to ChatGPT and similar? We all know now that ChatGPT is just autocomplete on steroids. It produces plausibly convincing patterns of speech. But from the way it's built and trained, it's not like there's even any kind of factual confidence level you could threshold, or anything. The concept of factuality doesn't exist…

Add to your prompt: "For every factual statement, assign a certainty float 0..1, where 0 means you're very uncertain, and 1 means you're absolutely certain it is true".

Specific example: "why do we have first-person subjective experiences? List current theories. For every theory, assign a truthiness float 0..1, where 0 means you're sure it is wrong, and 1 means you're absolutely sure it is true"

From experimenting with this, it will shift the output, sometimes drastically so, as the model now has to reason about it's own certainty; it tends to make significantly less shit up (for example, the non-truth-marked version of the output for the query above also listed panpsychism; whereas the truth-marked version listed only scientific hypotheses).

So the model _can_ reason about it's certainty, and truth-value; and I strongly suspect it was just not rewarded during RLHF for omitting things it knew to be false -basically, percolating the social lies people tell to eachother- which seems to show up in coding as well.

Edit: see https://twitter.com/sdrinf/status/1629084909422931969 for results

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#249

Post by pwdisswordfishc »

Izkata wrote: post #119
jxy wrote: post #68

Earlier quoted context omitted.

This is hilarious. ChatGPT even gave me a more bizarre example. > does 2 pounds of bricks weigh more than 1 pound of bricks? > No, 2 pounds of bricks do not weigh less or more than 1 pound of bricks. 2 pounds of bricks and 1 pound of bricks refer to the same unit of weight, which is a pound. Therefore, they weigh the same, which is one pound. The difference between them is only the quantity, not the weight. > It's si…

The wording on this one sounds like it picked up an old riddle/trivia question and mixed it together the wrong way: What weighs more, a pound of feathers or a pound of gold? The trick answer is that the pound of feathers weighs more, because gold is (was) measured in a system where 1 pound = 12 ounces, while feathers would be weighed using the modern system where 1 pound = 16 ounces. https://en.wikipedia.org/wiki/Tro…

This is why SI units are superior. Less opportunity to deceive.

Re: Don't believe ChatGPT – we do not offer a "phone lookup" service

#250

Post by wildrhythms »

sdrinf wrote: post #248
crazygringo wrote: post #54

I'm curious -- does anyone know of ML directions that could add any kind of factual confidence level to ChatGPT and similar? We all know now that ChatGPT is just autocomplete on steroids. It produces plausibly convincing patterns of speech. But from the way it's built and trained, it's not like there's even any kind of factual confidence level you could threshold, or anything. The concept of factuality doesn't exist…

Add to your prompt: "For every factual statement, assign a certainty float 0..1, where 0 means you're very uncertain, and 1 means you're absolutely certain it is true". Specific example: "why do we have first-person subjective experiences? List current theories. For every theory, assign a truthiness float 0..1, where 0 means you're sure it is wrong, and 1 means you're absolutely sure it is true" From experimenting wi…

I initialized with that prompt and it did not give me any 0..1 certainty values on any subsequent output to my queries.
Post reply on HN