Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We are beginning to roll out new voice and image capabilities in ChatGPT

openai.com

Post reply on HN
Print view View original

891–900 of 914 posts

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#891

Post by famouswaffles »

TerrifiedMouse wrote: post #888
famouswaffles wrote: post #783

Earlier quoted context omitted.

>although some like Bard provide alternate drafts but they are all from the same source and can all be hallucinations ... Yes and no. If the LLM is repeating the same thing on multiple drafts then it's very unlikely to be a hallucination. It's when multiple generations are all saying different things that you need to take notice. LLMs hallucinate yes but getting the same hallucination multiple times is incredibly rar…

Then why aren’t hallucinations being eliminated by comparing drafts?

automatically comparing drafts for every single query would be expensive.

and that wouldn't eliminate hallucinations just tell you if large details have likely been hallucinated.

But it's a method some research has used.

https://arxiv.org/abs/2303.08896

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#892

Post by famouswaffles »

PoignardAzur wrote: post #863
famouswaffles wrote: post #783

Earlier quoted context omitted.

>although some like Bard provide alternate drafts but they are all from the same source and can all be hallucinations ... Yes and no. If the LLM is repeating the same thing on multiple drafts then it's very unlikely to be a hallucination. It's when multiple generations are all saying different things that you need to take notice. LLMs hallucinate yes but getting the same hallucination multiple times is incredibly rar…

Wait, is that true? I feel like that claim needs a lot of disclaimers.

https://arxiv.org/abs/2305.18248

"In particular, we find that LMs often hallucinate differing authors of hallucinated references when queried in independent sessions, while consistently identify authors of real references. This suggests that the hallucination may be more a generation issue than inherent to current training techniques or representation."

https://arxiv.org/abs/2303.08896

"SelfCheckGPT leverages the simple idea that if a LLM has knowledge of a given concept, sampled responses are likely to be similar and contain consistent facts. However, for hallucinated facts, stochastically sampled responses are likely to diverge and contradict one another."

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#893

Post by TerrifiedMouse »

famouswaffles wrote: post #891
TerrifiedMouse wrote: post #888

Earlier quoted context omitted.

Then why aren’t hallucinations being eliminated by comparing drafts?

automatically comparing drafts for every single query would be expensive. and that wouldn't eliminate hallucinations just tell you if large details have likely been hallucinated. But it's a method some research has used. https://arxiv.org/abs/2303.08896

How expensive could it be? Google Bard, a free service, offers the drafts for free. Just do the comparison on the user’s machine if the LLM provider is that cheap.

P.S. Also aren’t LLMs deterministic if you set their “temperature” to zero? Are there drafts if the temperature is zero? If not, then that’s the same as removing the randomness no?

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#895

Post by ComplexSystems »

Great demo, but this is wrong:

"The phrase “potato, potahto” comes from a song titled “Let’s Call the Whole Thing Off”, written by George and Ira Gershwin for the 1937 film “Shall We Dance”, starring Fred Astaire and Ginger Rogers. The song humorously highlights regional differences in American English pronunciation. The lyrics go through a series of words with alternate pronunciations, like “tomato, tomahto” and “potato, potahto”. The idea is that, despite these differences, we should move past them, hence the line “let’s call the whole thing off”. Over time, the phrase has been adopted in everyday language to signify a minor disagreement or difference in opinion that isn’t worth arguing about."

It's comparing American and British pronunciations, not different regional American ones. Also, "let's call the whole thing off" suggests they should break up over their differences, with the bridge and later choruses then involving a change of heart ("let's call the calling off off").

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#896

Post by famouswaffles »

TerrifiedMouse wrote: post #893
famouswaffles wrote: post #891

Earlier quoted context omitted.

automatically comparing drafts for every single query would be expensive. and that wouldn't eliminate hallucinations just tell you if large details have likely been hallucinated. But it's a method some research has used. https://arxiv.org/abs/2303.08896

How expensive could it be? Google Bard, a free service, offers the drafts for free. Just do the comparison on the user’s machine if the LLM provider is that cheap. P.S. Also aren’t LLMs deterministic if you set their “temperature” to zero? Are there drafts if the temperature is zero? If not, then that’s the same as removing the randomness no?

The drafts have to be evaluated either by a human or llm. Doing that for every request does not scale when you have millions of users.

>Just do the comparison on the user’s machine if the LLM provider is that cheap.

This is not possible. Users don't have the resources to run these gigantic models. LLM inference is not cheap. Open ai, Google aren't running profit on free cGPT or Bard.

>P.S. Also aren’t LLMs deterministic if you set their “temperature” to zero? Are there drafts if the temperature is zero? If not, then that’s the same as removing the randomness no?

It's not a problem of randomness. a temp of 0 doesn't reduce hallucinations. LLMs internally know when they are hallucinating/taking a wild guess. randomness influences how that guess manifests each time but the decision to guess was already made.

https://arxiv.org/abs/2304.13734

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#897

Post by arandomatom »

modeless wrote: post #459

Voice has the potential to be awesome. This demo is really underwhelming to me because of the multi-second latency between the query and response, just like every other lame voice assistant. It doesn't have to be this way! I have a local demo using Llama 2 that responds in about half a second and it feels like talking to an actual person instead of like Siri or something. I really should package it up so people can t…

[dead]

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#898

Post by [deleted] »

modeless wrote: post #796
dotancohen wrote: post #785

Earlier quoted context omitted.

> determining when the user is done talking is tough. Sometimes that task is tough for the speaker too, not just the listener. Courteous interruptions or the lack thereof might be a shibboleth for determining when we are speaking to an AI.

Yes interruptions are key, both ways. Having the user interrupt the bot is easy, but to have the bot interrupt the human will again require a model to predict when that should happen. But I do believe it is desirable for natural conversation.

[deleted]

Re: We are beginning to roll out new voice and image capabilities in ChatGPT

#900

Post by wccrawford »

Jensson wrote: post #648
wccrawford wrote: post #192

Earlier quoted context omitted.

I got sick of searching Google for in-game recipes for Disney Dreamlight because most of the results are a bunch of pointless text, and then finally the recipe hidden in it somewhere. I used Bing yesterday and it was able to parse out exactly what I wanted, and then give me idiot-proof steps to making the recipe in-game. (I didn't need the steps, but it gave me what I wanted up front, easily.) I tried it twice and it…

> I got sick of searching Google for in-game recipes for Disney Dreamlight You mean these? Took me a few seconds to find, not sure how an LLM would make that easier. I guess the biggest benefit of LLM then is for people who don't know how to find stuff. https://dreamlightvalleywiki.com/Cooking

Yes, but each time, I only actually care about 1 recipe, and it's easier to just search for that recipe than find a list of recipes and then search through that.

Bing made it even easier.

Also, I've found some of those lists to be missing some recipes.

Post reply on HN