Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: 6 months later. How is Bard doing?

news.ycombinator.com

Post reply on HN
Print view View original

211–218 of 218 posts

Re: Ask HN: 6 months later. How is Bard doing?

#211

Post by jncfhnb »

famouswaffles wrote: post #208
jncfhnb wrote: post #203

Earlier quoted context omitted.

I don’t get how you’re failing to see the difference between knowing that you have uncertainty at all and being precise about uncertainty when making a guess. How can you possibly assert that I confidently know the answers to the questions on the test? That makes zero sense. I don’t know the answers. I might be able to guess correctly. That doesn’t mean I know them. It is decisively a guess. What’s your mom’s name? o…

>I don’t get how you’re failing to see the difference between knowing that you have uncertainty at all and being precise about uncertainty when making a guess. I'm not failing to see that. I'm saying that humans can be wrong about if some assertions they have are guesses or not. They're not always wrong but they're not always right either. If you make an assertion and you say you have a 100% confidence in that assert…

Making an assertion while being wrong does not mean you were guessing. You were simply wrong. Yet the vast majority of the time, when we are not guessing, we are correct. And when we are guessing, we can convey the ambiguity we feel. Guessing is not defined by the guarantee of accuracy.

LLMs struggle to convey uncertainty. Some fine tuning has allowed it to aggressively point out gaps. But it doesn’t really know what it knows even if maybe under the hood probabilities vary. Further, ask it if it is sure on things and it’ll frequently assume it was wrong, even if it proceeds to spit out the same answer.

Re: Ask HN: 6 months later. How is Bard doing?

#213

Post by kuchenbecker »

yieldcrv wrote: post #131
sdfsdfsdf98 wrote: post #123

Earlier quoted context omitted.

Bard often does this - but I've also found that _most_ of the time if you respond with something like > this would be very helpful for me and I think you're able to, please try it will actually give you the output you wanted... which is annoying to do - but there we are :)

I only care about learning to prompt in one style for LLM’s ChatGPT might actually have a moat here if people aren’t willing to make a conversational style one

There is going to be an imperative vs declarative AI flame war in a few years.

Conversational, interactive, and stateful vs Declarative, static, and "correct" AI UX

Re: Ask HN: 6 months later. How is Bard doing?

#214

Post by Someone »

theptip wrote: post #166
version_five wrote: post #93

Earlier quoted context omitted.

What is the upside for google? Their business is advertising and they have a massive platform for this already. What does a chatbot add? Showing that they are keeping up with evolving technology and that they also can train a competitive offering? From a pride perspective I see why they'd want to compete with OpenAI but from a business perspective? GPT+Bing has come and gone... I'm sure google worries about their bus…

What's in it for Google is not getting disrupted by OpenAI and losing their entire business, if it turns out that Chat is the new Search. While we seem to be cresting the peak of exuberance and coasting towards the trough of disillusionment, recall how freaked out everybody got a few months ago when everybody started using ChatGPT; Google's stock price tanked and they were getting slaughtered by analysts, so they nee…

But to avoid being disrupted, I don’t think they have to have a product out today.

Also, as is always the case in these “established giant vs nimble newcomer” cases, they have more to lose than OpenAI, and thus have to be more careful than them in what they release.

For Google, that’s doubly so, given that they’re the portal to product search for half the world.

Let’s say they release a product that (rightfully or not) starts claiming a product of one of their large advertisers is expensive and doesn’t work? What if it stops mentioning some products because it ‘thinks’ nobody should buy them? How be confident their offering isn’t biased, racist or sexist?

> and it's going to be lucrative for AdTech companies like Google

Would it? The true disruptor would direct you to the product that’s best for you, irrespective or how much its seller is wanting to bribe it to make that suggestion.

Re: Ask HN: 6 months later. How is Bard doing?

#215

Post by shanusmagnus »

jncfhnb wrote: post #211
famouswaffles wrote: post #208

Earlier quoted context omitted.

>I don’t get how you’re failing to see the difference between knowing that you have uncertainty at all and being precise about uncertainty when making a guess. I'm not failing to see that. I'm saying that humans can be wrong about if some assertions they have are guesses or not. They're not always wrong but they're not always right either. If you make an assertion and you say you have a 100% confidence in that assert…

Making an assertion while being wrong does not mean you were guessing. You were simply wrong. Yet the vast majority of the time, when we are not guessing, we are correct. And when we are guessing, we can convey the ambiguity we feel. Guessing is not defined by the guarantee of accuracy. LLMs struggle to convey uncertainty. Some fine tuning has allowed it to aggressively point out gaps. But it doesn’t really know what…

I'm not the person you're arguing with, but going back to the original meta-point of this thread, I too think you're vastly over-estimating people's introspective power on their internal states, including states of knowing.

The distinction you're drawing between "guessing" and "being sure of something but being wrong about it" is hazy at best, from a cognitive science point of view, and the fact that it doesn't _feel_ hazy to a person's conscious experience is exactly why this is interesting and maybe even philosophically important.

More briefly, people are just horseshit at knowing themselves, their motivations, their state of knowledge, the origins of their knowledge. We see some of these 'failures' in LLMs, but we (as a general rule, the 'royal we') are abysmal at seeing it in ourselves.

Re: Ask HN: 6 months later. How is Bard doing?

#216

Post by CamperBob2 »

jncfhnb wrote: post #211
famouswaffles wrote: post #208

Earlier quoted context omitted.

>I don’t get how you’re failing to see the difference between knowing that you have uncertainty at all and being precise about uncertainty when making a guess. I'm not failing to see that. I'm saying that humans can be wrong about if some assertions they have are guesses or not. They're not always wrong but they're not always right either. If you make an assertion and you say you have a 100% confidence in that assert…

Making an assertion while being wrong does not mean you were guessing. You were simply wrong. Yet the vast majority of the time, when we are not guessing, we are correct. And when we are guessing, we can convey the ambiguity we feel. Guessing is not defined by the guarantee of accuracy. LLMs struggle to convey uncertainty. Some fine tuning has allowed it to aggressively point out gaps. But it doesn’t really know what…

But it doesn’t really know what it knows

To be fair we don't know what we know, either. Epistemology is the bedrock that all of philosophy ultimately rests on. If it were a solved problem nobody would talk about it or study it anymore. It's not.

One of the most interesting things about current ML research is that thousands of years of philosophical navel-gazing is suddenly relevant. These tools are going to teach us a lot about ourselves.

Re: Ask HN: 6 months later. How is Bard doing?

#217

Post by famouswaffles »

jncfhnb wrote: post #211
famouswaffles wrote: post #208

Earlier quoted context omitted.

>I don’t get how you’re failing to see the difference between knowing that you have uncertainty at all and being precise about uncertainty when making a guess. I'm not failing to see that. I'm saying that humans can be wrong about if some assertions they have are guesses or not. They're not always wrong but they're not always right either. If you make an assertion and you say you have a 100% confidence in that assert…

Making an assertion while being wrong does not mean you were guessing. You were simply wrong. Yet the vast majority of the time, when we are not guessing, we are correct. And when we are guessing, we can convey the ambiguity we feel. Guessing is not defined by the guarantee of accuracy. LLMs struggle to convey uncertainty. Some fine tuning has allowed it to aggressively point out gaps. But it doesn’t really know what…

>Making an assertion while being wrong does not mean you were guessing. You were simply wrong.

This distinction is made up. It doesn't really exist in cognitive science. What does "simply wrong" even mean really ? Why is it different ?

>Yet the vast majority of the time, when we are not guessing, we are correct.

We're not good at knowing when we're not guessing in the first place. Just because it doesn't feel that way to you doesn't mean it isn't so.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/

If you asked most of the participants in this paper, they'd tell you straight faced and fully believing how decision x was the better choice and give elaborate reasons why.

The clincher in this paper (and similar others) isn't that the human has made a decision and doesn't know why. It's that he has no idea why he has made a decision but doesn't realize he doesn't know why. He believes his rationalization.

What you feel holds no water.

>But it doesn’t really know what it knows

Yeah and neither do people.

Post reply on HN