Bard is much worse at puzzle solving than ChatGPT • Hacker News Board

Re: Bard is much worse at puzzle solving than ChatGPT

#71

Post by celim307 » Wed, Mar 22, 2023, 9:41 AM UTC

hackpert wrote: post #29

Wow I had hoped for a more productive discussion than these 1-1 comparisons of Bard vs ChatGPT that I'm seeing everywhere. The model deployed with this version of Bard is clearly a smaller model than the biggest LaMDA/PaLM models Google has been working on for ages. Which, according to their publications, show unprecedented results on _proof writing_ of all things (see Minerva). While their strategic decisions may be…

Thats like saying car brand X is more reliable than car brand Y, because brand X won formula one.

Re: Bard is much worse at puzzle solving than ChatGPT

#72

Post by inimino » Wed, Mar 22, 2023, 9:43 AM UTC

mike_d wrote: post #26

Tiberium wrote: post #12

Earlier quoted context omitted.

> and it crashed I hope you understand that it's only because of inability of OpenAI's servers to keep up with demand or some issue in their backend code - language models themselves can't "crash" like normal programs on some kind of input, because they "just" generate new tokens.

> language models themselves can't "crash" That is like saying the Excel document didn't crash, but Excel did when it tried to parse it. As far as I know there is no proof that you can't cause a LLM to crash with user input. > because they "just" generate new tokens. I can write a program that counts to 100 that crashes reliably.

> As far as I know there is no proof that you can't cause a LLM to crash with user input.

Well, then you simply don't understand how they work.

Re: Bard is much worse at puzzle solving than ChatGPT

#73

Post by cableshaft » Wed, Mar 22, 2023, 9:48 AM UTC

porphyra wrote: post #57

CamouflagedKiwi wrote: post #47

Earlier quoted context omitted.

What most people consider a rhyme is that the vowel and coda of the last syllable match (of course we don't mostly reach for the technical definition). I guess the examples there might be accent dependent. Protein/poutine is the only one of those first examples that really rhymes to me; skeleton/gelatin and cactus/practice both have different vowels. Maybe different for you though.

Protein and poutine do not rhyme if you pronounce poutine the proper way in Canadian French. And you are right of course that gelatin and practice have the short "ee" sound at the end whereas skeleton and and cactus have the "uh" sound.

> Protein and poutine do not rhyme if you pronounce poutine the proper way in Canadian French.

I looked it up, and sure, it sounds about the same in Canadian French as someone saying Vladimir Putin[1]. But I've never heard anyone say it that way myself, and in neutral French (according to the linked video, at least), it's pronounced 'pooh-teen', which sounds exactly like protein (I don't know if you pronounce protein different, but for me it's 'pro-teen').

[1]: https://youtu.be/yyis4TgmXYg

Re: Bard is much worse at puzzle solving than ChatGPT

#74

Post by masakreTech » Wed, Mar 22, 2023, 9:51 AM UTC

hackpert wrote: post #29

Wow I had hoped for a more productive discussion than these 1-1 comparisons of Bard vs ChatGPT that I'm seeing everywhere. The model deployed with this version of Bard is clearly a smaller model than the biggest LaMDA/PaLM models Google has been working on for ages. Which, according to their publications, show unprecedented results on _proof writing_ of all things (see Minerva). While their strategic decisions may be…

you can always go back to reddit

Re: Bard is much worse at puzzle solving than ChatGPT

#75

Post by ShamelessC » Wed, Mar 22, 2023, 9:52 AM UTC

vineyardmike wrote: post #52

igravious wrote: post #42

Earlier quoted context omitted.

> The author doesn't seem to understand rhyming too well (cactus practice is a weak rhyme at best) cactus / practice ? they rhyme to my mind skeleton / gelatin ? also rhyme to my ears protein / poutine also rhyme enough to be considered to rhyme You appear to be operating under the impression that the every syllable of a rhyming couplet has to rhyme exactly for it to be considered a rhyme. This is an incorrect assump…

> You appear to be operating under the impression that the every syllable of a rhyming couplet has to rhyme exactly for it to be considered a rhyme I'm not operating under that impression, but the author is [1]. To me, the final "sounds" should match - not every syllable, an end rhyme according to wiki [0]. Specifically I would consider a rhyme to require matching sounds "at least from last vowel to end", but I don't…

> Perhaps it's an accent thing but "-us" in cactus is not the same sound as "-ice" in practice.

Indeed, it's an accent thing. In America at least, pronouncing "cactus" with an "is" or an "us" sound are both valid.

I think that given that the author provided a working definition, and your provided failure example is actually passing that definition (just only for the author's dialect), and given that you are now essentially trying to change the subject to "my definition of rhyme is the correct one"; well, you're just being...pedantic? argumentative? I'm not sure.

Re: Bard is much worse at puzzle solving than ChatGPT

#76

Post by amelius » Wed, Mar 22, 2023, 9:58 AM UTC

random_cynic wrote: post #53

hackpert wrote: post #29

Wow I had hoped for a more productive discussion than these 1-1 comparisons of Bard vs ChatGPT that I'm seeing everywhere. The model deployed with this version of Bard is clearly a smaller model than the biggest LaMDA/PaLM models Google has been working on for ages. Which, according to their publications, show unprecedented results on _proof writing_ of all things (see Minerva). While their strategic decisions may be…

> its almost silly to question Google's ability to build useful LLMs. Unless they release a model one can "use" and verify their claims it's literally silly to make this statement.

There are useful open source LLMs. Or are you questioning their ability to configure; make install?

Re: Bard is much worse at puzzle solving than ChatGPT

#77

Post by hoseja » Wed, Mar 22, 2023, 11:05 AM UTC

mike_d wrote: post #26

Tiberium wrote: post #12

Earlier quoted context omitted.

> and it crashed I hope you understand that it's only because of inability of OpenAI's servers to keep up with demand or some issue in their backend code - language models themselves can't "crash" like normal programs on some kind of input, because they "just" generate new tokens.

> language models themselves can't "crash" That is like saying the Excel document didn't crash, but Excel did when it tried to parse it. As far as I know there is no proof that you can't cause a LLM to crash with user input. > because they "just" generate new tokens. I can write a program that counts to 100 that crashes reliably.

It's a fixed number of matrix multiplications. You can't make matrix multiplication crash by feeding it any sort of strange numbers (unless they haven't handled their Infs and NaNs).

Re: Bard is much worse at puzzle solving than ChatGPT

#78

Post by DeathArrow » Wed, Mar 22, 2023, 11:09 AM UTC

hackpert wrote: post #29

Wow I had hoped for a more productive discussion than these 1-1 comparisons of Bard vs ChatGPT that I'm seeing everywhere. The model deployed with this version of Bard is clearly a smaller model than the biggest LaMDA/PaLM models Google has been working on for ages. Which, according to their publications, show unprecedented results on _proof writing_ of all things (see Minerva). While their strategic decisions may be…

> its almost silly to question Google's ability to build useful LLMs.

It's almost silly to presume anything without proofs. People are judging Google based on what Google has shown.

Re: Bard is much worse at puzzle solving than ChatGPT

#79

Post by ramblerman » Wed, Mar 22, 2023, 12:38 PM UTC

mikewarot wrote: post #5

If I understand how Large Language models work, they don't actually know about spelling.... they are given tokens that represent words, and can only infer things from the context of those tokens across terabytes of data that they're given. Any rhyming done is an impressive result.

It is amazing, but somewhat explicable as an emergent effect.

Dont forget that the model has seen all the poems and raps on the internet. It built some latent space where certain words always cluster together in the context of poems, and in which location.

In this case it really has the best database available to say, what next word would slot in nicely here - as that is precisely what it was trained to do.

It is amazing, but somewhat explicable as an emergent effect.

Dont forget that the model has seen all the poems and raps on the internet. It built some latent space where certain words always cluster together in the context of poems, and in which location.

In this case it really has the best database available to say, what next word would slot in nicely here - as that is precisely how it was trained.

I find it more amazing tbh that you can ask for a poem about something, and that it then sticks to the plot, makes references to the start etc than the actual rhyming.

Re: Bard is much worse at puzzle solving than ChatGPT

#80

Post by carlmr » Wed, Mar 22, 2023, 1:01 PM UTC

Traubenfuchs wrote: post #65

hackpert wrote: post #29

Wow I had hoped for a more productive discussion than these 1-1 comparisons of Bard vs ChatGPT that I'm seeing everywhere. The model deployed with this version of Bard is clearly a smaller model than the biggest LaMDA/PaLM models Google has been working on for ages. Which, according to their publications, show unprecedented results on _proof writing_ of all things (see Minerva). While their strategic decisions may be…

They knew the war they were entering, they knew their enemies, they knew how they'd get evaluated and still decided to get this model out in its current state, leading to the conclusion: Yes, this is really the best they can do and it's much worse than the state of the art. In any case, it's a massive marketing blunder, the public opinion formed within the last hours was overwhelmingly "Bard sucks compared to ChatGPT…

>Yes, this is really the best they can do and it's much worse than the state of the art.

This is the best they can do under pressure.

ChatGPT surprised the world with how good it was, then Google scrambled to get something out quick.

A project like this is a massive undertaking, the first mover has the advantage that they can calmly refine their model until they find it presentable.

The question is, is what Google is delivering good for the timeframe since OpenGPT exploded in popularity enough for Google leadership to take note. Since that moment, realistically, is when they put pressure on their devs to push something out the door.

I think we'll see a better iteration soon. Not only from Google, but from other competitors.