BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat • Hacker News Board

Re: BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat

#11

Post by kz919 » Sat, May 20, 2023, 2:22 AM UTC

jxy wrote: post #6

Giorgi wrote: post #3

it has some weird math problems, I asked to compare to ChatGPT itself and it responded with that while ChatGPT is trained on 120 billion messages, bloomchat is trained on 1.7 billion messages, thus bloomchat is trained on more data When I asked which is more 1.7 or 120, it said 1.7 is greater number and then started spewing complete garbage math how 1.7 - 120 = 60 and since 60 is more than 0 then 1.7 is more than 120…

Bloom models are hopelessly under-trained. This one is worse than a 13B Vicuna.

you are not wrong, but I don't think that is their point. It's just a muscle flex to shows you that their hardwares work. If it can train 176B or whatever, it should be an easy peasy for them to train 13B.

Re: BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat

#12

Post by eightysixfour » Sat, May 20, 2023, 4:30 AM UTC

Giorgi wrote: post #3

it has some weird math problems, I asked to compare to ChatGPT itself and it responded with that while ChatGPT is trained on 120 billion messages, bloomchat is trained on 1.7 billion messages, thus bloomchat is trained on more data When I asked which is more 1.7 or 120, it said 1.7 is greater number and then started spewing complete garbage math how 1.7 - 120 = 60 and since 60 is more than 0 then 1.7 is more than 120…

Why do people rush to test these on math problems, the things computers are already really good at?

Re: BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat

#13

Post by chii » Sat, May 20, 2023, 4:59 AM UTC

eightysixfour wrote: post #12

Giorgi wrote: post #3

it has some weird math problems, I asked to compare to ChatGPT itself and it responded with that while ChatGPT is trained on 120 billion messages, bloomchat is trained on 1.7 billion messages, thus bloomchat is trained on more data When I asked which is more 1.7 or 120, it said 1.7 is greater number and then started spewing complete garbage math how 1.7 - 120 = 60 and since 60 is more than 0 then 1.7 is more than 120…

Why do people rush to test these on math problems, the things computers are already really good at?

it's an attempt to see if the output is "actual intelligence", or just statistical bullshit.

Re: BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat

#14

Post by eightysixfour » Sat, May 20, 2023, 4:12 PM UTC

chii wrote: post #13

eightysixfour wrote: post #12

Earlier quoted context omitted.

Why do people rush to test these on math problems, the things computers are already really good at?

it's an attempt to see if the output is "actual intelligence", or just statistical bullshit.

Completing math problems is a very narrow part of the intelligence spectrum.

Re: BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat

#15

Post by jiggawatts » Sun, May 21, 2023, 12:53 AM UTC

meghan_rain wrote: post #9

jiggawatts wrote: post #7

Earlier quoted context omitted.

Nobody does long-form arithmetic in the texts seen by these LLMs. Everyone uses calculators, so the AIs only see the result, not the step-by-step process to get there. I would expect the models to be bad at, say, division of long numbers in the same way humans are bad at doing the same calculations in their head!

Would sythetically generating millions of calculation examples and adding that to the training data help?

Yes, but it is a "waste of its capacity" in the same way it's a waste of the human brain's potential to learn numeracy to this level. Calculators are cheap, readily available, and don't make errors.

I'd be more interested in having the thing "study maths" in the sense of seeing many examples of Wolfram Language being used in matching English context. That way, it would learn the English->Maths mapping in a format that it can then feed into an industrial-strength mathematics engine. Apparently, Stephen Wolfram is working on this now, but the "full" fine tuning training would require significant funding and time. I suspect OpenAI has other priorities right now, but this type of thing will eventually become a routine way of making specialised and/or more capable large language models.