Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How is ChatGPT's behavior changing over time?

arxiv.org

Post reply on HN
Print view View original

91–100 of 187 posts

Re: How is ChatGPT's behavior changing over time?

#91

Post by ramesh31 »

Sosh101 wrote: post #36
flangola7 wrote: post #19

Earlier quoted context omitted.

Llama really isn't open source, at least not in the sense of FOSS licenses like GPL or MIT. It comes with a number of use-case conditions and gives Meta many avenues to revoke a license if they feel like it. They also have a hard cap on the number of allowed users you may have using your Llama-based product above which you must seek further Meta approval. Furthermore, Llama remains well below GPT-3 on human rated tes…

> and $10,000+ of compute hardware per inference session. What hardware would you need to run it at home?

>What hardware would you need to run it at home?

Step 1: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/ma...

Step 2: https://github.com/ggerganov/llama.cpp

Step 3: you're welcome

Re: How is ChatGPT's behavior changing over time?

#92

Post by gwd »

dontupvoteme wrote: post #76
dudeinhawaii wrote: post #6

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…

Could not agree more. It doesn't understand what a number is, why is everyone trying to quiz it on maths instead of, perhaps, seeing how good it is at language tasks, or even foreign languages? I suspect it has gotten a lot worse in non-english since launch.

Why do you think this?

FWIW I use GPT-4 regularly to explain Koine Greek from the New Testament to me; its ability there certainly hasn't diminished in the last two months.

Re: How is ChatGPT's behavior changing over time?

#93

Post by reportgunner »

xp84 wrote: post #43
dudeinhawaii wrote: post #6

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…

Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.

What is the point of doing anything if you can't use flashy technologies ? Leave that to the old people. /s

Re: How is ChatGPT's behavior changing over time?

#94

Post by ChemSpider »

For _medical_ questions, the public version of ChatGPT has become useless. Even a simple question such as "What are the typical side effects of XYZ" is answered with a gigantic disclaimer and then often a fancy version of "I am not sure". Yeah, that much I knew already.

It used to be better in the past.

Re: How is ChatGPT's behavior changing over time?

#95

Post by reportgunner »

lumost wrote: post #55
xp84 wrote: post #43

Earlier quoted context omitted.

Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.

I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.

Were you in a room that had it's walls slowly caving in like in indiana jones or why does it matter that it was faster ?

Re: How is ChatGPT's behavior changing over time?

#96

Post by muzani »

deelly wrote: post #72
dudeinhawaii wrote: post #6

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…

No, we definitely should continue to quiz LLMs on mathematics and absolutely any other topics. Otherwise how do we know and understand limitation of the system?

We should also test its capabilities on cooking steak, flying rockets, and making love. Only then will we know if AI can be superior to humans on all things.

Re: How is ChatGPT's behavior changing over time?

#98

Post by SanderNL »

xp84 wrote: post #43
dudeinhawaii wrote: post #6

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…

Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.

It’s not about the results, it’s about its ability to “reason”. Math is about as close to pure reasoning we get so I don’t get the pessimism.

If it is bad at math and can’t be taught, then you have a fundamental problem. It’s a matter of time before this limit gets hit in other domains.

Re: How is ChatGPT's behavior changing over time?

#99

Post by josefx »

lumost wrote: post #55
xp84 wrote: post #43

Earlier quoted context omitted.

Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.

I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.

wouldn't it be less time to just type one yourself? Like open notepad, hit the keys like a deranged monkey, select the first 64 characters, done?

Re: How is ChatGPT's behavior changing over time?

#100

Post by manojlds »

dontupvoteme wrote: post #76
dudeinhawaii wrote: post #6

I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…

Could not agree more. It doesn't understand what a number is, why is everyone trying to quiz it on maths instead of, perhaps, seeing how good it is at language tasks, or even foreign languages? I suspect it has gotten a lot worse in non-english since launch.

The paper is talking about DELTA though. It used to do well and doesn't now.
Post reply on HN