Earlier quoted context omitted.
>Llama really isn't open source, at least not in the sense of FOSS licenses like GPL or MIT. Note that I never said "open source" just "open models". As in, I can now actually build things with GPT3 capability that run locally. I couldn't care less about the model code. >Llama 2 still requires multiple kilowatts of electricity and $10,000+ of compute hardware per inference session. I'm running llama-2-7b-chat on my 8…
If you compiled llama.cpp with Metal I wouldn’t say its ~all CPU based. But it is still incredible! I need to set this up myself this weekend :)
How is ChatGPT's behavior changing over time?
101–110 of 187 posts
Re: How is ChatGPT's behavior changing over time?
#102Earlier quoted context omitted.
Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.
I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.
Obligatory bash solution:
$ random_bytes() { xxd -plain -c 0 -l "$1" /dev/urandom; }
$ random_bytes 32
e6a4a7bbea69a0164cbb66c89f8f528af93c6d2459fd28d2640e2952c031b618Re: How is ChatGPT's behavior changing over time?
#103Earlier quoted context omitted.
I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.
wouldn't it be less time to just type one yourself? Like open notepad, hit the keys like a deranged monkey, select the first 64 characters, done?
That is very much not random, but generally enough for 99% of all use cases.
Re: How is ChatGPT's behavior changing over time?
#104I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…
Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.
> Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.
Using GPT to do maths is probably like using a 737 to drive around on the ground.
Teaching GPT to do maths might be like teaching a child the times tables - a skill that can help overall reasoning.
Re: How is ChatGPT's behavior changing over time?
#105Earlier quoted context omitted.
> Furthermore, Llama remains well below GPT-3 on human rated tests such as programming, and GPT-3 is already over three years old. It's just not part of the entended use-case, and hasn't been trained to do so. It's almost like complaining that StableDiffusion isn't good at text generation… > It is also misleading to suggest Llama 2 can be ran on consumer hardware - the smaller and quantized models can but those are e…
>It's just not part of the entended use-case, and hasn't been trained to do so. It's almost like complaining that StableDiffusion isn't good at text generation… For the most part none of these systems were trained to do anything. The capabilities are emergent. My statement about the benchmarks remains unfazed. >The biggest models will be abble to run on the CPU just fine with llama.cpp as long as you have enough (che…
> For the most part none of these systems were trained to do anything. The capabilities are emergent.
The base models yes, but the chat version have been explicitly tuned for specific use-cases, it's not just basic next-token-prediction.
> Time is money
No, time is time, only context gives it monetary value: a single ms in some context can be much more expensive than an hour in another.
> Standing up racks of maxed out RAM server slots
Why are you talking about servers we're talking about the ability to run things on consumer hardware here…
> There is a reason ChatGPT incurs electrical bills on the order of a million dollars per day
Because they spend many MWh, to run all of their customers' requests, but when you're running your own requests on your own hardware, then the energy consumption is tiny, even if you needed a lot of instantaneous power.
Re: How is ChatGPT's behavior changing over time?
#106Earlier quoted context omitted.
wouldn't it be less time to just type one yourself? Like open notepad, hit the keys like a deranged monkey, select the first 64 characters, done?
That is very much not random, but generally enough for 99% of all use cases.
I would not bet on whether chatgpt's results would be more or less random than this process.
Re: How is ChatGPT's behavior changing over time?
#107I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…
Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.
> GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you.
One significant difference is that in both of those examples it is (or quickly becomes) plain why it’s a ridiculous idea. Even if you don’t understand it yourself, you’ll get external feedback fast. Not so with LLMs, where even people with technical needs may fail to see what is or isn’t a good use of the tool. Case in point: https://news.ycombinator.com/item?id=36782446
“You’re holding it wrong” isn’t a valid argument in perpetuity. At a certain point it becomes the fault of the designer, not the user.
Re: How is ChatGPT's behavior changing over time?
#108I think we should stop trying to quiz LLMs on mathematics, something for which they are explicitly not designed to do with their tokenized view of the world. Ask GPT-4 to use its Wolfram plugin and it returns the answers quickly and correctly. Second, I think the code generation bit of this paper is blown out of proportion. The code can't be immediately injected into a codebase due to a formatting change (triple quot…
I think knowing if the code can be used verbatim is actually the more important part practically speaking. That is the actually useful part. Quality is important to humans, because humans have to read it, but correctness is what people using ChatGPT for code actually need. So long as the quality and performance is good enough, then it will be useful. Performance is such a nuanced topic that you need very context awar…
GPT is very useful for the scenario of when you have a complex, dialect specific bit of SQL and need to turn it into an ORM query like sqlalchemy.
It get's you about 90% the way there
Re: How is ChatGPT's behavior changing over time?
#109Earlier quoted context omitted.
Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.
It’s not about the results, it’s about its ability to “reason”. Math is about as close to pure reasoning we get so I don’t get the pessimism. If it is bad at math and can’t be taught, then you have a fundamental problem. It’s a matter of time before this limit gets hit in other domains.
In my experience, it can reason usably well but acts like it has dyscalculia. It'll set up a proper algorithm, step through it and trip over digits.
Re: How is ChatGPT's behavior changing over time?
#110Am I the only lucky person who never invested a dime for GPT4?