This discussion is so dumb - finetuning a base model costs ~$1 with LORA/QLORA and can yield same performance as gpt-4, but at 1/100 of the cost per token. What Bloomberg did for $10M was not finetuning..
"finetuning a base model costs ~$1 with LORA/QLORA and can yield same performance as gpt-4, but at 1/100 of the cost per token" That's a big claim - can you back that up with any examples?
GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M
131–134 of 134 posts
Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M
#132Earlier quoted context omitted.
Right, like the other commenter suggested, that's an infrastructure-level thing, not a model-level thing. Given that you're talking about ChatGPT, I assume you aren't accessing GPT-3.5 or GPT-4 directly through the API but using the app or the interface provided at chat.openai.com. The magic that makes the kinds of interactions you're describing possible amounts to a bit of clever prompting sprinkled on top of some r…
> Right, like the other commenter suggested, that's an infrastructure-level thing, not a model-level thing. By default, the ChatGPT "model" knows to not try to do math and instead write code to do the math then run it. I get that it's set up infrastructure wise to be able to run it, but why is Claude's main chat UI not trying to instead respond "hey, do this calculation on your own since I can't" or something of this…
Because ChatGPT ships with a system prompt which instructs the underlying model to do exactly that. A similar web application could be developed for Claude, and it would perform similarly with the right prompt, as it's quite good at tool use.
For example, I'm able to get Claude-3-Opus to write Python and call a Python interpreter of its own accord when questioned about time series data in some of my data analysis workflows, though I haven't glued together a pretty GUI for it yet (e.g., plots are simply saved to disk). While I haven't run into any problems around calculations yet, I'm sure it wouldn't be too hard to further refine the system prompt and ensure that all calculations are performed or checked using Python.
Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M
#133Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M
#134Earlier quoted context omitted.
greatly outperform GPT4 *for* just a prompt your overfitting to training data convinces no-one that you created a "better GPT4"
Do you always assume other people are incompetent? That's not very nice of you. I mostly work on AI, so I know if I'm overfitting or not. It performs provably better in it's domain (a niche programming language). GPT4 can barely write a hello world for it. I'm not creating a "better GPT4" general chatbot. I'm finetuning for a specific task.
You are making an extraordinary claim, and they require extraordinary evidence. Unless presented it is a good idea to assume they are bogus.