Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

threads.net

Post reply on HN
Print view View original

91–100 of 134 posts

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#91

Post by xanderlewis »

jorvi wrote: post #66
Solvency wrote: post #51

Earlier quoted context omitted.

Whoa this guy says "computation" and not grammatically bastardized techbrospeak "compute" like some neckbeard equivalent of a caveman! For that alone I commend him.

Compute is.. I don’t know the exact English grammatical term but it’s like water. Computation is not. “I have 1000 flops of compute” - works. “I have 1000 flops of computation” - doesn’t work. “That compute failed” - doesn’t work. “That computation failed” - works. They’re different.

As far as I know, it’s usually called an uncountable noun.

…but ‘computation’ is also uncountable, and your second sentence seems to be perfectly fine to me.

Your examples do not constitute an argument. You haven’t articulated the (purported) difference between the two words; you’ve just decided arbitrarily that some sentences don’t work, and not elaborated or explained at all.

I can make up words too, and provide example sentences: “karrotz are delicious” works. “carrots are delicious” doesn’t. “inside the karrotz” doesn’t work. “inside the carrots” does.

I don’t actually think there is any difference. The above comment about ‘brospeak’ was snarky but I do think it’s more of a cultural phenomenon than a semantic one — unless someone is willing to kindly explain the difference rather than just rolling their eyes!

What exactly is wrong with the sentence ‘this would require huge amounts of computation’? Saying ‘compute’ seems more to be a synonym of ‘computation’ that’s caught on recently than a useful gap-filling addition to the language. Again: reasoned arguments please. Or just ‘we think it sounds cool so we use it’ — that’s fine, too.

EDIT: pondering briefly, perhaps one could argue the difference is something like ‘you can own compute, but you can’t own computation.’ ‘Compute’ is the capacity to carry out computation. …although ‘compute’ seems to be used to refer to the ‘abstract’ computation being done as well as the computational resources, so I don’t know.

I’m stretching it. To be honest I’m not sure it’s a useful (or even real) distinction. I think it’s a matter of fashion, and that’s fine and normal.

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#93

Post by cosmojg »

MuffinFlavored wrote: post #12
monsieurbanana wrote: post #8

Earlier quoted context omitted.

Isn't that something you get from the infrastructure surrounding the llm? I thought the "running code" feature didn't need specific support from the llm, besides being able to output conforming json or code when asked to.

The LLM (Claude) currently doesn't know to not hallucinate numbers and instead write code + run it (something ChatGPT used to do but they fixed it)

Right, like the other commenter suggested, that's an infrastructure-level thing, not a model-level thing. Given that you're talking about ChatGPT, I assume you aren't accessing GPT-3.5 or GPT-4 directly through the API but using the app or the interface provided at chat.openai.com. The magic that makes the kinds of interactions you're describing possible amounts to a bit of clever prompting sprinkled on top of some rather impressive frontend design and engineering.

Correctly prompted, even Mistral-7B can write and run code in response to questions, and it's a model that can run on laptops from half a decade ago, with two or three orders of magnitude fewer parameters that GPT-4.

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#94

Post by TMWNN »

krasin wrote: post #78

Finetuning LLMs is currently the most promising way for next-gen robotics. One of such works (PaLM-e) among other things measured the impact of finetuning on general purpose tasks: https://twitter.com/DannyDriess/status/1632904698108821505 In short, an 8B model could degrade almost 10x after being finetuned on robotics tasks, while 500B model experiences a very minor degradation (~4%) and there's a hope that with a l…

>there's a hope that with a larger-sized model, it could become either zero or even negative (adding more experience improves general purpose reasoning).

The best single work of fiction ever created about LLMs' capabilities (and, perhaps, dangers) is Colossus by Jones. Although I think the film is even better than the book, only the latter mentions how, despite being created specifically for US national defense, Colossus is also fed unrelated data including Shakespeare's sonnets, because its creators do not know if it could be important.

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#95

Post by cosmojg »

rubymamis wrote: post #10
MuffinFlavored wrote: post #4

Does anything currently beat GPT-4? I saw some comments here say to check out Claude. From what I can tell, Claude hasn't figure out yet how to do the whole "generate Python code and run it in a Juptyer notebook" for math yet.

A programming task where Mistral-large beats both GPT-4 and Claude Opus: https://sdk.vercel.ai/s/Thi7RDx9e8VOZo1Ee6We5 (only Mistral got the current syntax) Although based on other tasks, overall, GPT-4 seems to be the best, but by a very small margin, so I cancelled my subscription. Although the native mobile app is really great.

Do you prefer Mistral-Large or Claude-Opus?

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#96

Post by TMWNN »

FrustratedMonky wrote: post #79
marviel wrote: post #61

Earlier quoted context omitted.

Several MSFT AI/ML friends actively dissuaded me and my team from fine-tuning. They said that it's pretty clear in all their internal tests that it "lobotomizes" the general reasoning capabilities of the model, unless you're really careful. "All work and no play makes GPT a very dull AI"

"bitter lesson that building in how we think we think does not work in the long run" Guess. Stop trying to shape the NN. And let it learn on its own.

>And let it learn on its own.

The best single work of fiction ever created about LLMs' capabilities (and, perhaps, dangers) is Colossus by Jones. Although I think the film is even better than the book, only the latter mentions how, despite being created specifically for US national defense, Colossus is also fed unrelated data including Shakespeare's sonnets, because its creators do not know if it could be important.

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#97

Post by Xenoamorphous »

greenavocado wrote: post #82
Xenoamorphous wrote: post #56

Earlier quoted context omitted.

> RAG is already irrelevant with large context windows Just last Friday I took the contents of the 2024 folder of one of the teams at the company I work for, for which we use RAG at the moment. I dumped the text index, concatenated it and used Google’s API to return the token count, to see if it would fit in Gemini’s 1M context window; turned out it was 5.7M tokens. And that’s less than 3 months worth of documents fo…

Did I read this correctly? You uploaded millions of words of your company's internal communications to Google?

I did. But this is under an enterprise deal with them that warrants privacy, not the generally available stuff. OpenAI has similar arrangements (Enterprise ChatGPT) and MS Azure before them.

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#98

Post by kergonath »

ldjkfkdsjnv wrote: post #28
minimaxir wrote: post #24

Earlier quoted context omitted.

> Also, most LLM code being written right now (prompts, RAG, etc) will be obsolete once the next model comes out. Not true. Most prompt techniques that work on current modern LLM models will work on different or future models, although it will require a QA pass for any regressions.

Yeah they will work, but they will also be unnecessary. You will also bake all sorts of logic into your application that will be solved natively on the stronger model. If you believe in the scaling theory, then writing LLM applications is non sensical.

> If you believe in the scaling theory, then writing LLM applications is non sensical.

But not doing it is an opportunity cost. You don’t built skills, tooling and experience, and you don’t get feedback on what works and where you should go.

It’s like computers in the 1990s: there’s always a better one 6 months away, so if you wait for it to stabilise, then you don’t do anything for a decade. Just enjoy the ride, bearing in mind that things change very fast and some things will be obsolete next year.

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#99

Post by kergonath »

Xenoamorphous wrote: post #52
ldjkfkdsjnv wrote: post #28

Earlier quoted context omitted.

Yeah they will work, but they will also be unnecessary. You will also bake all sorts of logic into your application that will be solved natively on the stronger model. If you believe in the scaling theory, then writing LLM applications is non sensical.

How will a more powerful model be a substitute for RAG, which is usually used with private data that won’t be present in any training dataset?

One of the idea is to just stuff all the documents in the prompt, which still keeps them private but avoids having to faff around with chunking, embedding, and vector stores. That’s not really the end of RAG as a concept, but it would change all the current tooling and infrastructure we built for it.

I don’t think RAG is going away, at least not because of this. But I expect new techniques to become available fairly regularly.

Re: GPT-4, without specialized training, beat a GPT-3.5 class model that cost $10M

#100

Post by kossTKR »

redox99 wrote: post #87
LASR wrote: post #42

I lead AI teams at my company. I've advised leadership against any kind of training / fine-tuning anything. We're not in the business of training models. We will never be as good as OpenAI / Anthropic etc. Where the real value in applications is smarter prompting techniques and RAG. There is a lot of room at the bottom in doing "dumb" things and simply feeding models with the right context to deliver customer value.

That's a pretty odd stance. I've finetuned llama/mistral models that greatly outperform GPT4 with just a prompt. You have to know when to RAG, finetune, or RAG+finetune.

How narrow is the dataset to be outperforming greatly?

Just curious about what the usecase is for a 7b model in a business context - ie. what does it do?

Post reply on HN