Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLaMA2 Chat 70B outperformed ChatGPT

tatsu-lab.github.io

Post reply on HN
Print view View original

21–30 of 135 posts

Re: LLaMA2 Chat 70B outperformed ChatGPT

#21

Post by meepmorp »

lolinder wrote: post #18
luckystarr wrote: post #14

The value of GPT-4 also lies in its stored knowledge. A 70B model can't store that much.

The advantage of LLaMA 2 is that a company can fine tune it on the knowledge that they actually care about and then run it on their own hardware without paying API fees or relying on an unstable dependency that's constantly being tweaked.

> without paying API fees or relying on an unstable dependency that's constantly being tweaked.

and without handing a whole bunch of data to a 3rd party and hope they're securing it properly

Re: LLaMA2 Chat 70B outperformed ChatGPT

#23

Post by ramesh31 »

accrual wrote: post #8

Does this mean it may be possible to self-host a ChatGPT clone assuming you have a 70B model? I've used a 13B model with LLaMA1 and it's surprisingly good, but still nowhere near ChatGPT for coding questions.

>Does this mean it may be possible to self-host a ChatGPT clone assuming you have a 70B model?

Not only possible but quite easy. Inference for 70B can be done with llama.cpp using CPU only, on any commodity hardware with >64GB of RAM

Re: LLaMA2 Chat 70B outperformed ChatGPT

#24

Post by Zambyte »

accrual wrote: post #8

Does this mean it may be possible to self-host a ChatGPT clone assuming you have a 70B model? I've used a 13B model with LLaMA1 and it's surprisingly good, but still nowhere near ChatGPT for coding questions.

When you say "coding questions" do you mean questions that should be answered by producing code, or questions about code ("explain this")? Or both?

Re: LLaMA2 Chat 70B outperformed ChatGPT

#25

Post by cosmojg »

bazmattaz wrote: post #10
nmfisher wrote: post #3

I haven't had a chance to use the GPT-4 API yet - is it that much better than the GPT-4 available via ChatGPT? Or am I misunderstanding?

ChatGPT uses the GPT-4, but there are conspiracy theories circling that ChatGPT is neutered and thus not as good as GPT-4 through the API. The theory being that OpenAI are thottling the free version of GPT-4 (ChatGPT)

That's, uh, not a conspiracy theory. The free version of ChatGPT uses an entirely different model on the backend.

Re: LLaMA2 Chat 70B outperformed ChatGPT

#26

Post by bkanber »

nmfisher wrote: post #3

I haven't had a chance to use the GPT-4 API yet - is it that much better than the GPT-4 available via ChatGPT? Or am I misunderstanding?

ChatGPT is a wrapper to the GPT Completion API with some sane defaults. With a new beta feature you can edit the system prompt via ChatGPT, but you still can't adjust the other parameters you can reach with the API.

Re: LLaMA2 Chat 70B outperformed ChatGPT

#27

Post by cs702 »

freedomben wrote: post #6

Llama2 beat ChatGPT 3.5 with a 92.66% win rate to 89.37%, but lost to GPT-4 which got 95.28%. Still pretty amazing though!

Not really close, because performance is logarithmic in training compute.

That is, each additional percentage point of performance requires exponentially greater investment in compute during pretraining.

Llama 2 was pretrained on 2 trillion tokens -- a significant investment in compute, for sure, but still not enough to get close to GPT-4.

And this is only one benchmark.

Re: LLaMA2 Chat 70B outperformed ChatGPT

#28

Post by Jackson__ »

*When asked by GPT4 to compare the outputs.

I'm a staunch believer that it would be foolish to rely on GPT4 for quality comparisons, and it has been mind boggling to see so many people do it and treat it as perfect proof of anything.

It would be slightly more understandable if there was a study to see how human and gpt4 preferences compare, but I'm unaware of any such thing.

Post reply on HN