Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLaMA2 Chat 70B outperformed ChatGPT

tatsu-lab.github.io

Post reply on HN
Print view View original

101–110 of 135 posts

Re: LLaMA2 Chat 70B outperformed ChatGPT

#101

Post by oceanplexian »

wahnfrieden wrote: post #73
Folcon wrote: post #66

Earlier quoted context omitted.

Funnily enough I think you both might be right here, there isn't such a thing as an average person, but ChatGPT may be the synthesis of the average opinion.

what is an average opinion? it is the sum of opinions which disagree with the result

It gives you the average opinion of someone on the Internet, particularly places like Reddit, which is FAR from the “average opinion” of most people. This has been a problem for a while, people keep assuming the Internet represents some kind of moral or ethical consensus on so many issues when it’s not even close.

Re: LLaMA2 Chat 70B outperformed ChatGPT

#103

Post by bottlepalm »

beepbooptheory wrote: post #75
bottlepalm wrote: post #58

Earlier quoted context omitted.

It's funny how ChatGPT really does give you the most balanced, middle of the road answers. It feels like a distillation of all human knowledge and sentiments. I use it constantly to get advice on plans, architectures, thoughts, etc.. to get an idea of pretty much what the average person would think. It often points out things I've overlooked which I'll improve my design with and go back and forth with ChatGPT until w…

Would put slight caution around asking it anything more than "what should I read next about this?" For whatever reason*, it is particularly bad at discussing philosophy I find. When I was grading philosophy 101, I would have probably given it a passing grade against the overall curve, but that's about it. Philosophy is a discipline of careful, sometimes jargoney, and always very couched assertions that can be easily…

I asked it where moral relativism fit in with philosophy and it came back with this

    Philosophy
        Ethics/Moral Philosophy
            Meta-Ethics: The study of moral thought, language, and properties
                Moral Realism: Belief that there are objective moral facts
                Moral Anti-Realism: Denial of the existence of objective moral facts
                    Moral Relativism: The belief that moral judgments are true or false only relative to some particular standpoint
I thought that was pretty good, what do you think?

Re: LLaMA2 Chat 70B outperformed ChatGPT

#104

Post by jerrygenser »

beefield wrote: post #41
ramesh31 wrote: post #23

Earlier quoted context omitted.

>Does this mean it may be possible to self-host a ChatGPT clone assuming you have a 70B model? Not only possible but quite easy. Inference for 70B can be done with llama.cpp using CPU only, on any commodity hardware with >64GB of RAM

I have 64gb on my 5 year old thinkpad. What kind of performance (tokens per sec) I could expect on that nowadays for a 70B model?

Llama cpp speed is dramatically improved by avx instructions. If your CPU has those it would be much faster than not.

And if it doesn't you need to do some workarounds with compiling and it gets a bit harder to run.

Re: LLaMA2 Chat 70B outperformed ChatGPT

#108

Post by 2c2c2c »

sdenton4 wrote: post #46
letmevoteplease wrote: post #35

Earlier quoted context omitted.

There is one: "The agreement between GPT-4 and humans reaches 85%, which is even higher than the agreement among humans (81%). This means GPT-4’s judgments closely align with the majority of humans. We also show that GPT-4’s judgments may help humans make better judgments. During our data collection, when a human’s choice deviated from GPT-4, we presented GPT-4’s judgments to humans and ask if they are reasonable. De…

That's helpful! I've done a lot of work in audio synthesis, which is notoriously difficult measure. The gold-standard is human ratings of audio quality, but it is tough to design good tests (easy to fatigue raters) and the iteration time waiting for results is quite long. Instead, there's now some projects which use neural networks trained on human ratings to predict audio quality, such as ViSQoL: https://github.com/…

just curious, are there any open models doing the opposite of audio synthesis? As in able to generate the stems for a song?

Re: LLaMA2 Chat 70B outperformed ChatGPT

#109

Post by sundarurfriend »

bottlepalm wrote: post #58
letmevoteplease wrote: post #35

Earlier quoted context omitted.

There is one: "The agreement between GPT-4 and humans reaches 85%, which is even higher than the agreement among humans (81%). This means GPT-4’s judgments closely align with the majority of humans. We also show that GPT-4’s judgments may help humans make better judgments. During our data collection, when a human’s choice deviated from GPT-4, we presented GPT-4’s judgments to humans and ask if they are reasonable. De…

It's funny how ChatGPT really does give you the most balanced, middle of the road answers. It feels like a distillation of all human knowledge and sentiments. I use it constantly to get advice on plans, architectures, thoughts, etc.. to get an idea of pretty much what the average person would think. It often points out things I've overlooked which I'll improve my design with and go back and forth with ChatGPT until w…

> what the average person would think

Keep in mind that due to the nature of the data and the RLHF training, it's more like a weighted average, something like

    0.5*(average American view) + 0.4*(average WEIRD view) + 0.1*(average human view).
(where WEIRD = Western, Educated, Industrialized, Rich, and Democratic, standard terminology in psychological research.)

This may or may not matter depending on the questions you're asking, just something to keep in mind.

Re: LLaMA2 Chat 70B outperformed ChatGPT

#110

Post by mycall »

oceanplexian wrote: post #101
wahnfrieden wrote: post #73

Earlier quoted context omitted.

what is an average opinion? it is the sum of opinions which disagree with the result

It gives you the average opinion of someone on the Internet, particularly places like Reddit, which is FAR from the “average opinion” of most people. This has been a problem for a while, people keep assuming the Internet represents some kind of moral or ethical consensus on so many issues when it’s not even close.

On the other hand, since AIs are taught using user content from Reddit, once other cultures start using these AIs, they will begin to conform more towards those norms (or will actively go against it and The Great Sort will continue forward).
Post reply on HN