Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

71–80 of 860 posts

Re: Llama 2

#71

Post by moffkalast »

whimsicalism wrote: post #6
crop_rotation wrote: post #3

They don't benchmark against the OpenAI models at all for some reason. In my experience so far even GPT-3.5 was so much better than any OSS models I tried. Meta must be missing having a cloud business at this point. AI would have fit perfectly with that. Since they ran Whatsapp for so many years kinda without interfering too much, they could have also tried a somewhat independent cloud unit.

You don't benchmark foundation model against RLHF model, results aren't very useful.

This does seem to be a RLHF model, not a base model. Unless 'supervised fine-tuning' and 'human preference' mean something else.

Re: Llama 2

#72

Post by msp26 »

vorticalbox wrote: post #51

Seems there is 7b, 13b and 70b models https://huggingface.co/meta-llama

"We have also trained 34B variants, which we report on in this paper but are not releasing."

"We are delaying the release of the 34B model due to a lack of time to sufficiently red team."

From the Llama 2 paper

Re: Llama 2

#73

Post by 1024core »

Is there some tool out there that will take a model (like the Llama-2 model that Meta is offering up to download) and render it in a high-level way?

Re: Llama 2

#74

Post by _b »

Making advanced LLMs and releasing them for free like this is wonderful for the world. It saves a huge number of folks (companies, universities & individuals) vast amount of money and engineering time. It will enable many teams to do research and make products that they otherwise wouldn't be able to.

It is interesting to ponder to what extent this is just a strategic move by Meta to make more money in the end, but whatever the answer to that, it doesn't change how much I appreciate them doing it.

When AWS launched, I was similarly appreciative, as it made a lot of work a lot easier and affordable. The fact AWS made Amazon money didn't lower my appreciation of them for making AWS exist.

Re: Llama 2

#75

Post by chaxor »

It doesn't look like anything to me.

A lot of marketing, for sure. That's all that seems to crop up these days. After a few decent local models were released in March to April or so (Vicuna mostly) not much progress has really been made in terms of performance of model training. Improvements with Superhot and quantization are good, but base models haven't really done much.

If they released the training data for Galactica. Now that would be more revolutionary.

Re: Llama 2

#76

Post by dooraven »

whimsicalism wrote: post #66
dooraven wrote: post #61

Earlier quoted context omitted.

We still don't have access to Imagen last I checked, it's still in restricted access. We don't have access to SoundStorm or MusicLM https://imagen.research.google/ https://google-research.github.io/seanet/soundstorm/examples... https://google-research.github.io/seanet/musiclm/examples/ Why would it be surprising that they have better models for resarch that they don't want to give out yet?

Because I work in NLP so I have a good sense of the different capabilities of different firms and for the Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. The other things you are describing are just standard for research paper releases.

> Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened.

Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz.

Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype.

Vertex AI is their real proper offering, and I want to see how PaLM 2 does in comparison.

Re: Llama 2

#77

Post by jmiskovic »

capableweb wrote: post #62
jmiskovic wrote: post #58

Earlier quoted context omitted.

Then why not read their paper? "The largest Llama 2-Chat model is competitive with ChatGPT. Llama 2-Chat 70B model has a win rate of 36% and a tie rate of 31.5% relative to ChatGPT."

Do they specify which GPT version they used? Could Llama 2 really beat GPT-4?

The 70B Llama2 model ties in with 173B ChatGPT-0301 model. The GPT-4 still stands unchallenged.

Re: Llama 2

#78

Post by lacker »

stu2b50 wrote: post #19
minimaxir wrote: post #14

Earlier quoted context omitted.

That's an oddly high number for blocking competition. OpenAI's ChatGPT hit 100 million MAUs in January, and has gone down since. It's essentially a "Amazon and Google don't use this k thx."

I think more Apple. It's not like Google or Microsoft would want to use LLaMA when they have fully capable models themselves. I wouldn't be surprised if Amazon does as well. Apple is the big laggard in terms of big tech and complex neural network models.

I think Google or Microsoft probably would want to use LLaMa for various purposes like benchmarking and improving their own products. Check out this other condition from the license:

v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).

https://github.com/facebookresearch/llama/blob/main/LICENSE

Just like Google scrapes the internet to improve their models, it might make sense to ingest outputs from other models to improve their models. This licensing prevents them from doing that. Using Llama to improve other LLMs is specifically forbidden, but Google will also be forbidden from using Llama to improve any other AI products they might be building.

Re: Llama 2

#79

Post by sanjarbek16 »

swyx wrote: post #69
rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

its not clear but can we also download the model with this Llama v2 Cog thing? EDIT: Meta is being extremely prompt, just got sent the download instructions https://twitter.com/swyx/status/1681351712718876673 also is it now Llama or LLaMA since the website says Llama? lol

Meta sent me the link for the weights right away after I filled out the form.

Re: Llama 2

#80

Post by majorbadass »

cheeseface wrote: post #11

Would really want to see some benchmarks against ChatGPT / GPT-4. The improvements in the given benchmarks for the larger models (Llama v1 65B and Llama v2 70B) are not huge, but hard to know if still make a difference for many common use cases.

"In addition to open-source models, we also compare Llama 2 70B results to closed-source models. As shown in Table 4, Llama 2 70B is close to GPT-3.5 (OpenAI, 2023) on MMLU and GSM8K, but there is a significant gap on coding benchmarks. Llama 2 70B results are on par or better than PaLM (540B) (Chowdhery et al., 2022) on almost all benchmarks. There is still a large gap in performance between Llama 2 70B and GPT-4 and PaLM-2-L."
Post reply on HN