Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

561–570 of 860 posts

Re: Llama 2

#561

Post by anonylizard »

whimsicalism wrote: post #33
ipsum2 wrote: post #21

Earlier quoted context omitted.

There's a million different language model (not wrapper) companies offering APIs already. OpenAI, Anthropic, Cohere, Google, etc. It wouldn't be profitable.

There are really only three companies offering good language model APIs: OpenAI, Anthropic, and Microsoft Azure by serving up OpenAI's models. That is it.

That's like saying there's 3 competing search engines (Google, Bing, brave?). Or three competing video hosts (Youtube, tiktok, instagram). Or 3 competing cloud providers.

LLMs are infrastructure level services, 3 is a lot of competition already.

Re: Llama 2

#562

Post by jdkee »

chaxor wrote: post #513
GreedClarifies wrote: post #108

Earlier quoted context omitted.

Hard disagree. Google has made it plainly clear that they don't have anything useable in this space. Bard scores below all other commercial model. Google is getting the asses handed to them, badly. I figured that the code red would whip them into shape but the rot runs deep.

It seems you didn't quite hear the argument. I agree with you that the models Google has released to the public are absolutely worthless. That certainly does not mean they don't have extremely performant models at all however. If you actually have worked in the area of NLP for about 10 years, you would recognize how the work from Deepmind is much more novel and innovative than other groups. OpenAI certainly has great…

On that front, Google's Gemini sounds interesting.

See https://www.tomsguide.com/news/googles-new-gemini-ai-could-b...

Re: Llama 2

#563

Post by atonse »

b33j0r wrote: post #463
andsoitis wrote: post #348

Earlier quoted context omitted.

> get this question correct I am willing to bet a million dollars that it is unlikely any single model will ever be able to answer any question correctly. The implications then are that one cannot use a single question evaluate whether a model is useful or not.

I got that question wrong, I still have no idea what the correct answer would be. That is extremely obscure. Any intelligence or simulation might try to guess at an answer to that third-level-of-hell interrogation. “Why was Spartacus filmed in California near pizza noodle centurions?”

I'd struggle to find any humans that understand that question without going "huh?"

I've read it 3-4 times and it still doesn't make sense towards the end. So why would we expect these models to make sense of them?

Re: Llama 2

#564

Post by facu17y »

If we have the budget for pre-training an LLM the architecture itself is a commodity, so what does llama2 add here?

It's all the pre-training that we look to bigCo to do which can cost millions of dollars for the biggest models.

Llama2 has too small of a window for this long of a wait, which suggests that http://Meta.AI team doesn't really have much of a budget as a larger context would be much more costly.

The whole point of a base LLM is the money spent pre-training it.

But it performs badly out of the gate on coding, which is what I'm hearing, then maybe fine-tuning with process/curriculum supervision would help, but that's about it. .

Better? yes. Revolutionary? Nope.

Re: Llama 2

#565

Post by sebzim4500 »

stavros wrote: post #371
jimmySixDOF wrote: post #225

Earlier quoted context omitted.

Try the Chat Arena with ELO ratings based on end user side by side blind tests. It run out of UC Berkley by lmsys the same team that released Vicuna. https://arena.lmsys.org/

That's a terrible system, it doesn't represent gaps in performance. If the first model is orders of magnitude better than the second, that system still says "99% as good" or whatever.

The relative difference between elo ratings is meaningless, you need to look at the absolute difference.

Re: Llama 2

#566

Post by el_nahual »

whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

People keep saying this is commoditize your complement but that's not what this is!

Goods A and B are economic complements if, when the price of A goes down, demand for B goes up.

LLMs are not complements to social media platforms. There is zero evidence that if "the price of LLMs goes down" then "demand for social media apps go up".

This is a case of commoditizing the competition but that's not the same thing.

Commoditizing your complement:

- All-inclusive resorts. Restaurants are a complement to hotels. If food is free I might go on vacation more.

- Smartphone app-stores. Apps are a complement to phones. If apps cost $0.99 there will be more demand for iphones than if apps cost $20.

This is Zuck being an absolute shark and not wanting his competitors to have a monopoly over LLMs in case they win at some other game. It has nothing to do with "commoditize your complement."

Re: Llama 2

#567

Post by famouswaffles »

doctoboggan wrote: post #555
lappa wrote: post #515

Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…

Good to see these results, thanks for posting. I wonder if GPT-4's dominance is due to some secret sauce or if its just the first mover advantage and Llama will be there soon.

It's just scale. But scale that comes with more than an order of magnitude more expense than the Llama models. I don't see anyone training such a model and releasing it for free anytime soon

Re: Llama 2

#568

Post by wkat4242 »

Does anyone have a download link?

I only see a "request" to download it. That's not what I would consider "open source".

I hope someone makes a big ZIP with all the model sizes soon just like with LLaMa 1.

Re: Llama 2

#569

Post by ankeshanand »

whimsicalism wrote: post #26
stu2b50 wrote: post #19

Earlier quoted context omitted.

I think more Apple. It's not like Google or Microsoft would want to use LLaMA when they have fully capable models themselves. I wouldn't be surprised if Amazon does as well. Apple is the big laggard in terms of big tech and complex neural network models.

Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. > I wouldn't be surprised if Amazon does as well. I would - they are not a very major player in this space. TikTok also meets this definition and probably doesn't have LLM.

Has anyone in this subthread actually read the papers and compared the benchmarks? LLama2 is behind PALM-2 on all major benchmarks, I mean they spell this out in the paper explicitly.

Re: Llama 2

#570

Post by simonw »

ipv6ipv4 wrote: post #496
simonw wrote: post #414

I just added Llama 2 support to my LLM CLI tool: https://simonwillison.net/2023/Jul/18/accessing-llama-2/ So you can now access the Replicate hosted version from the terminal like this: pip install llm # or brew install simonw/llm/llm llm install llm-replicate llm keys set replicate # Paste in your Replicate API key llm replicate add a16z-infra/llama13b-v2-chat \ --chat --alias llama2 # And run a prompt llm -m llama2…

It doesn't work for me. brew install simonw/llm/llm ... ==> Installing llm from simonw/llm Error: Your Command Line Tools are too outdated. Update them from Software Update in System Settings. If that doesn't show you any updates, run: sudo rm -rf /Library/Developer/CommandLineTools sudo xcode-select --install Alternatively, manually download them from: https://developer.apple.com/download/all/. You should download t…

That's frustrating. I recommend using `pip` or `pipx` to install it instead if Homebrew doesn't work - provided you have a working Python 3 installation it should work just fine.
Post reply on HN