Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 2

ai.meta.com

Post reply on HN
Print view View original

451–460 of 860 posts

Re: Llama 2

#451

Post by quickthrower2 »

sebzim4500 wrote: post #278
gentleman11 wrote: post #138

Earlier quoted context omitted.

On HN, I really expected better of people. Why does everybody think this is open source? Every third comments is about how it's open source, but it's not Does everyone also think OpenAI is open source? Or a non-profit? Does putting open in the name just mean you're going to love it?

> On HN, I really expected better of people. Why does everybody think this is open source? If you put five HN users in a room you will get at least a dozen definitions of "open source". >Does everyone also think OpenAI is open source? No one thinks that OpenAI is open source. > Or a non-profit? I think most people just assume that OpenAI is a for profit company (which it is). There is of course a non-profit foundatio…

Open Source is very subject to the true scotsman.

Someone said on HN that just sticking all your source on Github with a permissive license isn’t true open source: it is about the community too!

Re: Llama 2

#452

Post by ilaksh »

When I turn the temp down and increase the repetition penalty slightly and add chain-of-thought, it handled my simple programming task.

"Please write a JavaScript function to sort an array of numbers and return only the even numbers in sorted order. First analyze the user's real intent, then think through the solution step-by-step."

Without the last two sentences and parameter tweaks, it checks for even in the sort compare instead of just sorting first.

Is anyone planning on doing a programming fine-tune of any Llama 2 model?

Re: Llama 2

#453

Post by bobvanluijt »

rajko_rad wrote: post #18

Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…

Ooo very cool

Re: Llama 2

#454

Post by sbierwagen »

swyx wrote: post #430
brucethemoose2 wrote: post #401

Earlier quoted context omitted.

70B should work reasonably well on 64GB CPU RAM + any decent GPU, or maybe a 24GB GPU + 32GB CPU RAM.

do you mind teaching how to do CPU/GPU RAM math? all i know is 34B 16bit = 68GB total RAM needed (because 1B of 8bytes = 1GB definitionally), but i dont know how it splits between CPU/GPU and whether the tradeoff in tok/s is acceptable

People running LLMs on CPU are generally running them integer quantized, so they use fewer bits per parameter.

Re: Llama 2

#455

Post by koonsolo »

xsmasher wrote: post #400
andsoitis wrote: post #348

Earlier quoted context omitted.

> get this question correct I am willing to bet a million dollars that it is unlikely any single model will ever be able to answer any question correctly. The implications then are that one cannot use a single question evaluate whether a model is useful or not.

"I don't know" is more correct than making up an answer.

With ChatGPT I sometimes prompt "also indicate how certain you are that your answer is correct". Works pretty good actually.

Re: Llama 2

#456

Post by burkaman »

_ea1k wrote: post #441
burkaman wrote: post #428

Earlier quoted context omitted.

I think that makes it a better test. An ideal model would recognize the ambiguity and either tell you what assumption it's making or ask a followup question.

While that is true, I'm not aware of any model that has been trained to do that. And all models can do is to do what they were trained to do.

They are just trained to generate a response that looks right, so they are perfectly capable of asking clarifying questions. You can try "What's the population of Springfield?" for an example.

Re: Llama 2

#457

Post by visarga »

minimaxir wrote: post #14
whimsicalism wrote: post #4

Key detail from release: > If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under thi…

That's an oddly high number for blocking competition. OpenAI's ChatGPT hit 100 million MAUs in January, and has gone down since. It's essentially a "Amazon and Google don't use this k thx."

School is out, it will pick up again.

Re: Llama 2

#458

Post by ilaksh »

Charlieholtz wrote: post #57

This is really exciting. I work at Replicate, where we've already setup a hosted version for anyone to try it: https://replicate.com/a16z-infra/llama13b-v2-chat

Is it possible to run the 70b on replicate?

Re: Llama 2

#459

Post by visarga »

lacker wrote: post #78
stu2b50 wrote: post #19

Earlier quoted context omitted.

I think more Apple. It's not like Google or Microsoft would want to use LLaMA when they have fully capable models themselves. I wouldn't be surprised if Amazon does as well. Apple is the big laggard in terms of big tech and complex neural network models.

I think Google or Microsoft probably would want to use LLaMa for various purposes like benchmarking and improving their own products. Check out this other condition from the license: v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof). https://github.com/facebookresearch/llama/blob/main/LICENS…

That's an ugly position on Meta's part. But Llama models are small, they are not going to be preferred to generate synthetic data, GPT-4 is the darling of synth datasets.

Re: Llama 2

#460

Post by cevn »

CSMastermind wrote: post #432
renewiltord wrote: post #375

Earlier quoted context omitted.

Interesting. Chat-GPT 4 has the following answer https://chat.openai.com/share/94ef3004-f944-4fc6-bbe5-597d54... > When does the bowl of the winds get used in the wheel of time books? > Mention the events leading up to it, the book number and name, and the chapter title if one exists. > Response: In Robert Jordan's Wheel of Time series, the Bowl of the Winds is used in the 8th book, "The Path of Daggers." The bowl is…

Oh that's interesting, the last time I checked GPT-4 I got a much worse result. Yes that answer looks correct to me though it leaves off some later uses in the book series (which is what makes this a slightly tricky question to answer). But in terms of first use it nails it. Did you add the "Mention the events leading up to it, the book number and name, and the chapter title if one exists." part to the prompt? I'm wo…

This sounds pretty good according to my memory. I did think it was first mentioned earlier than Path of Daggers. I don't remember it being used in The Last Battle but that was a pretty long chapter ...
Post reply on HN