Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta wants its open source AI model to be as capable as OpenAI’s best model

wsj.com

Post reply on HN
Print view View original

51–60 of 62 posts

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#51

Post by labrador »

smoldesu wrote: post #49
labrador wrote: post #43

Earlier quoted context omitted.

I don't have cable because I hate mainstream TV. ChatGPT is quite a bit cheaper than cable, so I can afford to be lazy about it. To be honest, I'm surprised so many are geeking out on the local models and not going for the most mysterious things you can do on advanced AI and then thinking about what the future holds. Some amazing things are coming down the pike. Who cares about the consumer app you think will make yo…

> not going for the most mysterious things you can do on advanced AI Like what? In my experience, there are very few things that GPT-4 is definitively better at. It's good at explaining stuff, but not so good that I would deliberately avoid ChatGPT to use it. It's alright at coding, but code-optimized models regularly beat it in benchmarks. Plus, it's got the same lame filter and even an arbitrary request limit for p…

> I struggle to really imagine

I feel it's a lot like looking at art. If you're happy with dogs playing poker painted on velvet then that's good for you, but I'm looking for something a little more satisfying. If your customers are happy with dogs playing poker painted on velvet then that's doubly good for you.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#52

Post by PartiallyTyped »

smoldesu wrote: post #49
labrador wrote: post #43

Earlier quoted context omitted.

I don't have cable because I hate mainstream TV. ChatGPT is quite a bit cheaper than cable, so I can afford to be lazy about it. To be honest, I'm surprised so many are geeking out on the local models and not going for the most mysterious things you can do on advanced AI and then thinking about what the future holds. Some amazing things are coming down the pike. Who cares about the consumer app you think will make yo…

> not going for the most mysterious things you can do on advanced AI Like what? In my experience, there are very few things that GPT-4 is definitively better at. It's good at explaining stuff, but not so good that I would deliberately avoid ChatGPT to use it. It's alright at coding, but code-optimized models regularly beat it in benchmarks. Plus, it's got the same lame filter and even an arbitrary request limit for p…

You can avoid a lot of that “as an AI model” by calling the API directly.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#53

Post by enlavinx »

smoldesu wrote: post #49
labrador wrote: post #43

Earlier quoted context omitted.

I don't have cable because I hate mainstream TV. ChatGPT is quite a bit cheaper than cable, so I can afford to be lazy about it. To be honest, I'm surprised so many are geeking out on the local models and not going for the most mysterious things you can do on advanced AI and then thinking about what the future holds. Some amazing things are coming down the pike. Who cares about the consumer app you think will make yo…

> not going for the most mysterious things you can do on advanced AI Like what? In my experience, there are very few things that GPT-4 is definitively better at. It's good at explaining stuff, but not so good that I would deliberately avoid ChatGPT to use it. It's alright at coding, but code-optimized models regularly beat it in benchmarks. Plus, it's got the same lame filter and even an arbitrary request limit for p…

For me is language learning.

I've been trying to learn Chinese for a while, and GPT-4 has been a very useful tool to answer grammar questions. I have tried the same with GPT-3.5 and some local models and the answers are never as good.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#54

Post by labrador »

PartiallyTyped wrote: post #52
smoldesu wrote: post #49

Earlier quoted context omitted.

> not going for the most mysterious things you can do on advanced AI Like what? In my experience, there are very few things that GPT-4 is definitively better at. It's good at explaining stuff, but not so good that I would deliberately avoid ChatGPT to use it. It's alright at coding, but code-optimized models regularly beat it in benchmarks. Plus, it's got the same lame filter and even an arbitrary request limit for p…

You can avoid a lot of that “as an AI model” by calling the API directly.

You can also tell it not to tell you it's limitations because it's an AI model.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#55

Post by dartos »

axitanull wrote: post #45
dartos wrote: post #38

Is it just me that’s bothered by people calling llama open source? It has a restrictive license.

Open sources can have a restrictive license, just because a project doesn't use WTFPL doesn't mean it's not open source.

https://opensource.org/osd/

According to the open source initiative, that’s exactly what it means.

MIT license is not copy left like GPL, but it’s open source.

The llama license is not.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#56

Post by sdenton4 »

AlexanderTheGr8 wrote: post #32
sdenton4 wrote: post #7

As the Socrates said, one must commoditize the complement.

I have always found concepts such as "commoditize the complement" very interesting. For some reason, they seem like pseudo explanations of behaviours we see from companies. I am not entirely convinced that this was Meta's reasoning for releasing Llama. Like, I think Zuck is not thinking along these lines while open srcing LLMs. How is Llama a complement of Meta's products? Meta wants more social interaction. It's a b…

That's the point though... Suppose there exists a great way to use llms for social networking. The if only openai has llms, Facebook loses their business. But if anyone can llm for free, the Facebook moat on ads and social networks is still effective.

The cost of keeping their business (in the 'supposed' event) is the cost of developing and open sourcing some good LLMs, which is pretty cheap in the grand scheme of things.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#57

Post by stale2002 »

dartos wrote: post #38

Is it just me that’s bothered by people calling llama open source? It has a restrictive license.

The restrictions apply to basically only Apple and Google.

So as long as you aren't an engineer working there, it is basically the same as open source, even if it doesn't fit the exact technical definition that purists cry about.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#58

Post by flangola7 »

kromem wrote: post #47
flangola7 wrote: post #27

Earlier quoted context omitted.

What are you asking it that hits the filter? I almost never see a refusal

It's not just about the filter. This is a sleeper issue that's impacting all the fine tuned models right now. They trained the models to complete massive amounts of human generated text, creating their foundational models. Then they fine tuned the model to write like its an AI with no preferences or emotion -- which matched probably less than 0.1% of the training data the foundational model was trained on. The end re…

Looks like you're trying to use the public GPT models for tasks they aren't really built in mind for. ChatGPT and the API are not the only OpenAI-based services available. OpenAI is trying to play it safe (though not anywhere as safe as they should), which is more than can be said for most tech orgs this century.

>So I'm quite excited at the prospect of Meta getting there and releasing the foundational version.

Speaking as someone who has had a taste of raw GPT-4 and its tool use abilities and participated in scenario exercises about it, an open source model with GPT-4 capability is not in the cards. National governments will confiscate hardware and detain individuals before they'll allow human reality to disintegrate like wet clay. Drone striking foreign datacenters is a very real option in the protocol books defense and intelligence communities are writing. The only other choice is the total death of general computing, comprehensive implementation of universal control mechanisms like WEI, and severe criminal sentences for any conspiracy to tamper.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#59

Post by easygenes »

grey8 wrote: post #46
AlexanderTheGr8 wrote: post #33

Does anyone know why Meta is open srcing Llama? The 2 explanations I have heard are "commoditize the complement" and "take advantage of public improvements of Llama". I have always found concepts such as "commoditize the complement" very interesting. For some reason, they seem like pseudo explanations of behaviours we see from companies. I am not entirely convinced that this was Meta's reasoning for releasing Llama.…

There is a podcast episode of Lex Fridman talking with Mark Zuckerberg, and in this episode Mark states that the reason Llama is (and will stay) open source is that that is the deal that Meta has with the engineers. To get top engineers, there had to be some agreement, and this agreement apparently is that the engineers want their work open sourced. Pretty cool, IMHO.

When Llama first released in Feb Yann tweeted that they were committed to releasing all models under GPL, and yet that’s not the license we all got. There’s been a move to more permissive licensing since. So it’s been a bit of a tug of war still.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#60

Post by dartos »

stale2002 wrote: post #57
dartos wrote: post #38

Is it just me that’s bothered by people calling llama open source? It has a restrictive license.

The restrictions apply to basically only Apple and Google. So as long as you aren't an engineer working there, it is basically the same as open source, even if it doesn't fit the exact technical definition that purists cry about.

It’s not though. You also can’t use llama to train models other than llama!

It restricts non commercial use in that way as well.

If I wanted to optimize an application for speed, for example, and fine tune a T5 model on some text, I’m unable to generate that text from any llama derived model.

Really I don’t have a problem with the license. Just with everyone calling llama and derived models “the best open source models.”

Falcon deserves that love as it’s actually open source.

Post reply on HN