Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Falcon 40B LLM (which beats Llama) now Apache 2.0

twitter.com

Post reply on HN
Print view View original

121–130 of 143 posts

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#121

Post by Zemtomo »

ntonozzi wrote: post #13
LrnByTeach wrote: post #3

This Falcon-40B royalty free license may force Meta ... that LLama-7B/13B may soon be fully open sourced as Meta wants open source LLM advancements and contributions on its own LLM architecture.

Why do people think that Meta released their model in order to get open source coders to improve their models? They will get absolutely no competitive advantage from this. Every other team developing a closed source LLM can easily copy the innovations that open source coders have applied to Llama on their own, closed source models. There's no advantage here. Meta just spent $10 million on releasing fun chaos into the…

I'm also not following this

I believe it's more to make sure that others also continue to share their research.

Or also a general genuine good mindset of people involved in those groups.

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#122

Post by Zemtomo »

fareesh wrote: post #26

In terms of building something that's usable (considering cost, speed, scale, etc) if comparing an OpenAI API call to these, it's difficult for me to see a current path where these have any viable application outside some niche scenario. From what I understand, even to run locally you/your team needs to be able to afford a machine with a 4090. These are super expensive in some countries. I played around with the smal…

This is bleeding edge stuff.

All advances in this direction do indicate that it will be easier and easier for more people to do things with it.

This doesn't need to work for everyone.

A 4090 costs today 2k, the 3090 with also 24gb costs today 1k and costed 2k.

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#123

Post by sebzim4500 »

ntonozzi wrote: post #13
LrnByTeach wrote: post #3

This Falcon-40B royalty free license may force Meta ... that LLama-7B/13B may soon be fully open sourced as Meta wants open source LLM advancements and contributions on its own LLM architecture.

Why do people think that Meta released their model in order to get open source coders to improve their models? They will get absolutely no competitive advantage from this. Every other team developing a closed source LLM can easily copy the innovations that open source coders have applied to Llama on their own, closed source models. There's no advantage here. Meta just spent $10 million on releasing fun chaos into the…

Yeah, but some of the innovations being made at OpenAI will be replicated by the open source community, and Meta can use those for free.

They aren't looking to create a technological edge themselves, they want to remove the edge that OpenAI has so that they can win using their user count/brand recognition/etc.

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#124

Post by sebzim4500 »

nc wrote: post #92
low_tech_punk wrote: post #91

Hypothetically speaking, if Falcon 40B could out-perform GPT-3.5, would that force OpenAI to open-source GPT-3.5?

It would as open source improvements would start to exceed performance of 3.5 for specific use-cases. At the very least they would have to make it fine-tunable.

Sam Altman said recently that they are already working on making GPT-3.5/GPT-4 finetunable, they are just limited by the availability of compute (partially since none of their SFT infrastructure uses LoRa).

I had previously assumed it was safety concerns, since I don't see what stops someone from finetuning away all guard rails.

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#125

Post by sebzim4500 »

dheera wrote: post #88
amstan wrote: post #65

40B is pretty large, right? I expect it would take 70GB or so of RAM to run it. That's some expensive hardware ($10,000 or more).

Stupid question but for feed-forward models why do we not yet have some kind of CPU RAM memory swap mechanism? Why is Pytorch still trying to load the whole damn model into GPU RAM at once and then complaining when it can't, instead of swapping portions of the model to CPU RAM, or hell, even SSD? Sure, it might be a lot slower, but that's a lot better than "I give up, go buy $20K worth of hardware"

I haven't run the numbers, but I would expect that doing that would make it slower than just running on the CPU.

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#126

Post by caeril »

Zemtomo wrote: post #122
fareesh wrote: post #26

In terms of building something that's usable (considering cost, speed, scale, etc) if comparing an OpenAI API call to these, it's difficult for me to see a current path where these have any viable application outside some niche scenario. From what I understand, even to run locally you/your team needs to be able to afford a machine with a 4090. These are super expensive in some countries. I played around with the smal…

This is bleeding edge stuff. All advances in this direction do indicate that it will be easier and easier for more people to do things with it. This doesn't need to work for everyone. A 4090 costs today 2k, the 3090 with also 24gb costs today 1k and costed 2k.

Any particular reason not to run this model on a single Jetson AGX Orin 64GB?

GP Core count is much lower than than the 4090 but it still does 275 int8 TOPS for only $2k

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#127

Post by johnwheeler »

valine wrote: post #82
kossTKR wrote: post #63

Earlier quoted context omitted.

Very cool but how does this compare to GPT-4 (before it was nerfed)? I feel like the best benchmark atm is the orig gpt-4 version.

GPT-4 wasn’t nerfed. https://news.ycombinator.com/item?id=36155267

It most certainly was, despite what anyone from openai says. It’s not working the same.

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#128

Post by ralfn »

Havoc wrote: post #120
version_five wrote: post #105

Am I the only one who finds this very sketchy? They had the whole license things, there's been some loud complaining by the HF CTO on social media that this model is not getting enough attention, and there are also press releases about how Falcon tops the "leaderboard": https://www.morningstar.com/news/business-wire/2023052900504... I've never seen this kind of "strategy" with an ML model before. Maybe I'm seeing som…

Not sure I care to be honest. With an open license the community can now take this and roll with it. What PR noise they make is secondary to that in my mind.

Yeah. When companies do actually good things for PR/selfish reasons the society that rewards that with attention will thrive.

Cynics logic is always a race to the moral bottom

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#129

Post by fennecfoxy »

klysm wrote: post #46

If it turns out that there is a consistent architecture which works really well, how long before we see an ASIC?

As with everything, follow the money, pretty much.

There will be an ASIC as soon as serious money is being made from LLMs, most use cases atm seem to be in prototype/toy stage, but I imagine we'll start seeing that change.

Re: Falcon 40B LLM (which beats Llama) now Apache 2.0

#130

Post by Zemtomo »

caeril wrote: post #126
Zemtomo wrote: post #122

Earlier quoted context omitted.

This is bleeding edge stuff. All advances in this direction do indicate that it will be easier and easier for more people to do things with it. This doesn't need to work for everyone. A 4090 costs today 2k, the 3090 with also 24gb costs today 1k and costed 2k.

Any particular reason not to run this model on a single Jetson AGX Orin 64GB? GP Core count is much lower than than the 4090 but it still does 275 int8 TOPS for only $2k

I'm the wrong person to ask but performance wise a 4090 has over a pflop and a Google search also showed a factor of 3-4 for a 3090.
Post reply on HN