Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The state of open source AI

stateofopensource.ai

Post reply on HN
Print view View original

261–270 of 379 posts

Re: The state of open source AI

#261

Post by chaosharmonic »

mft_ wrote: post #171
chaosharmonic wrote: post #59

Earlier quoted context omitted.

I don't think it needs to be framed purely as generosity. You just need a sufficiently self-interested actor that sees open ecosystems as a necessary part of reducing their own risk profile, relative to the alternative of complete reliance of a third-party business that can take an exorbitant cut and/or Sherlock them at any time. Valve and SteamOS are a good example of what this idea looks like in practice. (Though t…

> You just need a sufficiently self-interested actor that sees open ecosystems as a necessary part of reducing their own risk profile, relative to the alternative of complete reliance of a third-party business that can take an exorbitant cut and/or Sherlock them at any time. This would be an argument for an organisation developing its own model; but not per se for releasing the weights openly. The possible explanatio…

I acutally see the reduced burden that comes with actually sharing resources.

To extend the previous analogy: Valve didn't make desktop Linux viable on their own. A lot of it is owed to another self-interested actor -- Google -- through the reduced need for dedicated desktop apps (largely pushed by Chrome), and various enhancements to wireless and power management that were necessary to make it a viable mobile platform (directly benefiting Android/ChromeOS, but then spreading out to laptops and mobile devices in general, including handhelds like the Steam Deck).

You can see it on a smaller scale in ecosystems like Android -- where handset makers regularly contribute features from their UI skins upstream, so that they no longer need to spend engineering resources maintaining distinct versions of theming engines/notification badges/multiwindow/various other stuff.

On a related note this is an argument for open source models, not just open weights. I think a lot of the diminishing returns relate to the opaque nature of most models.

Re: The state of open source AI

#262

Post by avaer »

blintz wrote: post #244
babblingfish wrote: post #6

Speculation: open models is what will kill Anthropic and OpenAI. Hyperscalers can run the models without a licensing fee. Apple can make them smaller and put them on the device. The frontier models are an edge and a liability. They're astronomically expensive to train. Without them, their models will fade into obscurity. Their marketing depends on people believing the models are meaningfully different, as people have…

Are frontier models actually astronomically expensive to train? GLM 5.2 was trained on ~30T tokens, so ~10^25 FLOPs. Say you get B300's for $5/h (pretty high), and you get 50% MFU; that's ~$15M. Of course there's also a bunch of risk that the training itself goes badly, post-training, etc. But still, compared to the inference spend after, it's not that crazy.

Nobody spends 15M on inference only to check at the end that their money was wasted.

Re: The state of open source AI

#263

Post by satvikpendem »

mark_l_watson wrote: post #217
satvikpendem wrote: post #213

Earlier quoted context omitted.

They'll lobby to ban them, especially Chinese models, as Amodei is already doing.

I worry about that but here are two positive things: 1. it would put us (USA) at a competitive disadvantage, and cooler heads will prevail in this fight 2. there are good US open models. I have the latest gemma4:27b with better tool support functioning at a high level in the pi coding harness. Thinking Machines seems to be on a good path, we will see what they and other US companies can do.

Cooler heads, in this administration?

Re: The state of open source AI

#264

Post by seba_dos1 »

inigyou wrote: post #157
Cyberdog wrote: post #128

Earlier quoted context omitted.

While that may be technically true for a strict definition of “smartphone,” there’s no denying the iPhone redefined the concept in a way that its competitors were forced to copy to have any hope of keeping up. Nobody hears the word “smartphone” and thinks of a Blueberry or Treo anymore.

What exactly did the iPhone do better?

It preceded its release with the iPod craze, making it a lifestyle product rather than something marketed for its capabilities (which were very humble compared to other contemporary devices and only started to catch up in second gen).

Re: The state of open source AI

#265

Post by holoduke »

mft_ wrote: post #33
babblingfish wrote: post #6

Speculation: open models is what will kill Anthropic and OpenAI. Hyperscalers can run the models without a licensing fee. Apple can make them smaller and put them on the device. The frontier models are an edge and a liability. They're astronomically expensive to train. Without them, their models will fade into obscurity. Their marketing depends on people believing the models are meaningfully different, as people have…

Open models are probably also comparatively astronomically expensive to train - just less so than the frontier models because they’re somewhat smaller, +/- the creators are more incentivised to focus on getting more from less compute because they’re have to, +/- they rely on distillation of the frontier models and this is more efficient. But efficiencies aside; creation of open models still requires a lot of money an…

In the US and Europe its extremely expensive indeed. In China it is much lower and expected to be much lower in the future when China can bake their own high end chips. You know China is focussed on optimizing production and manufacturing. Unlike the west that primarily focus on regulations that make things super expensive.

Re: The state of open source AI

#267

Post by xyzsparetimexyz »

pocketarc wrote: post #188
Codefrontier wrote: post #183

Earlier quoted context omitted.

I agree with your speculation to be honest. And yet I’ve tried several local open weights model now and none gives the same quality of answers as Claude gives me on a regular Sonnet model. Mind you: I am “running” 48GB of RAM so I can’t try every model. Where does this difference come from? Can we actually get close locally?

You’re running 48GB now but imagine a future where everyone has 512GB RAM or 1TB RAM in their computers (it might sound like a lot but also 20 years ago we had 512MB PCs). It’s not hard to imagine what 5-10 years of pressure to increase RAM will do to specs, on top of the normal tech improvements. That’s worth bearing in mind when thinking about local models. Plus, local models keep getting better and better; 2 years…

I seriously doubt that we'll be running 1TB ram setups any time soon. I was hearing about people running 64GB back in 2014 or so

Re: The state of open source AI

#268

Post by root_axis »

JoeAltmaier wrote: post #259
root_axis wrote: post #257

Earlier quoted context omitted.

> But how many years do you guess? I personally do not think it will take even 10 years for the situation to be commonplace. IMO it won't be possible for the foreseeable future. There's essentially zero possibility that phones will gain the hardware capacity to run today's Kimi, so the only other alternative is to squeeze the power of today's Kimi into something that can fit on a smartphone, which also seems fairly u…

A bold prediction. Phones have gigabytes today. There are famous laws of growth that put terabytes at just a few years away - perhaps 10 isn't too bad an estimate?

> Phones have gigabytes today.

Phones have zero HBM today.

> There are famous laws of growth that put terabytes at just a few years away

I assume you're referring to Moore's law here, but if you are, it doesn't really apply to HBM in the same way, especially in a smartphone form factor where LPDDR is the only practical option due to heat and energy constraints, and a variety of architectural complexities specific to HBM that make a TB of it in a smart phone something far beyond what we can hope for in any timeline we can project today.

Re: The state of open source AI

#269

Post by andai »

andymatuschak wrote: post #241

The prose is, of course, LLM-generated. https://www.pangram.com/history/29a71663-e6b2-4db6-87bd-b943... I'd be curious to learn more about how/why executives come to sign their names to this kind of writing. Maybe it feels like a natural iteration of the pre-existing experience of signing one's name to an assistant's manuscript, or a press release from staff in Comms? I don't know execs who do this kind of thing, so…

It's not just a good idea — it's necessary.

Re: The state of open source AI

#270

Post by vitally3643 »

zdragnar wrote: post #201
vitally3643 wrote: post #178

Earlier quoted context omitted.

That's probably pretty likely, but if we're honest, are LLMs built and funded by a hostile Chinese authoritarian regime any more dangerous or harmful than LLMs built and funded by a hostile American authoritarian regime? China absolutely does not have my best interests at heart, but America's technofascism is probably more immediately dangerous and harmful. Americans genuinely have more to fear from America than Chin…

You're moving the goal post here. The parent never mentioned anything about the Chinese models themselves being dangerous or harmful; that's a totally different topic. The point they were making was that the most successful open models- those coming out of China- are made my companies that are using those open models to get exposure in Western markets. The goal is to undercut the Western dominant players, not out of…

You've failed to understand the conversation. Nobody doubts that China is using open weight models to undercut Western ones. That's pretty obvious.

The interesting discussion is why they are doing this and why it's "bad". The goalposts are right where they always were, you've failed to see them past your own feet.

Post reply on HN