Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-weight AI is having its Kubernetes moment

tobi.knaup.me

Post reply on HN
Print view View original

81–90 of 346 posts

Re: Open-weight AI is having its Kubernetes moment

#81

Post by DrewADesign »

gwbrooks wrote: post #30
julianlam wrote: post #9

Earlier quoted context omitted.

Why do prescription medications cost so much, and generics so little (comparatively)? Artificial inflation to recoup R&D.

Not sure pricing to recoup costs is artificial.

I think the argument is that artificial comes in with IP law, which some people feel is superfluous.

I do think that corporate price gouging is a huge problem that does need to be addressed. But especially with smaller business types — creatives, et al— I still haven’t gotten any grownup answers about what would compel people to get professionally good at something and innovate in the complete absence of copyright: the vastly better business model would be waiting for someone else to do something new and interesting, stealing their work, and then undercutting them in the market because you don’t have R&D/et al costs to recoup. You can’t say that wouldn’t happen because it’s exactly what the AI companies did to billions of people, scoffing at any protest. And ironically, they’re now whining about the Chinese doing it to them.

Re: Open-weight AI is having its Kubernetes moment

#82

Post by chasd00 »

FTFA: American labs need to release frontier-grade open-weight models under licenses that startups can actually build on.

oh now i see, the Chinese government is funding the training and release of their best models to pressure OpenAI, Anthropic, and others to do the same for competition's sake. I don't buy it, this seems more like a way to get SOTA models RL'd to comply with Chinese government approved information distribution. If I have to trust a black box of answers to questions i would trust one from a US for-profit publicly traded company subject to market forces over one approved, and heavily subsidized, by the Chinese government.

Re: Open-weight AI is having its Kubernetes moment

#83

Post by jdub »

yard2010 wrote: post #75
xyzsparetimexyz wrote: post #11

Earlier quoted context omitted.

I still don't know what it is tbh. Something for docker?

I was in the same boat as you until I needed to learn how to use it in my $dayjob. It was like discovering a new continent. I couldn't care less about it before, but the moment I realized it's a kind of cloud OS I was astounded by how this thing is genius. It's the kind of thing that gives you dopamine rushes when you use it. Something about how there is a solution to every problem you didn't know even mattered turns…

This is the initial endorphin rush you get when wielding a complex system. The feeling changes when the complexity explodes in your face.

Re: Open-weight AI is having its Kubernetes moment

#84

Post by mlyle »

spwa4 wrote: post #17
firasd wrote: post #2

One of the strangest things in the AI industry is 'tokenomics'. It's not very clear why using GPT-4 in early 2023 was so expensive and then six months later 20 bucks could get you a fair amount of GPT-4 inference. This pattern has continued across various labs/providers for years--there is a continuous see-saw of pricing that doesn't seem related to anything. So what open weight models do is at least provide a baseli…

Because as per usual it's silicon valley misunderstanding economics. AI is HPC. And how the HPC market worked before: If you're the best performing "computing cluster" (ie. whatever you call the entity that can complete a massive calculation), you get a blank check from Congress. Why? Because you need those calculations to "pump" nuclear weapons. They are needed to calculate both the geometry to make fusion bombs pos…

> Because you need those calculations to "pump" nuclear weapons. They are needed to calculate both the geometry to make fusion bombs possible at all and to calculate the effect of a given geometry. They are the reason US/Russia/China have the biggest and strongest weapons known to humanity.

We have a couple new nuclear weapon designs, but not really going for bigger or stronger. Just packaging.

We built the big powerful ones with 1960s computing.

Now, stockpile stewardship -- being sure that stuff will keep working without ongoing testing -- is a bit expensive in compute. You need early 2010s supercomputer power.

In other words, I strongly disagree that nuclear weapons are the primary driver of high-end compute.

Re: Open-weight AI is having its Kubernetes moment

#85

Post by mountainriver »

firasd wrote: post #2

One of the strangest things in the AI industry is 'tokenomics'. It's not very clear why using GPT-4 in early 2023 was so expensive and then six months later 20 bucks could get you a fair amount of GPT-4 inference. This pattern has continued across various labs/providers for years--there is a continuous see-saw of pricing that doesn't seem related to anything. So what open weight models do is at least provide a baseli…

Quantization also started picking up around then, as well as distillation into smaller models

Re: Open-weight AI is having its Kubernetes moment

#86

Post by imachine1980_ »

Aurornis wrote: post #74
firasd wrote: post #2

One of the strangest things in the AI industry is 'tokenomics'. It's not very clear why using GPT-4 in early 2023 was so expensive and then six months later 20 bucks could get you a fair amount of GPT-4 inference. This pattern has continued across various labs/providers for years--there is a continuous see-saw of pricing that doesn't seem related to anything. So what open weight models do is at least provide a baseli…

> It's not very clear why using GPT-4 in early 2023 was so expensive and then six months later 20 bucks could get you a fair amount of GPT-4 inference. The price is what the market is willing to bear for the available compute capacity and competitive landscape. You can only discover that price after trying different price points and seeing what happens. Everyone is trying different pricing schemes and discounts as th…

Most unmature markets aren't subsidized to the point that LLM market is, most market have some level of baseline profitablity, this market doesn't, that's because most market subsidized the marketing or the capex but this market doesn't hold the opex, the capex not the amount of marketing let alone all of this together

Re: Open-weight AI is having its Kubernetes moment

#87

Post by aleph_minus_one »

cheriot wrote: post #73

Open-weight and OSS are wildly different and the article makes a poor comparison. What's the incentive for the Chinese labs to continue releasing weights 5 years from now? It's not a stable equilibrium and cannot last. - The lab spending large sums on research and training does not get the inference revenue to fund those efforts. - Unlike OSS where a single volunteer can keep a project going, training costs run into…

> - Unlike OSS where a single volunteer can keep a project going, training costs run into the $billions.

Just some thought: Wouldn't it make sense to build some kind of volunteer computing project to train the next-generation LLM by volunteers, similar to the BOINC [1] projects or Folding@home [2]?

N.B.: BOINC was particularly famous for SETI@home (completed), Einstein@Home, Rosetta@home and PrimeGrid.

I still remember the time when Einstein@Home was in its heyday, and many people who loved putting together fast PCs contributed sometimes even for the reason of showing off in the statistics [3].

---

[1] https://en.wikipedia.org/wiki/Berkeley_Open_Infrastructure_f...

[2] https://en.wikipedia.org/wiki/Folding@home

[3] https://einsteinathome.org/de/community/stats

Re: Open-weight AI is having its Kubernetes moment

#88

Post by aliasxneo »

chasd00 wrote: post #82

FTFA: American labs need to release frontier-grade open-weight models under licenses that startups can actually build on. oh now i see, the Chinese government is funding the training and release of their best models to pressure OpenAI, Anthropic, and others to do the same for competition's sake. I don't buy it, this seems more like a way to get SOTA models RL'd to comply with Chinese government approved information d…

What tools do we have to countermeasure the state sponsored bias in the Chinese models? Doesn’t seem like a smart plan if individuals can just compensate for the bias.

Re: Open-weight AI is having its Kubernetes moment

#89

Post by nicce »

sschueller wrote: post #72
amazingamazing wrote: post #44

Sadly until china scales production of hardware it really isn’t economical to run this stuff yourself. It is good it exists though to put pressure against the labs. Honestly imo this is just proof apple will win in the end. Eventually a phone will be able to run a model good enough to do most things and it then is game over.

Model-on-Chip is coming. GPU are for general computing but have a huge bottle neck for doing model inference. Even not being able to significantly update a model that is burned on a chip the performance gains are immense. You also don't need the latest chip fabs to make them drastically reducing the cost.

Many years until consumers can buy them at reasonable price. Nvdia and AMD are making GPUs bad in purpose for consumers so that nobody can build a datacenter from them. It will take a long time.

Re: Open-weight AI is having its Kubernetes moment

#90

Post by RobRivera »

firasd wrote: post #2

One of the strangest things in the AI industry is 'tokenomics'. It's not very clear why using GPT-4 in early 2023 was so expensive and then six months later 20 bucks could get you a fair amount of GPT-4 inference. This pattern has continued across various labs/providers for years--there is a continuous see-saw of pricing that doesn't seem related to anything. So what open weight models do is at least provide a baseli…

Market discovery
Post reply on HN