Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

391–400 of 909 posts

Re: Open models by OpenAI

#391

Post by metalliqaz »

strangecasts wrote: post #355
SV_BubbleTime wrote: post #329

Earlier quoted context omitted.

I wouldn’t stop at 16GB right now. 24 is the lowest I would go. Buy a used 3090. Picked one up for $700 a few months back, but I think they were on the rise then. The 3000 series can’t do FP8fast, but meh. It’s the OOM that’s tough, not the speed so much.

Are there any 24GB cards/3090s which fit in ~300mm without an angle grinder?

if you're going to get that kind of hardware, you need a larger case. IMHO this is not an unreasonable thing if you are doing heavy computing

Re: Open models by OpenAI

#392

Post by throw310822 »

herval wrote: post #382
datadrivenangel wrote: post #381

Earlier quoted context omitted.

Now to embrace jevon's paradox and expand usage until we're back to draining lakes so that your agentic refrigerator can simulate sentience.

In the future, your Samsung fridge will also need your AI girlfriend

In the future, while you're away your Samsung fridge will use electricity to chat up the Whirlpool washing machine.

Re: Open models by OpenAI

#393

Post by coltonv »

hrpnk wrote: post #387
coltonv wrote: post #374

Earlier quoted context omitted.

What did you set the context window to? That's been my main issue with models on my macbook, you have to set the context window so short that they are way less useful than the hosted models. Is there something I'm misisng there?

With LM Studio you can configure context window freely. Max is 131072 for gpt-oss-20b.

Yes but if I set it above ~16K on my 32gb laptop it just OOMs. Am I doing something wrong?

Re: Open models by OpenAI

#394

Post by syntaxing »

cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

Interesting, these models are better than the new Qwen releases?

Re: Open models by OpenAI

#395

Post by satyrun »

Workaccount2 wrote: post #11

Wow, today is a crazy AI release day: - OAI open source - Opus 4.1 - Genie 3 - ElevenLabs Music

wow I just listened to Eleven Music do flamenco singing. That is incredible.

Edit. I just tried it though and less impressed now. We are really going to need major music software to get on board before we have actual creative audio tools. These all seem made for non-musicians to make a very cookie cutter song from a specific genre.

Re: Open models by OpenAI

#396

Post by syntaxing »

black3r wrote: post #383
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

can you please give an estimate how much slower/faster is it on your macbook compared to comparable models running in the cloud?

You can get a pretty good estimate depending on your memory bandwidth. Too many parameters can change with local models (quantization, fast attention, etc). But the new models are MoE so they’re gonna be pretty fast.

Re: Open models by OpenAI

#397

Post by captainregex »

cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding?

Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger ones

It feels like we do it because we can more than because it makes sense- which I am all for! I just wonder if i’m missing some kind of major use case all around me that justifies chaining together a bunch of mac studios or buying a really great graphics card. Tools like exo are cool and the idea of distributed compute is neat but what edge cases truly need it so badly that it’s worth all the effort?

Re: Open models by OpenAI

#398

Post by AtlasBarfed »

renmillar wrote: post #246
ClassAndBurn wrote: post #158

Open models are going to win long-term. Anthropics' own research has to use OSS models [0]. China is demonstrating how quickly companies can iterate on open models, allowing smaller teams access and augmentation to the abilities of a model without paying the training cost. My personal prediction is that the US foundational model makers will OSS something close to N-1 for the next 1-3 iterations. The CAPEX for the fou…

There's no reason that models too large for consumer hardware wouldn't keep a huge edge, is there?

That is fundamentally a big O question.

I have this theory that we simply got over a hump by utilizing a massive processing boost from gpus as opposed to CPUs. That might have been two to three orders of magnitude more processing power.

But that's a one-time success. I don't hardware has any large scale improvements coming, because 3D gaming mostly plumb most of that vector processing hardware development in the last 30 years.

So will software and better training models produce another couple orders of magnitude?

Fundamentally we're talking about nines of of accuracy. What is the processing power required for each line of accuracy? Is it linear? Is it polynomial? Is it exponential?

It just seems strange to me with all the AI knowledge slushing through academia, I haven't seen any basic analysis at that level, which is something that's absolutely going to be necessary for AI applications like self-driving, once you get those insurance companies involved

Re: Open models by OpenAI

#399

Post by Metacelsus »

Running ollama on my M3 Macbook, gpt-oss-20b gave me detailed instructions for how to give mice cancer using an engineered virus.

Of course this could also give humans cancer. (To the OpenAI team's slight credit, when asked explicitly about this, the model refused.)

Re: Open models by OpenAI

#400

Post by throw310822 »

asadm wrote: post #353
highfrequency wrote: post #349

Earlier quoted context omitted.

I would guess the “secret sauce” here is distillation: pretraining on an extremely high quality synthetic dataset from the prompted output of their state of the art models like o3 rather than generic internet text. A number of research results have shown that highly curated technical problem solving data is unreasonably effective at boosting smaller models’ performance. This would be much more efficient than relying…

> research results have shown that highly curated technical problem solving data is unreasonably effective at boosting smaller models’ performance. same seems to be true for humans

Yes, if I understand correctly, what it means is "a very smart teacher can do wonders for their pupils' education".
Post reply on HN