Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

321–330 of 909 posts

Re: Open models by OpenAI

#321

Post by GHanku »

strangecasts wrote: post #139
minimaxir wrote: post #20

I'm disappointed that the smallest model size is 21B parameters, which strongly restricts how it can be run on personal hardware. Most competitors have released a 3B/7B model for that purpose. For self-hosting, it's smart that they targeted a 16GB VRAM config for it since that's the size of the most cost-effective server GPUs, but I suspect "native MXFP4 quantization" has quality caveats.

A small part of me is considering going from a 4070 to a 16GB 5060 Ti just to avoid having to futz with offloading I'd go for an ..80 card but I can't find any that fit in a mini-ITX case :(

[dead]

Re: Open models by OpenAI

#322

Post by SV_BubbleTime »

MutedEstate45 wrote: post #105
BoorishBears wrote: post #46

Earlier quoted context omitted.

I think it's pointless: if you SFT even their closed source models on a specific enough task, the guardrails disappear. AI "safety" is about making it so that a journalist can't get out a recipe for Tabun just by asking.

True, but there's still a meaningful difference in friction and scale. With closed APIs, OpenAI can monitor for misuse, throttle abuse and deploy countermeasures in real-time. With open weights, a single prompt jailbreak or exploit spreads instantly. No need for ML expertise, just a Reddit post. The risk isn’t that bad actors suddenly become smarter. It’s that anyone can now run unmoderated inference and OpenAI loses…

Given that the best jailbreak for an off-line model is still simple prompt injection, which is a solved issue for the closed source models… I honestly don’t know why they are talking about safety much at all for open source.

Re: Open models by OpenAI

#323

Post by SV_BubbleTime »

greenavocado wrote: post #144
ebiester wrote: post #22

Earlier quoted context omitted.

Honestly, it's a tradeoff. If you can reduce the size and make a higher quality in specific tasks, that's better than a generalist that can't run on a laptop or can't compete at any one task. We will know soon the actual quality as we go.

That's what I thought too until Qwen-Image was released

When Queen-Image was released… like yesterday? And what? What point are you making? QwebImage was released yesterday and like every image model, its base model shows potential over older ones but the real factor is will it be flexible enough for a fine tune or additional training Loras.

Re: Open models by OpenAI

#324

Post by [deleted] »

siliconc0w wrote: post #210

It seems like OSS will win, I can't see people willing to pay like 10x the price for what seems like 10% more performance. Especially once we get better at routing the hardest questions to the better models and then using that response to augment/fine-tune the OSS ones.

[deleted]

Re: Open models by OpenAI

#325

Post by roboyoshi »

pamelafox wrote: post #96

Anyone tried running on a Mac M1 with 16GB RAM yet? I've never run higher than an 8GB model, but apparently this one is specifically designed to work well with 16 GB of RAM.

M2 with 16GB: It's slow for me. ~13GB RAM usage, not locking up my mac, but took a very long time thinking and slowly outputting tokens.. I'd not consider this usable for everyday usage.

Re: Open models by OpenAI

#326

Post by SV_BubbleTime »

DSingularity wrote: post #6

Ha. Secure funding and proceed to immediately make a decision that would likely conflict viscerally with investors.

Undercutting other frontier models with your open source one is not an anti-investor move.

It is what China has been doing for a year plus now. And the Chinese models are popular and effective, I assume companies are paying for better models.

Releasing open models for free doesn’t have to be charity.

Re: Open models by OpenAI

#327

Post by hmottestad »

dust42 wrote: post #316

The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France). Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).

I don’t think they trained it for fact retrieval.

Would probably do a lot better if you give it tool access for search and web browsing.

Re: Open models by OpenAI

#328

Post by SV_BubbleTime »

tempay wrote: post #81
hubraumhugo wrote: post #16

Meta's goal with Llama was to target OpenAI with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape. Looks like OpenAI is now using the same playbook.

It seems like the various Chinese companies are far outplaying Meta at that game. It remains to be seen if they’re able to throw money at the problem to turn things around.

Good move for China. No one was going to trust their models outright, now they not only have a track record, but they were able to undercut the value of US models at the same time.

Re: Open models by OpenAI

#329

Post by SV_BubbleTime »

strangecasts wrote: post #139
minimaxir wrote: post #20

I'm disappointed that the smallest model size is 21B parameters, which strongly restricts how it can be run on personal hardware. Most competitors have released a 3B/7B model for that purpose. For self-hosting, it's smart that they targeted a 16GB VRAM config for it since that's the size of the most cost-effective server GPUs, but I suspect "native MXFP4 quantization" has quality caveats.

A small part of me is considering going from a 4070 to a 16GB 5060 Ti just to avoid having to futz with offloading I'd go for an ..80 card but I can't find any that fit in a mini-ITX case :(

I wouldn’t stop at 16GB right now.

24 is the lowest I would go. Buy a used 3090. Picked one up for $700 a few months back, but I think they were on the rise then.

The 3000 series can’t do FP8fast, but meh. It’s the OOM that’s tough, not the speed so much.

Re: Open models by OpenAI

#330

Post by aubanel »

_ache_ wrote: post #309
timmg wrote: post #112

Orthogonal, but I just wanted to say how awesome Ollama is. It took 2 seconds to find the model and a minute to download and now I'm using it. Kudos to that team.

To be fair, it's with the help of OpenAI. They did it together, before the official release. https://ollama.com/blog/gpt-oss

From experience, it's much more engineering work on the integrator's side than on OpenAI's. Basically they provide you their new model in advance, but they don't know the specifics of your system, so it's normal that you do most of the work. Thus I'm particularly impressed by Cerebras: they only have a few models supported for their extreme perf inference, it must have been huge bespoke work to integrate.
Post reply on HN