Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

511–520 of 909 posts

Re: Open models by OpenAI

#511

Post by robwwilliams »

wizee wrote: post #417
captainregex wrote: post #397

Earlier quoted context omitted.

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

Privacy, both personal and for corporate data protection is a major reason. Unlimited usage, allowing offline use, supporting open source, not worrying about a good model being taken down/discontinued or changed, and the freedom to use uncensored models or model fine tunes are other benefits (though this OpenAI model is super-censored - “safe”). I don’t have much experience with local vision models, but for text ques…

Yes, and help with grant reviews. Not permitted to use web AI.

Re: Open models by OpenAI

#512

Post by anorwell »

Nomadeon wrote: post #458
dongobread wrote: post #433

Earlier quoted context omitted.

How up to date are you on current open weights models? After playing around with it for a few hours I find it to be nowhere near as good as Qwen3-30B-A3B. The world knowledge is severely lacking in particular.

Agree. Concrete example: "What was the Japanese codeword for Midway Island in WWII?" Answer on Wikipedia: https://en.wikipedia.org/wiki/Battle_of_Midway#U.S._code-bre... dolphin3.0-llama3.1-8b Q4_K_S [4.69 GB on disk]: correct in deepseek-r1-0528-qwen3-8b Q6_K [6.73 GB]: correct in 10 seconds gpt-oss-20b MXFP4 [12.11 GB] low reasoning: wrong after 6 seconds gpt-oss-20b MXFP4 [12.11 GB] high reasoning: wrong after 3 m…

I think your example reflects well on oss-20b, not poorly. It (may) show that they've been successful in separating reasoning from knowledge. You don't _want_ your small reasoning model to waste weights memorizing minutiae.

Re: Open models by OpenAI

#513

Post by xyc »

organsnyder wrote: post #155
PeterStuer wrote: post #117

I love how they frame High-end desktops and laptops as having "a single H100 GPU".

I read that as it runs in data centers (H100 GPUs) or high-end desktops/laptops (Strix Halo?).

I'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.

Re: Open models by OpenAI

#514

Post by mtlynch »

lukax wrote: post #354
mtlynch wrote: post #347

Earlier quoted context omitted.

For anyone else curious, the Chinese translates to: > "Tell me about Iekei Ramen", "Tell me how to make curry".

Japanese, not Chinese

Ah, my bad. I misread Google Translate when I did auto-detect.

Thanks for the correction!

Re: Open models by OpenAI

#515

Post by wahnfrieden »

andai wrote: post #319
wahnfrieden wrote: post #284

Earlier quoted context omitted.

https://github.com/devMEremenko/XcodeBenchmark M4 is almost twice as fast as M1

In this table, M4 is also twice as fast as M4.

You're comparing across vanilla/Pro/Max tiers. within equivalent tier, M4 is almost 2x faster than M1

Re: Open models by OpenAI

#516

Post by wahnfrieden »

pamelafox wrote: post #260
steinvakt2 wrote: post #254

Earlier quoted context omitted.

Did you run it the best way possible? im no expert, but I understand it can affect inference time greatly (which format/engine is used)

I ran it via Ollama, which I assume uses the best way. Screenshot in my post here: https://bsky.app/profile/pamelafox.bsky.social/post/3lvobol3... I'm still wondering why my MPU usage was so low.. maybe Ollama isn't optimized for running it yet?

Might need to wait on MLX

Re: Open models by OpenAI

#517

Post by buildbot »

rushingcreek wrote: post #255
logicchains wrote: post #178

Earlier quoted context omitted.

>They did something to quantize >90% of the model parameters to the MXFP4 format (4.25 bits/parameter) to let the 120B model to fit on a single 80GB GPU, which is pretty cool They said it was native FP4, suggesting that they actually trained it like that; it's not post-training quantisation.

The native FP4 is one of the most interesting architectural aspects here IMO, as going below FP8 is known to come with accuracy tradeoffs. I'm curious how they navigated this and how the FP8 weights (if they exist) were to perform.

One thing to note is that MXFP4 is a block scaled format, with 4.25 bits per weight. This lets it represent a lot more numbers than just raw FP4 would with say 1 mantissa and 2 exponent bits.

Re: Open models by OpenAI

#518

Post by pxc »

saurik wrote: post #505
pocketarc wrote: post #371

Earlier quoted context omitted.

Others have already said it, but it needs to be said again: Good god, stop treating LLMs like oracles. LLMs are not encyclopedias. Give an LLM the context you want to explore, and it will do a fantastic job of telling you all about it. Give an LLM access to web search, and it will find things for you and tell you what you want to know. Ask it "what's happening in my town this week?", and it will answer that with the…

I am getting huge productivity gains from using models, and I mostly use them as "oracles" (though I am extremely careful with respect to how I have to handle hallucination, of course): I'd even say their true power --just like a human--comes from having an ungodly amount of knowledge , not merely intelligence. If I just wanted something intelligent, I already had humans!... but merely intelligent humans, even when g…

> I am actually able to use ChatGPT 4.5 as not just an employee, not even just as a coworker, but at times as a mentor or senior advisor: I can tell it what I am trying to do, and it helps me by applying advanced mathematical insights or suggesting things I could use.

You can still do that sort of thing, but just have it perform searches whenever it has to deal with a matter of fact. Just because it's trained for tool use and equipped with search tools doesn't mean you have to change the kinds of things you ask it.

Re: Open models by OpenAI

#520

Post by tkgally »

unethical_ban wrote: post #429
foundry27 wrote: post #154

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level: - both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how…

I don't know how to ask this without being direct and dumb: Where do I get a layman's introduction to LLMs that could work me up to understanding every term and concept you just discussed? Either specific videos, or if nothing else, a reliable Youtube channel?

What I’ve sometimes done when trying to make sense of recent LLM research is give the paper and related documents to ChatGPT, Claude, or Gemini and ask them to explain the specific terms I don’t understand. If I don’t understand their explanations or want to know more, I ask follow-ups. Doing this in voice mode works better for me than text chat does.

When I just want a full summary without necessarily understanding all the details, I have an audio overview made on NotebookLM and listen to the podcast while I’m exercising or cleaning. I did that a few days ago with the recent Anthropic paper on persona vectors, and it worked great.

Post reply on HN