Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

331–340 of 909 posts

Re: Open models by OpenAI

#331

Post by d3m0t3p »

mlyle wrote: post #59
artembugara wrote: post #14

Disclamer: probably dumb questions so, the 20b model. Can someone explain to me what I would need to do in terms of resources (GPU, I assume) if I want to run 20 concurrent processes, assuming I need 1k tokens/second throughput (on each, so 20 x 1k) Also, is this model better/comparable for information extraction compared to gpt-4.1-nano, and would it be cheaper to host myself 20b?

An A100 is probably 2-4k tokens/second on a 20B model with batched inference. Multiply the number of A100's you need as necessary. Here, you don't really need the ram. If you could accept fewer tokens/second, you could do it much cheaper with consumer graphics cards. Even with A100, the sweet-spot in batching is not going to give you 1k/process/second. Of course, you could go up to H100...

You can batch only if you have distinct chat in parallel,

Re: Open models by OpenAI

#332

Post by andai »

irthomasthomas wrote: post #301

I was hoping these were the stealth Horizon models on OpenRouter, impressive but not quite GPT-5 level. My bet: GPT-5 leans into parallel reasoning via a model consortium, maybe mixing in OSS variants. Spin up multiple reasoning paths in parallel, then have an arbiter synthesize or adjudicate. The new Harmony prompt format feels like infrastructural prep: distinct channels for roles, diversity, and controlled aggrega…

Fascinating, thanks for sharing. Are there any specific kind of problems you find this helps with?

I've found that LLMs can handle some tasks very well and some not at all. For the ones they can handle well, I optimize for the smallest, fastest, cheapest model that can handle it. (e.g. using Gemini Flash gave me a much better experience than Gemini Pro due to the iteration speed.)

This "pushing the frontier" stuff would seem to help mostly for the stuff that are "doable but hard/inconsistent" for LLMs, and I'm wondering what those tasks are.

Re: Open models by OpenAI

#333

Post by Invictus0 »

hmottestad wrote: post #327
dust42 wrote: post #316

The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France). Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).

I don’t think they trained it for fact retrieval. Would probably do a lot better if you give it tool access for search and web browsing.

What is the point of an offline reasoning model that also doesn't know anything and makes up facts? Why would anyone prefer this to a frontier model?

Re: Open models by OpenAI

#334

Post by chilipepperhott »

lukax wrote: post #219

Inference in Python uses harmony [1] (for request and response format) which is written in Rust with Python bindings. Another OpenAI's Rust library is tiktoken [2], used for all tokenization and detokenization. OpenAI Codex [3] is also written in Rust. It looks like OpenAI is increasingly adopting Rust (at least for inference). [1] https://github.com/openai/harmony [2] https://github.com/openai/tiktoken [3] https://g…

As an engineer that primarily uses Rust, this is a good omen.

Re: Open models by OpenAI

#335

Post by Invictus0 »

gnulinux wrote: post #79
sigmar wrote: post #64

Earlier quoted context omitted.

Non-rhetorically, why would someone pay for o3 api now that I can get this open model from openai served for cheaper? Interesting dynamic... will they drop o3 pricing next week (which is 10-20x the cost[1])? [1] currently $3M in/ $8M out https://platform.openai.com/docs/pricing

Not even that, even if o3 being marginally better is important for your task (let's say) why would anyone use o4-mini? It seems almost 10x the price and same performance (maybe even less): https://openrouter.ai/openai/o4-mini

Probably because they are going to announce gpt 5 imminently

Re: Open models by OpenAI

#336

Post by albertgoeswoof »

jcmontx wrote: post #182

I'm out of the loop for local models. For my M3 24gb ram macbook, what token throughput can I expect? Edit: I tried it out, I have no idea in terms of of tokens but it was fluid enough for me. A bit slower than using o3 in the browser but definitely tolerable. I think I will set it up in my GF's machine so she can stop paying for the full subscription (she's a non-tech professional)

3 year old M1 MacBook Pro 32gb, 42 tokens/sec on lm studio

Very much usable

Re: Open models by OpenAI

#337

Post by numpad0 »

Here's a pair of quick sanity check questions I've been asking LLMs: "家系ラーメンについて教えて", "カレーの作り方教えて". It's a silly test but surprisingly many fails at it - and Chinese models are especially bad with it. The commonalities between models doing okay-ish for these questions seem to be Google-made OR >70b OR straight up commercial(so >200B or whatever).

I'd say gpt-oss-20b is in between Qwen3 30B-A3B-2507 and Gemma 3n E4b(with 30B-A3B at lower side). This means it's not obsoleting GPT-4o-mini for all purposes.

Re: Open models by OpenAI

#338

Post by simonw »

Just posted my initial impressions, took a couple of hours to write them up because there's a lot in this release! https://simonwillison.net/2025/Aug/5/gpt-oss/

TLDR: I think OpenAI may have taken the medal for best available open weight model back from the Chinese AI labs. Will be interesting to see if independent benchmarks resolve in that direction as well.

The 20B model runs on my Mac laptop using less than 15GB of RAM.

Re: Open models by OpenAI

#339

Post by Invictus0 »

Shank wrote: post #271
rrrrrrrrrrrryan wrote: post #180

Earlier quoted context omitted.

It seems like a big part of GPT-5 will be that it will be able to intelligently route your request to the appropriate model variant.

That doesn’t sound good. It sounds like OpenAI will route my request to the cheapest model to them and the most expensive for me, with the minimum viable results.

Sounds just like what a human would do. Or any business for that matter.

Re: Open models by OpenAI

#340

Post by NitpickLawyer »

jlokier wrote: post #287
NitpickLawyer wrote: post #137

Earlier quoted context omitted.

It's apache2.0, so by definition it's open source. Stop pushing for training data, it'll never happen, and there's literally 0 reason for it to happen (both theoretical and practical). Apache2.0 IS opensource.

> It's apache2.0, so by definition it's open source. That's not true by any of the open source definitions in common use. Source code (and, optionally, derived binaries) under the Apache 2.0 license are open source. But compiled binaries (without access to source) under the Apache 2.0 license are not open source, even though the license does give you some rights over what you can do with the binaries. Normally the qu…

Please see the detailed response to a sibling post. tl;dr; weights are not binaries.
Post reply on HN