Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

791–800 of 909 posts

Re: Open models by OpenAI

#791

Post by amalcon »

ben_w wrote: post #742
cpursley wrote: post #733

Earlier quoted context omitted.

You’re in a bubble. It was no surprise to folks who touch grass on the regular.

> You’re in a bubble. Sure, all I have to go on from the other side of the Atlantic is the internet. So in that regard, kinda like the AI. One of the big surprises from the POV of me in Jan 2024, is that I would have anticipated Trump being in prison and not even available as an option for the Republican party to select as a candidate for office, and that even if he had not gone to jail that the Republicans would not…

I mean, the presumptive GOP primary candidates at the time were Trump, Trump-lite (DeSantis), about 10 Trump sycophants, and Haley. He had demonstrated a high level of influence over GOP primary voters in the 2022 midterm. It had been (internally) obvious since at least then that he was going to win the primary again. I can't speak to how much of that made it across the Atlantic.

Whether he would win the general was an open question then. In the American system, your prediction should never get very far from a coin flip a year out.

Re: Open models by OpenAI

#792

Post by gorbypark »

mark_l_watson wrote: post #766
wizee wrote: post #417

Earlier quoted context omitted.

Privacy, both personal and for corporate data protection is a major reason. Unlimited usage, allowing offline use, supporting open source, not worrying about a good model being taken down/discontinued or changed, and the freedom to use uncensored models or model fine tunes are other benefits (though this OpenAI model is super-censored - “safe”). I don’t have much experience with local vision models, but for text ques…

I agree totally. My only problem is local models running on my old macMini run very much slower than that for example Gemini-2.5-flash. I have my Emacs setup so I can switch between a local model and one of the much faster commercial models. Someone else responded to you about working for a financial organization and not using public APIs - another great use case.

These being mixture of expert (MOE) models should help. The 20b model only has 3.6b params active at any one time, so minus a bit of overhead the speed should be like running a 3.6b model (while still requiring the RAM of a 20b model).

Here's the ollama version (4.6bit quant, I think?) run with --verbose total duration: 21.193519667s load duration: 94.88375ms prompt eval count: 77 token(s) prompt eval duration: 1.482405875s prompt eval rate: 51.94 tokens/s eval count: 308 token(s) eval duration: 19.615023208s eval rate: 15.70 tokens/s

15 tokens/s is pretty decent for a low end MacBook Air (M2, 24gb of ram). Yes, it's not the ~250 tokens/s of 2.5-flash, but for my use case anything above 10 tokens/sec is good enough.

Re: Open models by OpenAI

#793

Post by yreg »

captainregex wrote: post #397
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

> I’m still trying to understand what is the biggest group of people that will use local AI?

iPhone users in a few months – because I predict app developers will love cramming calls to the foundation models into everything.

Android will follow.

Re: Open models by OpenAI

#794

Post by CrackerNews »

lukev wrote: post #343
dust42 wrote: post #316

The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France). Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).

This is precisely the wrong way to think about LLMs. LLMs are never going to have fact retrieval as a strength. Transformer models don't store their training data: they are categorically incapable of telling you where a fact comes from. They also cannot escape the laws of information theory: storing information requires bits. Storing all the world's obscure information requires quite a lot of bits. What we want out o…

What are the large context, strong reasoning, and linguistic facility for if there aren't facts underpinning them? Is a priori wholly independent of a posteriori? Is it practical for the former to be wholly independent of the latter?

Re: Open models by OpenAI

#795

Post by CrackerNews »

pocketarc wrote: post #371
dust42 wrote: post #316

The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France). Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).

Others have already said it, but it needs to be said again: Good god, stop treating LLMs like oracles. LLMs are not encyclopedias. Give an LLM the context you want to explore, and it will do a fantastic job of telling you all about it. Give an LLM access to web search, and it will find things for you and tell you what you want to know. Ask it "what's happening in my town this week?", and it will answer that with the…

LLMs should at least -know- the semantics about the text it analyzed as opposed to the syntax.

Re: Open models by OpenAI

#796

Post by FrustratedMonky »

username332211 wrote: post #777
ben_w wrote: post #742

Earlier quoted context omitted.

> You’re in a bubble. Sure, all I have to go on from the other side of the Atlantic is the internet. So in that regard, kinda like the AI. One of the big surprises from the POV of me in Jan 2024, is that I would have anticipated Trump being in prison and not even available as an option for the Republican party to select as a candidate for office, and that even if he had not gone to jail that the Republicans would not…

I'm pretty sure you are completely correct on the last part. Nobody in Republican management wanted a second Trump term. If the candidate wasn't Trump, Republicans would have had a guaranteed victory. Imagine that infamous debate, but with some 50-year-old youngster facing Joe Biden. It's the White House that wanted Trump to be candidate. They played Republican primary voters like a fiddle by launching a barrage of t…

You think the Democratic White House, manipulated Republicans into Voting for Trump. So it is the Democrats fault we have Trump??? Next Level Cope.

Re: Open models by OpenAI

#797

Post by dragonwriter »

9rx wrote: post #782
mark_l_watson wrote: post #764

Earlier quoted context omitted.

I think the lesson is: smaller models hallucinate more, so only use them in your applications where you load up large prompts with specific data to reason about. Then even the small Google gemma3n 4B model can be amazingly useful. I use the SOTA models from Google and OpenAI mostly for getting feedback on ideas, helping me think through designs, and sometimes for coding. Your question is clearly best answered using a…

> think the lesson is: smaller models hallucinate more The interesting part isn't the hallucination, but the sheer unwillingness to take in new information.

Might have dug in less on the confabulation about information outside of its knowledge cutoff if the new information weren't offered with support from a user “hallucination” about information within its knowledge cutoff. More detail:

https://news.ycombinator.com/item?id=44809145

Re: Open models by OpenAI

#798

Post by cwyers »

unethical_ban wrote: post #429
foundry27 wrote: post #154

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level: - both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how…

I don't know how to ask this without being direct and dumb: Where do I get a layman's introduction to LLMs that could work me up to understanding every term and concept you just discussed? Either specific videos, or if nothing else, a reliable Youtube channel?

This is a great book (parts of it are available as blog posts from the author if you want to get a taste of it):

https://www.manning.com/books/build-a-large-language-model-f...

Re: Open models by OpenAI

#799

Post by iamnotagenius »

nojito wrote: post #494
dongobread wrote: post #433

Earlier quoted context omitted.

How up to date are you on current open weights models? After playing around with it for a few hours I find it to be nowhere near as good as Qwen3-30B-A3B. The world knowledge is severely lacking in particular.

Why does it need knowledge when it can just call tools to get it?

Try to push your point to absurd you see why; hint - to analyze data pulled by tools you need knowledge already baked in. You have very limited context, you cannot just pull and pull data.

Re: Open models by OpenAI

#800

Post by 9rx »

dragonwriter wrote: post #797
9rx wrote: post #782

Earlier quoted context omitted.

> think the lesson is: smaller models hallucinate more The interesting part isn't the hallucination, but the sheer unwillingness to take in new information.

Might have dug in less on the confabulation about information outside of its knowledge cutoff if the new information weren't offered with support from a user “hallucination” about information within its knowledge cutoff. More detail: https://news.ycombinator.com/item?id=44809145

> Might have dug in less...

The digging in at all is what is interesting. Like an earlier comment alluded to, the presumptive value of these tools is being able to feed it your own information where that information is to be considered authoritative.

> More detail: [...] "He, in fact, did run for reelection"

A slow walk, maybe. He was in no condition to run. That is why he ultimately dropped out. But, really, that statement is just a silly game of semantics. "Run", when used in hindsight, often implies completion. This LLM model even says so too. If a model doesn't understand nuance, that is also interesting.

Post reply on HN