Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

421–430 of 909 posts

Re: Open models by OpenAI

#421

Post by superconduct123 »

lukev wrote: post #343
dust42 wrote: post #316

The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France). Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).

This is precisely the wrong way to think about LLMs. LLMs are never going to have fact retrieval as a strength. Transformer models don't store their training data: they are categorically incapable of telling you where a fact comes from. They also cannot escape the laws of information theory: storing information requires bits. Storing all the world's obscure information requires quite a lot of bits. What we want out o…

How can you reason correctly if you don't have any way to know which facts are real vs hallucinated?

Re: Open models by OpenAI

#422

Post by cco »

datadrivenangel wrote: post #381
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

Now to embrace jevon's paradox and expand usage until we're back to draining lakes so that your agentic refrigerator can simulate sentience.

What ~IBM~ TSMC giveth, ~Bill Gates~ Sam Altman taketh away.

Re: Open models by OpenAI

#424

Post by strangecasts »

metalliqaz wrote: post #391
strangecasts wrote: post #355

Earlier quoted context omitted.

Are there any 24GB cards/3090s which fit in ~300mm without an angle grinder?

if you're going to get that kind of hardware, you need a larger case. IMHO this is not an unreasonable thing if you are doing heavy computing

Noted for my next build - I am aware this is a problem I've made for myself, otherwise I like the mini-ITX form factor a lot

Re: Open models by OpenAI

#425

Post by captainregex »

unethical_ban wrote: post #413
captainregex wrote: post #397

Earlier quoted context omitted.

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

Privacy and equity. Privacy is obvious. AI is going to to be equivalent to all computing in the future. Imagine if only IBM, Apple and Microsoft ever built computers, and all anyone else ever had in the 1990s were terminals to the mainframe, forever.

I am all for the privacy angle and while I think there’s certainly a group of us, myself included, who care deeply about it I don’t think most people or enterprises will. I think most of those will go for the easy button and then wring their hands about privacy and security as they have always done while continuing to let the big companies do pretty much whatever they want. I would be so happy to be wrong but aren’t we already seeing it? Middle of the night price changes, leaks of data, private things that turned out to not be…and yet!

Re: Open models by OpenAI

#426

Post by futureshock »

lukev wrote: post #343
dust42 wrote: post #316

The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France). Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).

This is precisely the wrong way to think about LLMs. LLMs are never going to have fact retrieval as a strength. Transformer models don't store their training data: they are categorically incapable of telling you where a fact comes from. They also cannot escape the laws of information theory: storing information requires bits. Storing all the world's obscure information requires quite a lot of bits. What we want out o…

I think that sounds very reasonable, but unfortunately these models don’t know what they know and don’t. A small model that knew the exact limits of its knowledge would be very powerful.

Re: Open models by OpenAI

#427

Post by n_f »

There's something so mind-blowing about being able to run some code on my laptop and have it be able to literally talk to me. Really excited to see what people can build with this

Re: Open models by OpenAI

#428

Post by captainregex »

canvascritic wrote: post #404
captainregex wrote: post #397

Earlier quoted context omitted.

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

Healthcare organizations that can't (easily) send data over the wire while remaining in compliance Organizations operating in high stakes environments Organizations with restrictive IT policies To name just a few -- well, the first two are special cases of the last one RE your hallucination concerns: the issue is overly broad ambitions. Local LLMs are not general purpose -- if what you want is local ChatGPT, you will…

Aren’t there HIPPA compliant clouds? I thought Azure had an offer to that effect and I imagine that’s the type of place they’re doing a lot of things now. I’ve landed roughly where you have though- text stuff is fine but don’t ask it to interact with files/data you can’t copy paste into the box. If a user doesn’t care to go through the trouble to preserve privacy, and I think it’s fair to say a lot of people claim to care but their behavior doesn’t change, then I just don’t see it being a thing people bother with. Maybe something to use offline while on a plane? but even then I guess United will have Starlink soon so plane connectivity is gonna get better

Re: Open models by OpenAI

#429

Post by unethical_ban »

foundry27 wrote: post #154

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level: - both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how…

I don't know how to ask this without being direct and dumb: Where do I get a layman's introduction to LLMs that could work me up to understanding every term and concept you just discussed? Either specific videos, or if nothing else, a reliable Youtube channel?
Post reply on HN