Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

471–480 of 909 posts

Re: Open models by OpenAI

#471

Post by SchemaLoad »

a_wild_dandan wrote: post #62
x187463 wrote: post #23

Running a model comparable to o3 on a 24GB Mac Mini is absolutely wild. Seems like yesterday the idea of running frontier (at the time) models locally or on a mobile device was 5+ years out. At this rate, we'll be running such models in the next phone cycle.

Right? I still remember the safety outrage of releasing Llama. Now? My 96 GB of (V)RAM MacBook will be running a 120B parameter frontier lab model. So excited to get my hands on the MLX quants and see how it feels compared to GLM-4.5-air.

I feel like most of the safety concerns ended up being proven correct, but there's so much money in it that they decided to push on anyway full steam ahead.

AI did get used for fake news, propaganda, mass surveillance, erosion of trust and sense of truth, and mass spamming social media.

Re: Open models by OpenAI

#472

Post by m463 »

captainregex wrote: post #397
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

One use nobody mentions is hybrid use.

Why not run all the models at home, maybe collaboratively or at least in parallel?

I'm sure there are use cases where the paid models are not allowed to collaborate or ask each other.

also, other open models are gaining mindshare.

Re: Open models by OpenAI

#473

Post by resters »

Reading the comments it becomes clear how befuddled many HN participants are about AI. I don't think there has been a technical topic that HN has seemed so dull on in the many years I've been reading HN. This must be an indication that we are in a bubble.

One basic point that is often missed is: Different aspects of LLM performance (in the cognitive performance sense) and LLM resource utilization are relevant to various use cases and business models.

Another is that there are many use cases where users prefer to run inference locally, for a variety of domain-specific or business model reasons.

The list goes on.

Re: Open models by OpenAI

#474

Post by dankwizard »

pocketarc wrote: post #371
dust42 wrote: post #316

The 120B model badly hallucinates facts on the level of a 0.6B model. My go to test for checking hallucinations is 'Tell me about Mercantour park' (a national park in south eastern France). Easily half of the facts are invented. Non-existing mountain summits, brown bears (no, there are none), villages that are elsewhere, wrong advice ('dogs allowed' - no they are not).

Others have already said it, but it needs to be said again: Good god, stop treating LLMs like oracles. LLMs are not encyclopedias. Give an LLM the context you want to explore, and it will do a fantastic job of telling you all about it. Give an LLM access to web search, and it will find things for you and tell you what you want to know. Ask it "what's happening in my town this week?", and it will answer that with the…

I love how with this cutting edge tech people still dress up and pretend to be experts. Pleasure to meet you, pocketarc - Senior AI Gamechanger, 2024-2025 (Current)

Re: Open models by OpenAI

#475

Post by Cicero22 »

cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

Where did you get the top ten from?

https://huggingface.co/spaces/TIGER-Lab/MMLU-Pro

Are you discounting all of the self reported scores?

Re: Open models by OpenAI

#476

Post by seba_dos1 »

fnands wrote: post #223

Mhh, I wonder if these are distilled from GPT4-Turbo. I asked it some questions and it seems to think it is based on GPT4-Turbo: > Thus we need to answer "I (ChatGPT) am based on GPT-4 Turbo; number of parameters not disclosed; GPT-4's number of parameters is also not publicly disclosed, but speculation suggests maybe around 1 trillion? Actually GPT-4 is likely larger than 175B; maybe 500B. In any case, we can note i…

Just stop and think a bit about where a model may get the knowledge of its own name from.

Re: Open models by OpenAI

#477

Post by MYEUHD »

cco wrote: post #454
captainregex wrote: post #397

Earlier quoted context omitted.

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

> I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Well, the model makers and device manufacturers of course! While your Apple, Samsung, and Googles of the world will be unlikely to use OSS models locally (maybe Samsung?), they all have really big incentives to run models locally for a variety of reasons. Latency, privacy (Apple), cost to run these models on behalf of c…

>Google started shipping 16GB as the _lowest_ amount of RAM you can get on your Pixel 9.

The Pixel 9 has 12GB of RAM[0]. You probably meant the Pixel 9 Pro.

[0]: https://www.gsmarena.com/google_pixel_9-13219.php

Re: Open models by OpenAI

#478

Post by NoDoo »

Does anyone think people will distill this model? It is allowed. I'm new to running open source llms, but I've run qwen3 4b and phi4-mini on my phone before through ollama in termux.

Re: Open models by OpenAI

#479

Post by mattfrommars »

jakozaur wrote: post #71

The coding seems to be one of the strongest use cases for LLMs. Though currently they are eating too many tokens to be profitable. So perhaps these local models could offload some tasks to local computers. E.g. Hybrid architecture. Local model gathers more data, runs tests, does simple fixes, but frequently asks the stronger model to do the real job. Local model gathers data using tools and sends more data to the str…

Anyone know how long does the context last for running model locally vs running via OpenAPI or Cursor? My understanding is the model that run on the cloud have much greater context window that what we can have running locally.

Re: Open models by OpenAI

#480

Post by NoDoo »

Do you think someone will distill this or quantize it further than the current 4-bit from OpenAI so it could run on less than 16gb RAM? (The 20b version). To me, something like 7-8B with 1-3B active would be nice as I'm new to local AI and don't have 16gb RAM.
Post reply on HN