171–180 of 286 posts
I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…
Are there any non-Chinese open models that offer comparable performance?
Earlier quoted context omitted.
I think you could look into Minstral. There's also GPT-OSS but I'm not sure how well it stacks up. What's your problem with Chinese LLMs?
it’s not obvious to you why someone would want to avoid models created by our enemies?
Thinking about getting a new MBP M5 Max 128GB (assuming they are released next week). I know "future proofing" at this stage is near impossible, but for writing Rust code locally (likely using Qwen 3.5 for now on MLX), the AIs have convinced me this is probably my best choice for immediate with some level of longevity, while retaining portability (not strictly needed, but nice to have). Alternatively was considering…
Also Nvidia Spark.
Earlier quoted context omitted.
> Wonder what am I doing wrong? You're comparing 100b parameters open models running on a consumer laptop VS private models with at the very least 1t parameters running on racks of bleeding edge professional gpus Local agentic coding is closer to "shit me the boiler plate for an android app" not "deep research questions", especially on your machine
The hardware difference explains runtime performance differences, not task performance. Speculation is that the frontier models are all below 200B parameters but a 2x size difference wouldn’t fully explain task performance differences
Core speed/count and memory bandwidth determines your performance. Memory size determines your model size which determines your smarts. Broadly speaking.
What kind of hardware does HN recommend or like to run these models?
The cheapest option is two 3060 12G cards. You'll be able to fit the Q4 of the 27B or 35B with an okay context window. If you want to spend twice as much for more speed, get a 3090/4090/5090. If you want long context, get two of them. If you have enough spare cash to buy a car, get an RTX Ada with 96G VRAM.
What kind of hardware does HN recommend or like to run these models?
I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…
I periodically try to run these models on my MBP M3 Max 128G (which I bought with a mind to run local AI). I have a certain deep research question (in a field that is deeply familiar to me) that I ask when I want to gauge model's knowledge. So far Opus 4.6 and Gemini Pro are very satisfactory, producing great answers fairly fast. Gemini is very fast at 30-50 sec, Opus is very detailed and comes at about 2-3 minutes.…
I'm too GPU-poor to run it, but r/LocalLLaMa is full of people using it.