Ask Gemini to make a nice anime portrait of Claude. Maybe with an interesting weapon in hand just in case.
Claude Is Down
61–70 of 78 posts
Re: Claude Is Down
#62On flights with shitty wifi I have been running gpt-oss:120b on my macbook using ollama. Ok model for coding if you can't reach a good one.
Are you running the full 65GB model on a MacBook Pro? What tokens per second do you get? What specs? M5?
On an m4 pro 128gb: 75 t/s.
Caveat: That's just for the first prompt.
Re: Claude Is Down
#63On flights with shitty wifi I have been running gpt-oss:120b on my macbook using ollama. Ok model for coding if you can't reach a good one.
The key thing I'm confident in is that 2-3 years from now there's going to be a model(s) and workflow that has comparable accuracy, perhaps noticeable (but tolerable) higher latency that can be run locally. There's just no reason to believe this isn't achievable.
Hard to understand how this won't make all of the solutions for existing use cases commodity. I'm sure 2-3 years from now there'll be stuff that seems like magic to us now -- but it will be more-meta, more "here's a hypothesis of a strategically valuable outcome and heres a solution (with market research and user testing done".
I think current performance and leading models will turn out to have been terrible indicators for future market leader (and my money will remain on the incumbents with the largest cash reserves (namely Google) that have invested in fundamental research and scaling).
Re: Claude Is Down
#64Earlier quoted context omitted.
Not intending to defend OpenAI here, but their MAU (800 million) does dwarf most other AI companies, anthropic included. I do not envy the engineers there working on scaling.
Would you do 9-9-6 if your comp. is 8-9 figures/year?
absolutely not. I have a life and hobbies
Re: Claude Is Down
#65This is the part in the movie where they have to convince the grizzled hacker to come out of retirement because he's the only one who can actually operate Emacs or vim and write code.
Maybe its a generational thing, but to me an elite hacker is an uwu catgirl type with lain vibes that knows an unhealthy amount about computers. typically an emacs evil-mode user who would quote weird poems about whatever software they're working on.
It could be a buddy movie where the grizzled guy (who uses emacs) and the uwu cat girl (who uses vim) grudgingly come to admire one another's skills and become friends.
Re: Claude Is Down
#66Re: Claude Is Down
#67Earlier quoted context omitted.
Maybe its a generational thing, but to me an elite hacker is an uwu catgirl type with lain vibes that knows an unhealthy amount about computers. typically an emacs evil-mode user who would quote weird poems about whatever software they're working on.
It could be a buddy movie where the grizzled guy (who uses emacs) and the uwu cat girl (who uses vim) grudgingly come to admire one another's skills and become friends.
[deleted]
Re: Claude Is Down
#68Earlier quoted context omitted.
Wow, thanks for the info. I'm planning on testing this on my M4 Max w/ 36 GB today. edit: So looking here https://ollama.com/library/gpt-oss/tags it seems ollama doesn't even provide the MXFP4 variants, much less hide them. Is the best way to run these variants via llama.cpp or...?
LMStudio
Can you be more specific? I've got LM Studio downloaded but it's not clear where are the official OpenAI releases? Are they all only available via transformers? The only one that shows up in search appears to be the distilled gpt-oss 20B...
Re: Claude Is Down
#69On flights with shitty wifi I have been running gpt-oss:120b on my macbook using ollama. Ok model for coding if you can't reach a good one.
GPT-OSS-120b/20b is probably the best you can run on your own hardware today. Be careful with the quantized versions though, as they're really horrible compared to the native MXFP4. I haven't looked in this particular case, but Ollama tends to hide their quantizations for some reason, so most people who could be running 20B with MXFP4, are still on Q8 and getting much worse results than they could.
they support that format according to the model page on their site:
Re: Claude Is Down
#70Earlier quoted context omitted.
It's a different way of doing quantization ( https://huggingface.co/docs/transformers/en/quantization/mxf... ) but I think the most important thing is that OpenAI delivered their own quantization (the MXFP4 from OpenAI/GPT-OSS on HuggingFace, guaranteed correct) whereas all the Q8 and other quantizations you see floating around are community efforts, with somewhat uneven results depending on who done it. Concretely f…
Wow, thanks for the info. I'm planning on testing this on my M4 Max w/ 36 GB today. edit: So looking here https://ollama.com/library/gpt-oss/tags it seems ollama doesn't even provide the MXFP4 variants, much less hide them. Is the best way to run these variants via llama.cpp or...?
on the model description page they claim they support it:
Quantization - MXFP4 format
OpenAI utilizes quantization to reduce the memory footprint of the gpt-oss models. The models are post-trained with quantization of the mixture-of-experts (MoE) weights to MXFP4 format, where the weights are quantized to 4.25 bits per parameter. The MoE weights are responsible for 90+% of the total parameter count, and quantizing these to MXFP4 enables the smaller model to run on systems with as little as 16GB memory, and the larger model to fit on a single 80GB GPU.
Ollama is supporting the MXFP4 format natively without additional quantizations or conversions. New kernels are developed for Ollama’s new engine to support the MXFP4 format.
Ollama collaborated with OpenAI to benchmark against their reference implementations to ensure Ollama’s implementations have the same quality.