Could you do this for the new DeepSeek please?
Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
371–380 of 382 posts
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#372Earlier quoted context omitted.
Saying that something is thought terminating is thought terminating, it's the laziest "I win" bullshit approach ever. A more nuanced approach: don't produce slop and people won't dismiss it as lazy bullshit either.
No, that's not true at all. Thought-terminating cliches cause you to stop thinking; they give a quick shortcut that let's you be dismissive. That's what "AI slop" is, when someone bestows the moniker on a piece of prose that has "It's not this, it's that" in it. Look, there's is a wide variety of work being produced with AI, all the way from exceptional professional work to total trash done by amateurs. Painting all…
Yes, it’s dismissive, intentionally and abrasively so. Because the author is being disrespectful to the reader, expecting them to put more effort into it than they did. I will happily be uncurious about a text the author couldn't even bother to proofread and clean up. Life is finite, LLMs could generate more text than I could ever possibly read, there has to be a quality filter we all apply and I’ve drawn my line in the sand.
You’re arguing in favor of what I view as pollution. I’m not lacking thought, it didn’t terminate, you just value slop for reasons I don’t.
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#373This project will land you a job at either Apple or Google!
Uh, maybe, who knows. I am pretty bad solving leetcode, btw
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#374Could you do this for the new DeepSeek please?
I think there is a limit based on MoE number of active parameters and quantisation, bytes count for active experts. But I believe we will see more project like this for different models.
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#375[dead]
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#376Hi! Tried it and i'm impressed. The Mac app reports 4.4 token/s in the Mac Mini M2 with 8GB RAM. Not fast but still very much usable (my use rarely goes past from summarizing and generating pretty documentation). However, that mac sits in the rack cabinet and i ssh into it, so i would love to chat with it from the terminal, but because i generally use ollama i don't really know how to do that. Can someone help?
Thank you for testing and sharing results!
I think I might understand your use case. You ssh the Mac and want something like `ollama run` with an interactive chat in terminal. Am I right?
There is already experimental OpenAI-compatible server in this repo:
``` swift build -c release --product TurboFieldfareServer .build/release/TurboFieldfareServer \ --model scratch/gemma4.gturbo ```
After that a small terminal client can run inside the same ssh session and talk to `/v1/chat/completions`
The client needs to keep a messages array, add each user message, send the full array with `stream:true`, print SSE chunks until `[DONE]`, then add the response back to the array. `/reset` can clear it
There is a python example in the server docs. (https://github.com/drumih/turbo-fieldfare/blob/main/docs/OPE...)
It is non-streaming, but can be used as a starting point.
The server is still experimental and I am fixing some problems currently. But you can try to vibecode a simple terminal client around it.
If not, create an issue on Github and describe desired behaviour
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#377Guess now not only RAM bandwidth is important, but also flash storage bw.
Maybe Apple could attach more flash chips in parallel, increasing the bus width and thus the overall bandwidth?
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#378Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#379Hi! Tried it and i'm impressed. The Mac app reports 4.4 token/s in the Mac Mini M2 with 8GB RAM. Not fast but still very much usable (my use rarely goes past from summarizing and generating pretty documentation). However, that mac sits in the rack cabinet and i ssh into it, so i would love to chat with it from the terminal, but because i generally use ollama i don't really know how to do that. Can someone help?
Thank you for testing and sharing results! I think I might understand your use case. You ssh the Mac and want something like `ollama run` with an interactive chat in terminal. Am I right? There is already experimental OpenAI-compatible server in this repo: ``` swift build -c release --product TurboFieldfareServer .build/release/TurboFieldfareServer \ --model scratch/gemma4.gturbo ``` After that a small terminal clien…
Yes, that is the idea. did read the paragraph about the cli server but of course that goes beyon what i know about these tools, that's why i asked. Having a streaming chat via command line would be awesome, because then multiple users could use the machine at the same time (i think 2, max 3 on a 8GB device. It would really make our old mini useful again, instead of sitting in the corner taking space)
Re: Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
#380Earlier quoted context omitted.
So install Asahi Linux?
Asahi is a very cool project, and worthwhile if someone goes into it well aware of the major tradeoffs they're making, including reduced hardware functionality and support, which is improving, and significantly degraded security, which will likely always be the case. macOS is the only OS which fully supports M1 hardware and its security features. Please see Asahi Linux's documentation: https://asahilinux.org/docs/pla…
> and worthwhile if someone goes into it well aware of the major tradeoffs they're making
Are you suggesting me to write all tradeoffs of everything that I suggest?