Running local LLMs offline on a ten-hour flight
1–10 of 112 posts
Re: Running local LLMs offline on a ten-hour flight
#2Qwen 4.6 36B? Do they mean Qwen3.6-35B-A3B?
Re: Running local LLMs offline on a ten-hour flight
#3Qwen 4.6 36B? Do they mean Qwen3.6-35B-A3B?
yes
Re: Running local LLMs offline on a ten-hour flight
#4This has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max.
I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Sure, you can make it do something but certainly nothing useful or substantial.
Re: Running local LLMs offline on a ten-hour flight
#5Qwen 4.6 36B? Do they mean Qwen3.6-35B-A3B?
So I have a RTX 3080 10GB VRAM which I've been using with Qwen2.5 Coder and Gemma 4 E2B. Im wondering what models you have tried with which quants.
Re: Running local LLMs offline on a ten-hour flight
#6That window seat with the 14” laptop seems extremely claustrophobic.
That’s the real limitation on an economy flight - space rather than power or the internet… at least it would be for me.
The only times I was able to get my laptop out and do some productive work was when I either was sitting in premium economy isle seat with room to spare or when there was an empty seat next to me
Re: Running local LLMs offline on a ten-hour flight
#7That window seat with the 14” laptop seems extremely claustrophobic. That’s the real limitation on an economy flight - space rather than power or the internet… at least it would be for me. The only times I was able to get my laptop out and do some productive work was when I either was sitting in premium economy isle seat with room to spare or when there was an empty seat next to me
I'd probably choose the window seat myself, because while it is cramped, it is predictably so. When I sit in an aisle seat, it's not as cramped but I regularly get shoulder checked by passing people or beverage carts.
What really makes me nervous if I'm in an economy seat is the seat in front of me. Depending on how the seat is designed, if the person suddenly reclines (or hell, just flexes the seat a bunch while moving around), it can come pretty close to pinching the laptop screen. That would be bad news.
Re: Running local LLMs offline on a ten-hour flight
#8This has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max. I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Su…
Same here. Every time a new local model comes out, I give it a spin with a pretty vanilla coding task ("refactor this method to take two parameters instead of one", or "fix this class of compiler warning across the ~20 file codebase") and more often than not, they get in endless loops, or fail in very unusual ways. They don't yet even approach the usefulness of SOTA models. It's obviously not a fair comparison, though. My 20GB GPU is never going to beat whatever enormous backend Google or Anthropic have.
Re: Running local LLMs offline on a ten-hour flight
#9This has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max. I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Su…
You need to set sampling parameters for the llm. Had the same issue with Qwen3.5 when i first started. You can grab them off the model card page usually.
From Qwen3.6 page:
Thinking mode for general tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
Thinking mode for precise coding tasks (e.g. WebDev): temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
Instruct (or non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
Re: Running local LLMs offline on a ten-hour flight
#10This has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max. I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Su…
In the article the author describes what they made. It's definitely not bullshit, but it's also not as reliable or as handsfree as the 1t models.
For people who aren't completely vibe or agent coding these models are better than say copilot or the free models appearing after a Google search. Probably better than chatgpts flagships in some ways.
I mostly use 4b to 9b models for basic inquiries and code examples from libraries I haven't used before. Many of them can solve pretty hard math problems, and these are several steps away from say qwen3.6.
I would not discount running models locally. It's the best case scenario of a future with LLMs from a human rights and ecological perspective.