Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running local LLMs offline on a ten-hour flight

deploy.live

Post reply on HN
Print view View original

21–30 of 112 posts

Re: Running local LLMs offline on a ten-hour flight

#21

Post by walthamstow »

ryandrake wrote: post #20
rootusrootus wrote: post #7

Earlier quoted context omitted.

I'd probably choose the window seat myself, because while it is cramped, it is predictably so. When I sit in an aisle seat, it's not as cramped but I regularly get shoulder checked by passing people or beverage carts. What really makes me nervous if I'm in an economy seat is the seat in front of me. Depending on how the seat is designed, if the person suddenly reclines (or hell, just flexes the seat a bunch while mov…

That was the first thing I thought of when I saw the image. That's a very expensive computer that you risk destroying when the 300lb guy in front of you decides to lean back. The ergonomics of using a laptop on an economy-class tray table are not worth it. You're sitting there like a T-rex trying to make your arms as small as possible to tap on the keys. And the vertical viewing angle to your screen sometimes prevent…

In the image it's on his lap, not the tray table. I agree, using the tray is not worth it. The ideal is a tray that folds in half so I can use that to hold a drink and keep the machine on my lap.

The tradeoff of poor comfort is insane productivity, for me anyway. Being restricted in place, no wifi, inconvenient toilet breaks, not in control of meal times, all means I get a lot of work done

Re: Running local LLMs offline on a ten-hour flight

#22

Post by ryandrake »

j1000 wrote: post #15

To be honest, I think possibility to work and travel is con rather than perk of current times.

It hit different at different points in my life. When I was in my 20s I thought "Wow! I get to go on an international trip to a place I've never been, and work is paying for everything?!? I'll go whenever you need me to go!" Now that I'm almost 50, it's "Fuck. Another 14 hour international flight, to somewhere I'll likely only have time to see the inside of two buildings. What's the local language again? Do I drive on the left or right? Wait, how long do I need to stay? Please no."

Re: Running local LLMs offline on a ten-hour flight

#24

Post by NitpickLawyer »

deanc wrote: post #4

This has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max. I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Su…

> a lot of the hype around running these models locally is bullshit. Sure, you can make it do something but certainly nothing useful or substantial.

There is certainly a lot of hype around local models. Some of it is overhype, some of it is just "people finding out" and discovering what cool stuff you can do. I suspect the post is a reply to the other one a few days ago where someone from hf posted a pic with them in the plane, using a local model, and saying it's really really close to opus. That was BS.

That being said, I've been working with local LMs since before chatgpt launched. The progress we've made from the likes of gpt-j (6B) and gpt-neoX (22B) (some of the first models you could run on regular consumer hardware) is absolutely amazing. It has gone way above my expectations. We're past "we have chatgpt at home" (as it was when launched), and now it is actually usable in a lot of tasks. Nowhere near SotA, but "good enough".

I will push back a bit on the "substantial" part, and I will push a lot on "nothing useful". You can, absolutely get useful stuff out of these models. Not in a claude-code leave it to cook for 6 hours and get a working product, but with a bit of hand holding and scope reduction you can get useful stuff. When devstral came out (24B) I ran it for about a week as a "daily driver" just to see where it's at. It was ok-ish. Lots of hand holding, figured out I can't use it for planning much (looked fine at a glance, but either didn't make sense, or used outdated stuff). But with a better plan, it could handle implementation fine. I coded 2 small services that have been running in prod for ~6mo without any issues. That is useful, imo. And the current models are waaay better than devstral1.

As to substantial, eh... Your substantial can be someone else's taj mahal, and their substantial could be your toy project. It all depends. I draw the line at useful. If you can string together a couple of useful tasks, it starts to become substantial.

Re: Running local LLMs offline on a ten-hour flight

#26

Post by 2ndorderthought »

mft_ wrote: post #16
deanc wrote: post #4

This has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max. I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Su…

I’m frequently surprised how little I can find online about exactly this - different harnesses for local models and how to set them up. The documentation for opencode with local models is (IMO) pretty bad - and even Claude Opus (!) struggled to get it running. And so far I’ve not found a decent alternative to Claude Desktop. (I’ve recently discovered that you can pipe local models into Claude’s Code and Desktop, so t…

Qwen3.6 is brand new. But also, search engines are so plastered with AI slop that is written by tools and companies that have no interest in you using local models. Ollama makes it 1 command to run local small models, but with the newest ones there can be kinks to work out first.

/R/localllama is okay for some information but beyond that there is so much noise and very little signal. I think it's intentional.

Re: Running local LLMs offline on a ten-hour flight

#27

Post by bs7280 »

vladgur wrote: post #6

That window seat with the 14” laptop seems extremely claustrophobic. That’s the real limitation on an economy flight - space rather than power or the internet… at least it would be for me. The only times I was able to get my laptop out and do some productive work was when I either was sitting in premium economy isle seat with room to spare or when there was an empty seat next to me

I have a 16" M1 Max that I only got because it was $1500 cheaper than MSRP, and it sucks on planes. I have really long arms and I can barely get it out of my bag without elbowing my neighbor.

A few years ago I saw some very interesting custom ergonomic setups optimized for traveling + flying.

One person with a thinkpad is able to get the monitor to be 180 degrees flat w/ the keyboard, and can hang it off the seat. He also brings a split ergo keyboard with a lap mount.

Another person did something similar with a M1 laptop, but needs an Ipad to act as the external monitor (laptop stays in bag) with a built and designed from scratch split ergo keyboard.

Re: Running local LLMs offline on a ten-hour flight

#28

Post by zdw »

vladgur wrote: post #6

That window seat with the 14” laptop seems extremely claustrophobic. That’s the real limitation on an economy flight - space rather than power or the internet… at least it would be for me. The only times I was able to get my laptop out and do some productive work was when I either was sitting in premium economy isle seat with room to spare or when there was an empty seat next to me

That's a 16" (from the size of the speaker grille on each side of the keyboard), so even more claustrophobic.

Re: Running local LLMs offline on a ten-hour flight

#29

Post by xienze »

deanc wrote: post #4

This has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max. I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Su…

It's probably a combination of things:

* New models running in llama.cpp (what's under the hood of ollama et al) frequently require bug fixes.

* The GGUF models that run in llama.cpp frequently require bug fixes (Unsloth is notorious for this -- they release GGUF models about 10 minutes after official .safetensors releases).

* You're probably running a Local models really are great but I think a major problem are the people in groups like r/localllama who run models at absurd quantization levels in order to cram them on their underpowered hardware and convince themselves that they're running SOTA at home.

The best way to run these models is, frankly, a lot of VRAM and vLLM (which is what the people developing these models are almost certainly targeting).

Post reply on HN