Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

simonwillison.net

Post reply on HN
Print view View original

321–330 of 415 posts

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#321

Post by jasonjmcghee »

genewitch wrote: post #96
NitpickLawyer wrote: post #6

> Two years ago when I first tried LLaMA I never dreamed that the same laptop I was using then would one day be able to run models with capabilities as strong as what I’m seeing from GLM 4.5 Air—and Mistral 3.2 Small, and Gemma 3, and Qwen 3, and a host of other high quality models that have emerged over the past six months. Yes, the open-models have surpassed my expectations in both quality and speed of release. For…

I'll bite. How do i train/make and/or use LoRA, or, separately, how do i fine-tune? I've been asking this for months, and no one has a decent answer. websearch on my end is seo/geo-spam, with no real instructions. I know how to make an SD LoRA, and use it. I've known how to do that for 2 years. So what's the big secret about LLM LoRA?

brev.dev made an easy to follow guide a while ago but apparently Nvidia took it down or something when they bought them?

So here's the original

https://web.archive.org/web/20231127123701/https://brev.dev/...

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#323

Post by israrkhan »

qcnguy wrote: post #200
genewitch wrote: post #96

Earlier quoted context omitted.

I'll bite. How do i train/make and/or use LoRA, or, separately, how do i fine-tune? I've been asking this for months, and no one has a decent answer. websearch on my end is seo/geo-spam, with no real instructions. I know how to make an SD LoRA, and use it. I've known how to do that for 2 years. So what's the big secret about LLM LoRA?

LLM fine tuning tends to destroy the model's capabilities if you aren't very careful. It's not as easy or effective as with image generation.

do you have a suggestion or a way to measure if model capabilities are getting destroyed? how do one measure it objectively?

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#324

Post by throwaway0123_5 »

weberer wrote: post #205
regularfry wrote: post #102

Earlier quoted context omitted.

This one should just about fit on a box with an RTX 4090 and 64GB RAM (which is what I've got) at q4. Don't know what the performance will be yet. I'm hoping for an unsloth dynamic quant to get the most out of it.

Whats important is VRAM, not system RAM. The 4090 has 16gb of VRAM so you'll be limited to smaller models at decent speeds. Of course, you can run models from system memory, but your tokens/second will be orders of magnitude slower. ARM Macs are the exception since they have unified memory, allowing high bandwidth between the GPU and the system's RAM.

iirc 4090s have 24GB

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#325

Post by nicce »

slimebot80 wrote: post #322

(novice question) 64gb is pure RAM? I thought Apple Silicon was efficient at paging SSD as memory storage - how important is RAM if you've got a fast SSD?

Memory speed is the most important factor with LLMs and SSD is very slow when compared to RAM.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#326

Post by adastra22 »

ben_w wrote: post #247
dingnuts wrote: post #111

Earlier quoted context omitted.

It is nice to use LLMs to generate ffmpeg commands, because those can be pretty tricky, but really, you wouldn't have just used the man page before? That explains a lot about Django that the author is allergic to man pages lol

I remember when I was a kid, people asking a teacher how to spell a word, and the answer was generally "look it up in a dictionary"… which you can only do if you already have shortlist of possible spellings. *nix man pages are the same: if you already know which tool can solve your problem, they're easy to use. But you have to already have a shortlist of tools that can solve your problem, before you even know which m…

That’s what GNU info is for, of course.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#328

Post by simonw »

paufernandez wrote: post #317
simonw wrote: post #297

Earlier quoted context omitted.

Because it's much, much faster to review a hundred lines of code than it is to write a hundred lines of code. (I'm experienced at reading and reviewing code.)

Simon, don't you fear "atrophy" in your writing ability?

I think it will happen a bit, but I'm not worried about it.

My ability to write with a pen has suffered enormously now that I do most of my writing on a phone or laptop - but I'm writing way more.

I expect I'll become slower at writing code without an LLM, but the volume of (useful) code I produce will be worth the trade off.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#329

Post by stelonix »

wizzwizz4 wrote: post #302
Philpax wrote: post #287

Earlier quoted context omitted.

Translated a vague natural language query ("cli, extract frame 13s into video") into something immediately actionable with specific examples and explanations, surfacing information that I would otherwise not know how to search for. That's what I've done with my ffmpeg LLM queries, anyway - can't speak for simonw!

DuckDuckGo search results for "cli, extract frame 13s into video" (no quotes): • https://stackoverflow.com/questions/10957412/fastest-way-to-... • https://superuser.com/questions/984850/linux-how-to-extract-... • https://www.aleksandrhovhannisyan.com/notes/video-cli-cheat-... • https://www.baeldung.com/linux/ffmpeg-extract-video-frames • https://ottverse.com/extract-frames-using-ffmpeg-a-comprehen... Search engines h…

Yet the interface is fundamentally different, the output feels much more like bro pages[0] and it's within a click of clipboarding, one CTRL V away from extracting the 13th second screenshot. I've been using Google the past 24 years and my google-fu has always left people amazed; yet I can no longer bother to go through Stack Exchange's results when an LLM not only spits it out so nicely, but also does the equivalent of a explainshell[1].

Not comparable and I fail to see why going through Google's ads/results would be better?

[0] https://github.com/pombadev/bropages

[1] https://github.com/idank/explainshell

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#330

Post by RALaBarge »

israrkhan wrote: post #323
qcnguy wrote: post #200

Earlier quoted context omitted.

LLM fine tuning tends to destroy the model's capabilities if you aren't very careful. It's not as easy or effective as with image generation.

do you have a suggestion or a way to measure if model capabilities are getting destroyed? how do one measure it objectively?

Ask it a series of the same questions after you train that you posed before training started. Is the quality lower?
Post reply on HN