Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

simonwillison.net

Post reply on HN
Print view View original

91–100 of 415 posts

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#91

Post by simonw »

devmor wrote: post #89
aschobel wrote: post #54

Earlier quoted context omitted.

Bingo, it's magical but the learning curve is very very steep. The METR study on open-source productivity alluded to this a bit. I am definitely at a point where I am more productive with it, but it took a bunch of effort.

The subjects in the study you are referencing also believed that they were more productive with it. What metrics do you have to convince yourself you aren't under the same illusionary bias they were?

Yesterday I used ffmpeg to extract the frame at the 13 second mark of a video out as a JPEG.

If I didn't have an LLM to figure that out for me I wouldn't have done it at all.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#92

Post by __mharrison__ »

toyg wrote: post #32
diggan wrote: post #17

Earlier quoted context omitted.

Even "accidentally" it makes sense that "SVGs of pelicans riding bikes" are now included into datasets used for training as it has spread as a wildfire on the internet, making it less useful as a simple benchmark. This is why I keep all my benchmarks private and don't share anything about them publicly, as soon as you write about them anywhere publicly they'll stop being useful in some months.

> This is why I keep all my benchmarks private This is also why, if I were an artist or anyone commercially relying on creative output of any kind, I wouldn't be posting anything on the internet anymore, ever. The minute you make anything public, the engines will clone it to death and turn it into a commodity.

Somewhat defeats the purpose of being an artist, doesn't it?

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#93

Post by devmor »

jan_Sate wrote: post #69
NitpickLawyer wrote: post #20

Earlier quoted context omitted.

This comment is ~3 years late. Every model since gpt3 has had the entirety of available code in their training data. That's not a gotcha anymore. We went from chatgpt's "oh, look, it looks like python code but everything is wrong" to "here's a full stack boilerplate app that does what you asked and works in 0-shot" inside 2 years. That's the kicker. And the sauce isn't just in the training set, models now do post-tra…

Not exactly. The real utility value of LLM for programming is to come up with something new. For Space Invaders, instead of using LLM for that, I might as well just manually search for the code online and use that. To show that LLM actually can provide value for one-shot programming, you need to find a problem that there's no fully working sample code available online. I'm not trying to say that LLM couldn't to that.…

> The real utility value of LLM for programming is to come up with something new.

That's the goal for these projects anyways. I don't know that its true or feasible. I find the RAG models much more interesting myself, I see the technology as having far more value in search than generation.

Rather than write some markov-chain reminiscent frankenstein function when I ask it how to solve a problem, I would like to see it direct me to the original sources it would use to build those tokens, so that I can see their implementations in context and use my judgement.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#94

Post by sowbug »

bradly wrote: post #10

I appreciate you sharing both the chat log and the full source code. I would be interested to see a followup post on how adding moderately-sized features like High Score go. Also, IANAL but Space Invaders is owned IP. I have no idea the legality of a blog post describing steps to create and releasing an existing game, but I've seen headlines on HN of engs in trouble for things I would not expect to be problematic. Ma…

It doesn't infringe any kind of intellectual property.

This isn't copyright infringement; it isn't based on the original assembly code or artwork. A game concept can't be copyrighted. Even if one of SI's game mechanics were patented, it would have long expired. Trade secret doesn't apply in this situation.

That leaves trademark. No reasonable person would be confused whether Simon is trying to pass this creation off as a genuine Space Invaders product.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#95

Post by phkahler »

FeepingCreature wrote: post #55
bayindirh wrote: post #42

Earlier quoted context omitted.

Last time somebody asked for a "premium camera app for iOS", and the model (re)generated Halide. Models don't emit something they don't know. They remix and rewrite what they know. There's no invention, just recall...

True where trivial; where nontrivial, false. Trivially, humans don't emit something they don't know either. You don't spontaneously figure out Javascript from first principles, you put together your existing knowledge into new shapes. Nontrivially, LLMs can absolutely produce code for entirely new requirements. I've seen them do it many times. Will it be put together from smaller fragments? Yes, this is called "exper…

>> Nontrivially, LLMs can absolutely produce code for entirely new requirements. I've seen them do it many times.

I think most people writing software today are reinventing a wheel, even in corporate environments for internal tools. Everyone wants their own tweak or thinks their idea is unique and nobody wants to share code publicly, so everyone pays programmers to develop buggy bespoke custom versions of the same stuff that's been done 100 times before.

I guess what I'm saying is that your requirements are probably not new, and to the extent they are yes an LLM can fill in the blanks due to its fluency in languages.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#96

Post by genewitch »

NitpickLawyer wrote: post #6

> Two years ago when I first tried LLaMA I never dreamed that the same laptop I was using then would one day be able to run models with capabilities as strong as what I’m seeing from GLM 4.5 Air—and Mistral 3.2 Small, and Gemma 3, and Qwen 3, and a host of other high quality models that have emerged over the past six months. Yes, the open-models have surpassed my expectations in both quality and speed of release. For…

I'll bite. How do i train/make and/or use LoRA, or, separately, how do i fine-tune? I've been asking this for months, and no one has a decent answer. websearch on my end is seo/geo-spam, with no real instructions.

I know how to make an SD LoRA, and use it. I've known how to do that for 2 years. So what's the big secret about LLM LoRA?

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#97

Post by __mharrison__ »

simonw wrote: post #43
jplrssn wrote: post #4

Earlier quoted context omitted.

I also wouldn't be surprised if labs were starting to mix in a few pelican SVGs into their training data.

I'll believe they are doing that when one of the models draws me an SVG that actually looks like a pelican.

Someone needs to craft a beautifully bike donned by a pelican, throw in some seo, and see how long it takes a model to replicate it.

Simon probably wouldn't be happy about killing his multi-year evaluation metric though...

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#98

Post by thenaturalist »

joelthelion wrote: post #70

Apart from using a Mac, what can you use for inference with reasonable performance? Is a Mac the only realistic option at the moment?

This guy [0] does a ton of in-depth HW comparison/ benchmarking, including against Mac mini clusters and an M3 ultra.

0: https://www.youtube.com/@AZisk

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#99

Post by simonw »

devmor wrote: post #93
jan_Sate wrote: post #69

Earlier quoted context omitted.

Not exactly. The real utility value of LLM for programming is to come up with something new. For Space Invaders, instead of using LLM for that, I might as well just manually search for the code online and use that. To show that LLM actually can provide value for one-shot programming, you need to find a problem that there's no fully working sample code available online. I'm not trying to say that LLM couldn't to that.…

> The real utility value of LLM for programming is to come up with something new. That's the goal for these projects anyways. I don't know that its true or feasible. I find the RAG models much more interesting myself, I see the technology as having far more value in search than generation. Rather than write some markov-chain reminiscent frankenstein function when I ask it how to solve a problem, I would like to see i…

"I would like to see it direct me to the original sources it would use to build those tokens"

Sadly that's not feasible with transformer-based LLMs: those original sources are long gone by the time you actually get to use the model, scrambled a billion times into a trained set of weights.

One thing that helped me understand this is understanding that every single token output by an LLM is the result of a calculation that considers all X billion parameters that are baked into that model (or a subset of that in the case of MoE models, but it's still billions of floating point calculations for every token.)

You can get an imitation of that if you tell the model "use your search tool and find example code for this problem and build new code based on that", but that's a pretty unconventional way to use a model. A key component of the value of these things is that they can spit out completely new code based on the statistical patterns they learned through training.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#100

Post by tptacek »

stpedgwdgfhgdd wrote: post #44

Aside that space invaders from scratch is not representative for real engineering, it will be interesting to see what the business model for Anthropic will be if I can run a solid code generation model on my local machine (no usage tier per hour or week), let’s say, one year from now. At $200 per month for 2 years I can buy a decent Mx with 64GB (or perhaps even 128GB taking residual value into account)

OK, go write Space Invaders by hand.
Post reply on HN