Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

simonwillison.net

Post reply on HN
Print view View original

131–140 of 415 posts

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#131

Post by falcor84 »

phkahler wrote: post #117
simonw wrote: post #85

Earlier quoted context omitted.

Go for it. https://www.google.com/search?client=firefox-b-1-d&q=github+... has a bunch of results. Here's the source code GLM-4.5 Air spat out for me on my laptop: https://github.com/simonw/tools/blob/main/space-invaders-GLM... Based on my mental model of how these things work I'll be genuinely surprised if you can find even a few lines of code duplicated from one of those projects into the code that GLM-4.5 wrote fo…

So I scanned the beginning of the generated code, picked line 83: animation: glow 2s ease-in-out infinite; stuffed it verbatim into google and found a stack overflow discussion that contained this: animation: glow .5s infinite alternate; in under one minute. Then I found this page of CSS effects: https://alvarotrigo.com/blog/animated-backgrounds-css/ Another page has examples and contains: animation: float 15s infini…

The parent said

> find even a few lines of code duplicated from one of those projects

I'm pretty sure they meant multiple lines copied verbatim from a single project implementing space invaders, rather than individual lines copied (or likely just accidentally identical) across different unrelated projects.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#132

Post by ben_w »

phkahler wrote: post #117
simonw wrote: post #85

Earlier quoted context omitted.

Go for it. https://www.google.com/search?client=firefox-b-1-d&q=github+... has a bunch of results. Here's the source code GLM-4.5 Air spat out for me on my laptop: https://github.com/simonw/tools/blob/main/space-invaders-GLM... Based on my mental model of how these things work I'll be genuinely surprised if you can find even a few lines of code duplicated from one of those projects into the code that GLM-4.5 wrote fo…

So I scanned the beginning of the generated code, picked line 83: animation: glow 2s ease-in-out infinite; stuffed it verbatim into google and found a stack overflow discussion that contained this: animation: glow .5s infinite alternate; in under one minute. Then I found this page of CSS effects: https://alvarotrigo.com/blog/animated-backgrounds-css/ Another page has examples and contains: animation: float 15s infini…

So, your example of it copying snippets is… using the same API with fairly different parameters in a different order?

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#133

Post by hbn »

falcor84 wrote: post #74
stpedgwdgfhgdd wrote: post #44

Aside that space invaders from scratch is not representative for real engineering, it will be interesting to see what the business model for Anthropic will be if I can run a solid code generation model on my local machine (no usage tier per hour or week), let’s say, one year from now. At $200 per month for 2 years I can buy a decent Mx with 64GB (or perhaps even 128GB taking residual value into account)

How come it's "not representative for real engineering"? Other than copy-pasting existing code (which is not what an LLM does), I don't see how you can create a space invaders game without applying "engineering".

The prompt was

> Write an HTML and JavaScript page implementing space invaders

It may not be "copy pasting" but it's generating output as best it can be recreated from its training on looking at Space Invaders source code.

The engineers at Taito that originally developed Space Invaders were not told "make Space Invaders" and then did their best to recall all the source code they've looked at in their life to re-type the source code to an existing game. From a logistics standpoint, where the source code already exists and is accessible, you may as well have copy-pasted it and fudged a few things around.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#135

Post by tracker1 »

jan_Sate wrote: post #69
NitpickLawyer wrote: post #20

Earlier quoted context omitted.

This comment is ~3 years late. Every model since gpt3 has had the entirety of available code in their training data. That's not a gotcha anymore. We went from chatgpt's "oh, look, it looks like python code but everything is wrong" to "here's a full stack boilerplate app that does what you asked and works in 0-shot" inside 2 years. That's the kicker. And the sauce isn't just in the training set, models now do post-tra…

Not exactly. The real utility value of LLM for programming is to come up with something new. For Space Invaders, instead of using LLM for that, I might as well just manually search for the code online and use that. To show that LLM actually can provide value for one-shot programming, you need to find a problem that there's no fully working sample code available online. I'm not trying to say that LLM couldn't to that.…

I have a friend who has been doing just that... usually with his company he manages a handful of projects where a bulk of the development is outsourced overseas. This past year, he's outpaced the 6 devs he's had working on misc projects just with his own efforts and AI. Most of this being a relatively unique combination of UX with features that are less common.

He's using AI with note taking apps for meetings to enhance notes and flush out technology ideas at a higher level, then refining those ideas into working experiments.

It's actually impressive to see. My personal experience has been far more disappointing to say the least. I can't speak to the code quality, consistency or even structure in terms of most people being able to maintain such applications though. I've asked to shadow him through a few of his vibe coding sessions to see his workflow. It feels rather alien to me, again my experience is much more disappointing in having to correct AI errors.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#136

Post by Uehreka »

bayindirh wrote: post #42
quantumHazer wrote: post #13

Earlier quoted context omitted.

and probably some synthetic data are generated copy of the games already on the dataset? i have this feeling with LLM's generated react frontend, they all look the same

Last time somebody asked for a "premium camera app for iOS", and the model (re)generated Halide. Models don't emit something they don't know. They remix and rewrite what they know. There's no invention, just recall...

> Models don't emit something they don't know. They remix and rewrite what they know. There's no invention, just recall...

People really need to stop saying this. I get that it was the Smart Guy Thing To Say in 2023, but by this point it’s pretty clear that that it’s not true in any way that matters for most practical purposes.

Coding LLMs have clearly been trained on conversations where a piece of code is shown, a transformation is requested (rewrite this from Python to Go), and then the transformed code is shown. It’s not that they’re just learning codebases, they’re learning what working with code looks like.

Thus you can ask an LLM to refactor a program in a language it has never seen, and it will “know” what refactoring means, because it has seen it done many times, and it will stand a good chance of doing the right thing.

That’s why they’re useful. They’re doing something way more sophisticated than just “recombining codebases from their training data”, and anyone chirping 2023 sound bites is going to miss that.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#137

Post by toyg »

__mharrison__ wrote: post #92
toyg wrote: post #32

Earlier quoted context omitted.

> This is why I keep all my benchmarks private This is also why, if I were an artist or anyone commercially relying on creative output of any kind, I wouldn't be posting anything on the internet anymore, ever. The minute you make anything public, the engines will clone it to death and turn it into a commodity.

Somewhat defeats the purpose of being an artist, doesn't it?

Defeating the purpose of creating almost anything, really.

AI is definitely breaking the whole "labor for money" architecture of our world.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#138

Post by devmor »

simonw wrote: post #112
devmor wrote: post #108

Earlier quoted context omitted.

I am aware, and that's exactly why I don't think they're anywhere near as useful for this type of work as the people pushing them want them to be. I tried to push for this type of model when an org I worked with over a decade ago was first exploring using the first generation of Tensorflow to drive customer service chatbots and was sadly ignored.

I don't understand. For code, why would I want to remix existing code snippets? I totally get the value of RAG style patterns for information retrieval against factual information - for those I don't want the LLM to answer my question directly, I want it to run a search and show me a citation and directly quote a credible source as part of answering. For code I just want code that works - I can test it myself to make…

> I don't understand. For code, why would I want to remix existing code snippets?

That is what you're doing already. You're just relying on a vector compression and search engine to hide it from you and hoping the output is what you expect, instead of having it direct you to where it remixed those snippets from so you can see how they work to start with and make sure its properly implemented from the get-go.

We all want code that works, but understanding that code is a critical part of that for anything but a throw-away one time use script.

I don't really get this desire to replace critical thought with hoping and testing. It sounds like the pipe dream of a middle manager, not a tool for a programmer.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#139

Post by tracker1 »

lifestyleguru wrote: post #129

> my 2.5 year old laptop (a 64GB MacBook Pro M2) i My MacBook has 16GB of RAM and it is from a period when everyone was fiercely insisting that 8GB base model is all I'll ever need.

I'm kind of with you... while I've run 128gb on my desktop, and currently at 96gb with dr5 what it is, It's far less common for typical laptops. I'm a bit curious how the Ryzen 395+ with 128gb will handle some of these models. The 200gb options feel completely out of reach.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#140

Post by diggan »

kelvinjps10 wrote: post #86
diggan wrote: post #21

Earlier quoted context omitted.

For what? It's really hard to say what model is "generally" better then another, as they're all better/worse at specific things. My own benchmarks has a bunch of different tasks I use various local models for, and I run it when I wanna see if a new model is better than the existing ones I use. The output is basically a markdown table with a description of which model is best for what task. They're being sold as gener…

coding? they are coding models? what specific tasks is one performing better than the other?

They may be, but there are lots of languages, lots of approaches, lots of methodologies and just a ton of different ways to "code", coding isn't one homogeneous activity that one model beats all the other models at.

> what specific tasks is one performing better than the other?

That's exactly why you create your own benchmark, so you can figure that out by just having a list of models, instead of testing each individually and basing it on "feels better".

Post reply on HN