Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

simonwillison.net

Post reply on HN
Print view View original

211–220 of 415 posts

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#211

Post by bob1029 »

> still think it’s noteworthy that a model running on my 2.5 year old laptop (a 64GB MacBook Pro M2) is able to produce code like this—especially code that worked first time with no further edits needed.

I believe we are vastly underestimating what our existing hardware is capable of in this space. I worry that narratives like the bitter lesson and the efficient compute frontier are pushing a lot of brilliant minds away from investigating revolutionary approaches.

It is obvious that the current models are deeply inefficient when you consider how much you can decimate the precision of the weights post-training and still have pelicans on bicycles, etc.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#212

Post by bprew »

maksimur wrote: post #197

A $xxxx 2.5 year old laptop, one that's probably much more powerful than an average laptop bought today and probably next year as well. I don't think it's a fair reference point.

His point isn't that you can run a model on an average laptop, but that the same laptop can still run frontier models.

It speaks to the advancements in models that aren't just throwing more compute/ram at it.

Also, his laptop isn't that fancy.

> It claims to be small enough to run on consumer hardware. I just ran the 7B and 13B models on my 64GB M2 MacBook Pro!

From: https://simonwillison.net/2023/Mar/11/llama/

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#213

Post by simonw »

parsimo2010 wrote: post #209
maksimur wrote: post #197

A $xxxx 2.5 year old laptop, one that's probably much more powerful than an average laptop bought today and probably next year as well. I don't think it's a fair reference point.

The article is pretty good overall, but the title did irk me a little. I assumed when reading "2.5 year old" that it was fairly low-spec only to find out it was an M2 Macbook Pro with 64 GB of unified memory, so it can run models bigger than what an Nvidia 5090 can handle. I suppose that it could be intended to be read as "my laptop is only 2.5 years old, and therefore fairly modern/powerful" but I doubt that was the…

The reason I emphasize the laptop's age is that it is the same laptop I have been using ever since the first LLaMA release.

This makes it a great way to illustrate how much better the models have got without requiring new hardware to unlock those improved abilities.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#214

Post by jonas21 »

bob1029 wrote: post #211

> still think it’s noteworthy that a model running on my 2.5 year old laptop (a 64GB MacBook Pro M2) is able to produce code like this—especially code that worked first time with no further edits needed. I believe we are vastly underestimating what our existing hardware is capable of in this space. I worry that narratives like the bitter lesson and the efficient compute frontier are pushing a lot of brilliant minds a…

Wasn't the bitter lesson about training on large amounts of data? The model that he's using was still trained on a massive corpus (22T tokens).

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#215

Post by stavros »

devmor wrote: post #138
simonw wrote: post #112

Earlier quoted context omitted.

I don't understand. For code, why would I want to remix existing code snippets? I totally get the value of RAG style patterns for information retrieval against factual information - for those I don't want the LLM to answer my question directly, I want it to run a search and show me a citation and directly quote a credible source as part of answering. For code I just want code that works - I can test it myself to make…

> I don't understand. For code, why would I want to remix existing code snippets? That is what you're doing already. You're just relying on a vector compression and search engine to hide it from you and hoping the output is what you expect, instead of having it direct you to where it remixed those snippets from so you can see how they work to start with and make sure its properly implemented from the get-go. We all w…

I don't understand your point. You seem to be saying that we should be getting code from the source, then adapting it to our project ourselves, instead of getting adapted code to begin with.

I'm going to review the code anyway, why would I not want to save myself some of the work? I can "see how they work" after the LLM gives them to me just fine.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#216

Post by ducktective »

simonw wrote: post #187
bgwalter wrote: post #176

Earlier quoted context omitted.

Because we are told that they can solve IMO problems. Yet they fail at basic math problems, not only at factorization but also when probing them with relatively basic symbolic math that would not require the invocation of an external program. Also, you know it they fail they could say so instead of giving a hallucinated answer. First the models lie and say that a 20 digit number takes vast amounts of computing. Then,…

That's why the IMO results were so notable: that was one of those moments where new models were demonstrated doing something that they had previously been unable to do.

I can't fathom why more people aren't talking about the IMO story. Apparently the model they used is not just an LLM but some RL are involved too. If a model wins gold at IMO, is it still merely a "statistical parrot"?

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#217

Post by threeducks »

simonw wrote: post #126
phkahler wrote: post #117

Earlier quoted context omitted.

So I scanned the beginning of the generated code, picked line 83: animation: glow 2s ease-in-out infinite; stuffed it verbatim into google and found a stack overflow discussion that contained this: animation: glow .5s infinite alternate; in under one minute. Then I found this page of CSS effects: https://alvarotrigo.com/blog/animated-backgrounds-css/ Another page has examples and contains: animation: float 15s infini…

That's not an example of copying from an existing Space Invaders implementation. That's an LLM using a CSS animation pattern - one that it's seen thousands (probably millions) of times in the training data. That's what I expect these things to do: they break down Space Invaders into the components they need to build, then mix and match thousands of different coding patterns (like "animation: glow 2s ease-in-out infin…

I think LLMs are adapting higher level concepts. For example, the following JavaScript code generated by GLM (https://github.com/simonw/tools/blob/9e04fd9895fae1aa9ac78b8...) is clearly inspired by this C++ code (https://github.com/portapack-mayhem/mayhem-firmware/blob/28e...), but it is not an exact copy.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#218

Post by yahoozoo »

jonas21 wrote: post #214
bob1029 wrote: post #211

> still think it’s noteworthy that a model running on my 2.5 year old laptop (a 64GB MacBook Pro M2) is able to produce code like this—especially code that worked first time with no further edits needed. I believe we are vastly underestimating what our existing hardware is capable of in this space. I worry that narratives like the bitter lesson and the efficient compute frontier are pushing a lot of brilliant minds a…

Wasn't the bitter lesson about training on large amounts of data? The model that he's using was still trained on a massive corpus (22T tokens).

What does that have to do with quantizing?

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#219

Post by Keyframe »

indigodaddy wrote: post #127

Did pretty well with a boggle clone. I like that it tries to do a single html file (I didn't ask for that but was pleasantly surprised). It didn't include dictionary validation so needed a couple of prompts. Touch selection on mobile isn't the greatest but I've seen plenty worse https://chat.z.ai/space/z0gcn6qtu8s1-art https://chat.z.ai/s/74fe4ddc-f528-4d21-9405-0a8b15a96520

I went the other route with tetris clone the other day. It's definitely not a single prompt. It took me solid 15 hours until this stage to get here and most of that me thinking.. BUT, except one small trivial thing (space invader logo in pre tag) I haven't touched code - just looked at it. I made it mandatory for myself to see if I can first greenfield myself into this project and then brownfield features and fixes.. It's definitely a ton of work on my end, but it's also not something I'd be able to do in ~2 working days or less. As a cherry on top, even though it's still not done yet, I put in AI-generated music singing about the project itself. https://www.susmel.com/stacky/

Definitely a ton of things I learned about how to "develop" "with" AI along the way.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#220

Post by itsalotoffun »

jonas21 wrote: post #214
bob1029 wrote: post #211

> still think it’s noteworthy that a model running on my 2.5 year old laptop (a 64GB MacBook Pro M2) is able to produce code like this—especially code that worked first time with no further edits needed. I believe we are vastly underestimating what our existing hardware is capable of in this space. I worry that narratives like the bitter lesson and the efficient compute frontier are pushing a lot of brilliant minds a…

Wasn't the bitter lesson about training on large amounts of data? The model that he's using was still trained on a massive corpus (22T tokens).

I think GP means that if you internalize the bitter lesson (more data more compute wins), you stop imagining how to squeeze SOTA minus 1 performance out of constrained compute environments.
Post reply on HN