Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

simonwillison.net

Post reply on HN
Print view View original

301–310 of 415 posts

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#301

Post by CamperBob2 »

ben_w wrote: post #292
nottorp wrote: post #252

Earlier quoted context omitted.

We all do that, sometimes where it’s time critical sometimes where it isn’t. But I keep being told “AI” is the second coming of Ahura Mazda so it shouldn’t do stuff like that right?

> Ahura Mazda Niche reference, I like it. But… I only hear of scammers who say, and psychosis sufferers who think, LLMs are *already* that competent. Future AI? Sure, lots of sane-seeming people also think it could go far beyond us. Special purpose ones have in very narrow domains. But current LLMs are only good enough to be useful and potentially economically disruptive, they're not even close to wildly superhuman l…

Sure. If you ask ChatGPT to play chess, it will put up an amateur-level effort at best. Stockfish will indeed wipe the floor with it. But what happens when you ask Stockfish to write a Space Invaders game?

ChatGPT will get better at chess over time. Stockfish will not get better at anything except chess. That's kind of a big difference.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#302

Post by wizzwizz4 »

Philpax wrote: post #287
devmor wrote: post #274

Earlier quoted context omitted.

I'm not really sure what you're saying an LLM did in this case. Inspired a lost sense of curiosity?

Translated a vague natural language query ("cli, extract frame 13s into video") into something immediately actionable with specific examples and explanations, surfacing information that I would otherwise not know how to search for. That's what I've done with my ffmpeg LLM queries, anyway - can't speak for simonw!

DuckDuckGo search results for "cli, extract frame 13s into video" (no quotes):

• https://stackoverflow.com/questions/10957412/fastest-way-to-...

• https://superuser.com/questions/984850/linux-how-to-extract-...

• https://www.aleksandrhovhannisyan.com/notes/video-cli-cheat-...

• https://www.baeldung.com/linux/ffmpeg-extract-video-frames

• https://ottverse.com/extract-frames-using-ffmpeg-a-comprehen...

Search engines have been able to translate "vague natural language queries" into search results for a decade, now. This pre-existing infrastructure accounts for the vast majority of ChatGPT's apparent ability to find answers.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#303

Post by vessenes »

photon_lines wrote: post #201
simonw wrote: post #152

Earlier quoted context omitted.

I scanned the code and understood what it was doing, but I didn't spend much time on it once I'd seen that it worked. If I'm writing code for production systems using LLMs I still review every single line - my personal rule is I need to be able to explain how it works to someone else before I'm willing to commit it. I wrote a whole lot more about my approach to using LLMs to help write "real" code here: https://simon…

This is why I love using the Deep-Seek chain of reason output ... I can actually go through and read what it's 'thinking' to validate whether it's basing its solution on valid facts / assumptions. Either way thanks for all of your valuable write-ups on these models I really appreciate them Simon!

Nota bene - there is a fair amount of research that indicates models outputs and ‘thoughts’ do not necessarily align with their chain of reasoning output.

You can validate this pretty easily by asking some logic or coding questions: you will likely note that a final output is not necessarily the logical output of the end of the thinking; sometimes significantly orthogonal to it, or returning to reasoning in the middle.

All that to say - good idea to read it, but stay vigilant on outputs.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#304

Post by bgwalter »

simonw wrote: post #187
bgwalter wrote: post #176

Earlier quoted context omitted.

Because we are told that they can solve IMO problems. Yet they fail at basic math problems, not only at factorization but also when probing them with relatively basic symbolic math that would not require the invocation of an external program. Also, you know it they fail they could say so instead of giving a hallucinated answer. First the models lie and say that a 20 digit number takes vast amounts of computing. Then,…

That's why the IMO results were so notable: that was one of those moments where new models were demonstrated doing something that they had previously been unable to do.

The results were private and the methodology was not revealed. Even Tao, who was bullish on "AI", is starting to question the process.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#305

Post by greesil »

simonw wrote: post #299
wizzwizz4 wrote: post #298

Earlier quoted context omitted.

It can usually tweak things, if given specific instruction, but it doesn't know when to refactor (and can't reliably preserve functionality when it does), so the program gets further and further away from something sensible until it can't make edits any more.

For serious projects you can address that by writing (or having it write) unit tests along the way, that way it can run in a loop and avoid breaking existing functionality when it adds new changes.

Okay ask it to write unit tests for space invaders next time :)

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#306

Post by ben_w »

CamperBob2 wrote: post #301
ben_w wrote: post #292

Earlier quoted context omitted.

> Ahura Mazda Niche reference, I like it. But… I only hear of scammers who say, and psychosis sufferers who think, LLMs are *already* that competent. Future AI? Sure, lots of sane-seeming people also think it could go far beyond us. Special purpose ones have in very narrow domains. But current LLMs are only good enough to be useful and potentially economically disruptive, they're not even close to wildly superhuman l…

Sure. If you ask ChatGPT to play chess, it will put up an amateur-level effort at best. Stockfish will indeed wipe the floor with it. But what happens when you ask Stockfish to write a Space Invaders game? ChatGPT will get better at chess over time. Stockfish will not get better at anything except chess. That's kind of a big difference.

> ChatGPT will get better at chess over time

Oddly, LLMs got worse at specifically chess: https://dynomight.net/chess/

But even to the general point, there's absolutely no agreement how much better the current architectures can ultimately get, nor how quickly they can get there.

Do they have potential for unbounded improvements, albeit at exponential cost for each linear incremental improvement? Or will they asymptomatically approach someone with 5 years experience, 10 years experience, a lifetime of experience, or a higher level than any human?

If I had to bet, I'd say current models have an asymptomatic growth converging to a merely "ok" performance; and separately claim that even if they're actually unbounded with exponential cost for linear returns, we can't afford the training cost needed to make them act like someone with even just 6 years professional experience in any given subject.

Which is still a lot. Especially as it would be acting like it had about as much experience in every other subject at the same time. Just… not a literal Ahura Mazda.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#307

Post by bsder »

righthand wrote: post #146

Did you understand the implementation or just that it produced a result? I would hope an LLM could spit out a cobbled form of answer to a common interview question. Today a colleague presented data changes and used an LLM to build a display app for the JSON for presentation. Why did they not just pipe the JSON into our already working app that displays this data? People around me for the most part are using LLMs to e…

> However the solutions are absolutely useless for anyone else but the implementer.

Disposable code is where AI shines.

AI generating the boilerplate code for an obtuse build system? Yes, please. AI generating an animation? Ganbatte. (Look at how much work 3Blue1Brown had to put into that--if AI can help that kind of thing, it has my blessings). AI enabling someone who doesn't program to generate some prototype that they can then point at an actual programmer? Excellent.

This is fine because you don't need to understand the result. You have a concrete pass/fail gate and don't care about underneath. This is real value. The problem is that it isn't gigabuck value.

The stuff that would be gigabuck value is unfortunately where AI falls down. Fix this bug in a product. Add this feature to an existing codebase. etc.

AI is also a problem because disposable code is what you would assign to junior programmers in order for them to learn.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#308

Post by CamperBob2 »

Eggpants wrote: post #244
satvikpendem wrote: post #64

Earlier quoted context omitted.

This doesn't make sense thermodynamically because models are far smaller than the training data they purport to hold and recall, so there must be some level of "understanding" going on. Whether that's the same as human understanding is a different matter.

It’s a lossy text compression technique. It’s clever applied statistics. Basically an advanced association rules algorithm which has been around for decades but modified to consider order and relative positions. There is no understanding, regardless of the wants of all the capital investors in this domain.

It’s a lossy text compression technique.

That is a much, much bigger deal than you make it sound like.

Compression may, in fact, be all we need. For that matter, it may be all there is.

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#309

Post by lxgr »

switchbak wrote: post #248
lxgr wrote: post #227

This raises an interesting question I’ve seen occasionally addressed in science fiction before: Could today’s consumer hardware run a future superintelligence (or, as a weaker hypothesis, at least contain some lower-level agent that can bootstrap something on other hardware via networking or hyperpersuasion) if the binary dropped out of a wormhole?

This is what I find fascinating. What hidden capabilities exist, and how far could it be exploited? Especially on exotic or novel hardware. I think much of our progress is limited by the capacity of the human brain, and we mostly proceed via abstraction which allows people to focus on narrow slices. That abstraction has a cost, sometimes a high one, and it’s interesting to think about what the full potential could be…

Abstraction, or efficient modeling of a given system, is probably a feature, not a bug, given the strong similarity between intelligence and compression and all that.

A concise description of the right abstractions for our universe is probably not too far removed from the weights of a superintelligence, modulo a few transformations :)

Re: My 2.5 year old laptop can write Space Invaders in JavaScript now (GLM-4.5 Air)

#310

Post by simonw »

bgwalter wrote: post #304
simonw wrote: post #187

Earlier quoted context omitted.

That's why the IMO results were so notable: that was one of those moments where new models were demonstrated doing something that they had previously been unable to do.

The results were private and the methodology was not revealed. Even Tao, who was bullish on "AI", is starting to question the process.

The same thing has also been achieved by a Google DeepMind team and at least one group of independent researchers using publicly available models and careful promoting tricks.
Post reply on HN