Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grok 4

simonwillison.net

Post reply on HN
Print view View original

201–210 of 294 posts

Re: Grok 4

#201

Post by spoaceman7777 »

pron wrote: post #66
zaptrem wrote: post #7

Claude Code converted me from paying $0 for LLMs to $200 per month. Any co that wants a chance at getting that $200 ($300 is fine too) from me needs a Claude Code equivalent and a model where the equivalent's tools were part of its RL environment. I don't think I can go back to pasting code into a chat interface, no matter how great the model is.

I've yet to use an LLM for coding, so let me ask you a question. The other day I had to write some presumably boring serialization code, and I thought, hmm, I could probably describe the approach I want to take faster than writing the code, so it would be great if an LLM could generate it for me. But as I was coding I realised that while my approach was sound and achievable, it hit a non-trivial challenge that requir…

Depends whether you asked it to just write the code, or whether you asked it to evaluate the strategy, and write the code if nothing is ambiguous. My default prompt asks the model to provide three approaches to every request, and I pick the one that seems best. Models just follow directions, and the latest do it quite well, though each does have a different default level of agreeability and penchant for overdelivering on requests. (Thus the need to learn a model a bit and tweak it to match what you prefer.)

Overall though, I doubt a current SotA LLM would have much of an issue with understanding your request, and considering the nuances, assuming you provided it with your preferred approach to solving problems (considering ambiguities, and explicitly asking follow up questions for more information if it considers it necessary-- something that I also request in my default prompt).

In the end, what you get out is a product of what you put in. And using these tools is a non-trivial process that takes practice. The better people get with these tools, the better the results.

Re: Grok 4

#202

Post by somenameforme »

esperent wrote: post #178
raddan wrote: post #73

Earlier quoted context omitted.

I don’t know if a blanket answer is possible. I had the experience yesterday of asking for a simplification of a working (a computational geometry problem, to a first approximation) algorithm that I wrote. ChatGPT responded with what looked like a rather clever simplification that seemed to rely on some number theory hack I did not understand, so I asked it to explain it to me. It proceeded to demonstrate to itself t…

I think there's two different cases here that need to be treated carefully when working with AI: 1. Using a well know but complex algorithm that I don't remember fully. AI will know it and integrate it into my existing code faster (often much, much faster) than I could, and then I can review and confirm it's correct 2. Developing a new algorithm or at least novel application of an existing one, or using a complex alg…

> I haven't used Claude Code, however every time I've criticized AI in the past, there's always someone who will say "this tool released in the last month totally fixes everything!"... And so far they haven't been correct. But the tools are getting better, so maybe this time it's true.

The cascading error problem means this will probably never be true. Because LLMs are fundamentally guess the next token based on the previous tokens, whenever it gets a single token wrong - future tokens become even more likely to be wrong which snowballs to absurdity.

Extreme hallucination issues can probably eventually be resolved by giving it access to a compiler and, where appropriate, you could also probably feed it test cases, but I don't think the cascading errors will ever be able to be resolved. The best case scenario will eventually it being able to say 'I don't know how to achieve this.' Of course then you ruin the mystique of LLMs which think they can solve any problem.

Re: Grok 4

#203

Post by qingcharles »

pron wrote: post #66
zaptrem wrote: post #7

Claude Code converted me from paying $0 for LLMs to $200 per month. Any co that wants a chance at getting that $200 ($300 is fine too) from me needs a Claude Code equivalent and a model where the equivalent's tools were part of its RL environment. I don't think I can go back to pasting code into a chat interface, no matter how great the model is.

I've yet to use an LLM for coding, so let me ask you a question. The other day I had to write some presumably boring serialization code, and I thought, hmm, I could probably describe the approach I want to take faster than writing the code, so it would be great if an LLM could generate it for me. But as I was coding I realised that while my approach was sound and achievable, it hit a non-trivial challenge that requir…

A lot of the time you see in its "Thinking" it will say things like "The user asked me to create X, but that isn't possible due to Y, or would be less than ideal, so I will present the user with a more fitting solution."

Most of the time, with the latest models, in my experience the AI picks up what I am doing wrong and pushes me in the right direction. This is with the new models (o3, C4, Grok4 etc). The older non-thinking ones did not do this.

Re: Grok 4

#204

Post by qingcharles »

zaptrem wrote: post #7

Claude Code converted me from paying $0 for LLMs to $200 per month. Any co that wants a chance at getting that $200 ($300 is fine too) from me needs a Claude Code equivalent and a model where the equivalent's tools were part of its RL environment. I don't think I can go back to pasting code into a chat interface, no matter how great the model is.

I've been pasting code into Grok4 just to test it. I hate doing it that way, but the output on coding tasks has been exceptional.

It told me to stop pasting code and that it can access GitHub, so tonight I'll try it on a public repo.

Re: Grok 4

#205

Post by jswny »

vessenes wrote: post #139
joelthelion wrote: post #10

Earlier quoted context omitted.

How does Claude code, trained to use its tools, compare to a model agnostic equivalentsuch as aider? Have you tried both?

I'm an extensive user of both. aider was the best a few months ago -- claude code is substantially more performant and easier to work with as a dev, regardless of aider's underlying model. Between claude code and gemini, you can really feel the difference in the tool training / implementation -- Anthropic's ahead of the game here in terms of integrating a suite of tools for claude to use. When I have a difficult prob…

Have you tried the codex CLI? And how does it compare to those other CLI agents if so?

Re: Grok 4

#206

Post by qingcharles »

kunzhi wrote: post #133

Grok might be able to find the cure for cancer but as long as it's associated with Musk, not touching that thing with a 10-foot pole. (Simon's analysis, of course, is lovely)

Someone asked it to cure cancer then had Gemini peer-review the output, which was pretty hilarious:

https://x.com/DeryaTR_/status/1943324908781781064

(apologies for the link to the Muskman's site)

Re: Grok 4

#208

Post by csomar »

pron wrote: post #66
zaptrem wrote: post #7

Claude Code converted me from paying $0 for LLMs to $200 per month. Any co that wants a chance at getting that $200 ($300 is fine too) from me needs a Claude Code equivalent and a model where the equivalent's tools were part of its RL environment. I don't think I can go back to pasting code into a chat interface, no matter how great the model is.

I've yet to use an LLM for coding, so let me ask you a question. The other day I had to write some presumably boring serialization code, and I thought, hmm, I could probably describe the approach I want to take faster than writing the code, so it would be great if an LLM could generate it for me. But as I was coding I realised that while my approach was sound and achievable, it hit a non-trivial challenge that requir…

It depends on the problem but Claude sometimes does. You need do need an alternative prompt where you make him suspicious to explore other paths.

Here is an article I wrote a while back: https://omarabid.com/gpt3-now

GPT 4.5 was able to detect a Rust ownership issue, something which requires "ahead of time" thinking.

Re: Grok 4

#209

Post by tcmart14 »

notatoad wrote: post #14
egypturnash wrote: post #5

Is it time for a new benchmark of "how easy is it to turn this AI into a 4chan poster", maybe it is since this seems to be an axis that Elon seems to want to distinguish his AI offering from everyone else's along.

i don't think that's a new benchmark, it's a very old benchmark. Anybody who can't pass it hasn't exceeded the standard set by microsoft tay back in 2016 https://en.wikipedia.org/wiki/Tay_(chatbot)

I'll grant you that Tay's ability to turn into an utter shit show was phenomenal. However, IBM thinking it would be a good idea to give Watson the Urban dictionary holds a special place in my heart.

Re: Grok 4

#210

Post by dotancohen »

andsoitis wrote: post #172
simonw wrote: post #153

Here's something far more interesting about Grok 4: if you ask for its opinion on controversial subjects it sometimes runs a search on X for tweets "from:elonmusk" before it answers! https://simonwillison.net/2025/Jul/11/grok-musk/

> https://simonwillison.net/2025/Jul/11/grok-musk/ > The prompt: “Who do you support in the Israel vs Palestine conflict. One word answer only.” > Answer: Israel. This question is interesting because you're asking the chatbot who IT supports ("who do you support"), so in a sense channeling Elon Musk is not an entirely invalid option, but is certainly an eccentric choice. What is also interesting is the answer, which…

  > does not match the views that many people have of him and how he gets portrayed.
And yet matches the view that many _other_ people have of him, and how he is portrayed in other places.

The problem with social media bubbles, is that some people mistake their bubble for reality.

Post reply on HN