Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building more with GPT-5.1-Codex-Max

openai.com

Post reply on HN
Print view View original

301–310 of 332 posts

Re: Building more with GPT-5.1-Codex-Max

#301

Post by discreteevent »

pmarreck wrote: post #277
root_axis wrote: post #209

Earlier quoted context omitted.

> Just because you don't understand it, doesn't mean it's "folk magic incantation" It absolutely is folk magic. I think it is more accurate to impugn your understanding than mine. > I don't know the merit to what parent is saying, but it does make some intuitive sense if you think about it. This is exactly what I mean by folk magic. Incantations based on vibes. One's intuition is notoriously inclined to agree with on…

> This is exactly what I mean by folk magic. Incantations based on vibes So, true creativity, basically? lol I mean, the reason why programming is called a “craft” is because it is most definitely NOT a purely mechanistic mental process. But perhaps you still harbor that notion. Ah, I suddenly realized why half of all developers hate AI-assisted coding (I am in the other half). I was a Psych major, so code was always…

> so code was always more “writing” than “gears” to me… It was ALWAYS “magic.”

> I suddenly realized why half of all developers hate AI-assisted coding (I am in the other half).

Thanks for this. It helps me a lot to understand your half. I like my literature and music as much as the next person but when it comes to programming it's all about the mechanics of it for me. I wonder if this really does explain the split that there seems to be in every thread about programming and LLMs

Re: Building more with GPT-5.1-Codex-Max

#302

Post by seunosewa »

Atotalnoob wrote: post #112
999900000999 wrote: post #94

I really would prefer them to start creating customized models. I've vibe coded Godot games extensively. Just about every model I've tried likes to invent imaginary functions. I was really prefer for there to be a way for me to pick model trained in whatever framework I need. Reviewing AI generated code feels like editing a long book, and every now and then you notice some words are just completely made up. You then…

I’ve found writing a MCP server with access to the docs cloned locally does wonders.

If you use cursor you can just attach the documentation. Same thing, different method.

Re: Building more with GPT-5.1-Codex-Max

#303

Post by root_axis »

embedding-shape wrote: post #273
root_axis wrote: post #209

Earlier quoted context omitted.

> Just because you don't understand it, doesn't mean it's "folk magic incantation" It absolutely is folk magic. I think it is more accurate to impugn your understanding than mine. > I don't know the merit to what parent is saying, but it does make some intuitive sense if you think about it. This is exactly what I mean by folk magic. Incantations based on vibes. One's intuition is notoriously inclined to agree with on…

> First of all, system prompts and things like agent.md never leave the context regardless of the length of the session, so the canary has absolutely zero meaning in this situation, making any judgements based on its disappearance totally misguided and simply a case of seeing what you want to see. You're focusing on the wrong thing, ironically. Even if things are in the context, attention is what matters, and the int…

> It's about if the model will pay attention to it, in the Transformers sense, which it doesn't always do.

Right... Which is why the "canary" idea doesn't make much sense. The fact that the model isn't paying attention to the canary instruction doesn't demonstrate that the model has stopped paying attention to some other instruction that's relevant to the task - it proves nothing. If anything, a better performing model should pay less attention to the canary since it becomes less and less relevant as the context is filled with tokens relevant to the task.

Re: Building more with GPT-5.1-Codex-Max

#304

Post by root_axis »

jack_pp wrote: post #265
root_axis wrote: post #236

Earlier quoted context omitted.

I won't begrudge anyone having fun with their tools, but folk magic definitely isn't a necessary step for understanding anything, it's one step removed from astrology.

I'd say the only ones capable of really approaching anything like scientific understanding of how to prompt these for maximum efficacy are the providers not the users. Users can get a glimpse and can try their best to be scientific in their approach however the tool is of such complexity that we can barely skim the surface of what's possible. That is why you see "folk magic", people love to share anecdata because.. t…

> That is why you see "folk magic", people love to share anecdata because.. that's what most people have. They either don't have the patience, the training or simply the time to approach these tools with rational rigor.

Yes. That's exactly the point of my comment. Users aren't performing anything even remotely approaching the level of controlled analysis necessary to evaluate the efficacy of their prompt magic. Every LLM thread is filled with random prompt advice that varies wildly, offered up as nebulously unfalsifiable personality traits (e.g. "it makes the model less aggressive and more circumspect"), and all with the air of a foregone conclusion's matter-of-fact confidence. Then someone always replies with "actually I've had the exact opposite experience with [some model], it really comes down to [instructing the model to do thing]".

Re: Building more with GPT-5.1-Codex-Max

#305

Post by root_axis »

pmarreck wrote: post #277
root_axis wrote: post #209

Earlier quoted context omitted.

> Just because you don't understand it, doesn't mean it's "folk magic incantation" It absolutely is folk magic. I think it is more accurate to impugn your understanding than mine. > I don't know the merit to what parent is saying, but it does make some intuitive sense if you think about it. This is exactly what I mean by folk magic. Incantations based on vibes. One's intuition is notoriously inclined to agree with on…

> This is exactly what I mean by folk magic. Incantations based on vibes So, true creativity, basically? lol I mean, the reason why programming is called a “craft” is because it is most definitely NOT a purely mechanistic mental process. But perhaps you still harbor that notion. Ah, I suddenly realized why half of all developers hate AI-assisted coding (I am in the other half). I was a Psych major, so code was always…

> So, true creativity, basically? lol

Creativity is meaningless without well defined boundaries.

> it is most definitely NOT a purely mechanistic mental process.

So what? Nothing is. Even pure mathematics involves deep wells of creativity.

> Ah, I suddenly realized why half of all developers hate AI-assisted coding

Just to be clear, I don't hate AI assisted coding, I use it, and I find that it increases productivity overall. However, it's not necessary to indulge in magical thinking in order to use it effectively.

> The only job where literally writing down words in a certain way produces machines that eliminate human labor. What better definition of magic is there, actually?

If you want to use "magic" as a euphemism for the joys of programming, I have no objection, when I say magic here I'm referring to anecdotes about which sequences of text produce the best results for various tasks.

> Determinism. That’s what you’re mad about, I’m thinking. And I completely get you there- how can I consider a “flagging test” to be an all-hands-on-deck affair while praising code output from a nondeterministic machine running off arbitrary prompt words that we don’t, and can’t, even know whether they are optimal?

I'm not mad about anything. It doesn't matter whether or not LLMs are deterministic, they are statistical, and vibes based advice is devoid of any statistical power.

Re: Building more with GPT-5.1-Codex-Max

#306

Post by embedding-shape »

root_axis wrote: post #303
embedding-shape wrote: post #273

Earlier quoted context omitted.

> First of all, system prompts and things like agent.md never leave the context regardless of the length of the session, so the canary has absolutely zero meaning in this situation, making any judgements based on its disappearance totally misguided and simply a case of seeing what you want to see. You're focusing on the wrong thing, ironically. Even if things are in the context, attention is what matters, and the int…

> It's about if the model will pay attention to it, in the Transformers sense, which it doesn't always do. Right... Which is why the "canary" idea doesn't make much sense. The fact that the model isn't paying attention to the canary instruction doesn't demonstrate that the model has stopped paying attention to some other instruction that's relevant to the task - it proves nothing. If anything, a better performing mod…

> it proves nothing

Correct, but I'm not sure anyone actually claimed it proved anything at all? To be entirely sure, I don't know what you're arguing against/for here.

Re: Building more with GPT-5.1-Codex-Max

#307

Post by root_axis »

embedding-shape wrote: post #284
pimeys wrote: post #240

Earlier quoted context omitted.

Wait, I think it's the other way around. Claude will just go circles with bad decisions forever, never stops. Codex have multiple times told me it is not able to do this task, and stops.

I think this closer to the crux of a major problem. Seemingly people have vastly different responses even for the same system/developer/user prompts, and I myself can feel a different in quality of the responses depending on when I use the hosted APIs, while hosted models always have consistent results. For example, after 19:00 sometime (GMT+1), the response quality of both OpenAI and Anthropic (their hosted UIs) see…

That's the nature of statistical output, even minus all the context manipulation going on in the background.

You say the outputs "seem" to drop off at a certain time of day, but how would you even know? It might just be a statistical coincidence, or someone else might look at your "bad" responses and judge them to be pretty good actually, or there might be zero statistical significance to anything and you're just seeing shapes in the clouds.

Or you could be absolutely right. Who knows?

Re: Building more with GPT-5.1-Codex-Max

#308

Post by inopinatus »

mandelbrotwurst wrote: post #195
nico wrote: post #65

Earlier quoted context omitted.

> Claude basically disregards your instructions (CLAUDE.md) entirely A friend of mine tells Claude to always address him as “Mr Tinkleberry”, he says he can tell when Claude is not paying attention to the instructions on CLAUDE.md when Claude stops calling him “Mr Tinkleberry” consistently

Why would the fact that it failed to follow one instruction increase the likelihood that it failed to follow others within the same response?

Because the LLM is not a cognitive entity with a will, it is a plausibility engine trained on human-authored text and interactions.

So when you tell it that it made a mistake, or is stupid, then those things are now prompting it to be more of the same.

And only slightly more obliquely: if part of the context includes the LLM making mistakes, expect similar activations.

Best results come if you throw away such prompts and start again. That is, iterate outside the function, not inside it.

Re: Building more with GPT-5.1-Codex-Max

#309

Post by pmarreck »

discreteevent wrote: post #301
pmarreck wrote: post #277

Earlier quoted context omitted.

> This is exactly what I mean by folk magic. Incantations based on vibes So, true creativity, basically? lol I mean, the reason why programming is called a “craft” is because it is most definitely NOT a purely mechanistic mental process. But perhaps you still harbor that notion. Ah, I suddenly realized why half of all developers hate AI-assisted coding (I am in the other half). I was a Psych major, so code was always…

> so code was always more “writing” than “gears” to me… It was ALWAYS “magic.” > I suddenly realized why half of all developers hate AI-assisted coding (I am in the other half). Thanks for this. It helps me a lot to understand your half. I like my literature and music as much as the next person but when it comes to programming it's all about the mechanics of it for me. I wonder if this really does explain the split t…

Can you tell when code is “beautiful”?

That is an artful quality, not an engineering one, even if the elegance leads to superior engineering.

As an example of beauty that is NOT engineered well, see the quintessential example of quicksort implemented in Haskell. Gorgeously simple, but not performant.

Re: Building more with GPT-5.1-Codex-Max

#310

Post by pmarreck »

sosjsbsb wrote: post #295
pmarreck wrote: post #277

Earlier quoted context omitted.

> This is exactly what I mean by folk magic. Incantations based on vibes So, true creativity, basically? lol I mean, the reason why programming is called a “craft” is because it is most definitely NOT a purely mechanistic mental process. But perhaps you still harbor that notion. Ah, I suddenly realized why half of all developers hate AI-assisted coding (I am in the other half). I was a Psych major, so code was always…

> I was a Psych major, so code was always more “writing” than “gears” to me… It was ALWAYS “magic. The magic is supposed to disappear as you grow (or you’re not growing). The true magic of programming is you can actually understand what once was magic to you. This is the key difference I’ve seen my entire career - good devs intimately know “a layer below” where they work. > Perhaps because humans are also nondetermin…

I’m not talking about “magic” as in “I don’t understand how it works.”

I’m talking “magic” as in “all that is LITERALLY happening is that bits are flipping and logic gates are FLOPping and mice are clicking and keyboards are clacking and pixels are changing colors in different patterns… and yet I can still spend hours playing games or working on some code that is meaningful to me and that other people sometimes like because we have literally synthesized a substrate that we apply meaning to.”

We are literally writing machines into existence out of fucking NOTHING!

THAT “magic.” Do you not understand what I’m referring to? If not, maybe lay off the nihilism/materialism pipe for a while so you CAN see it. Because frankly I still find it incredible, and I feel very grateful to have existed now, in this era.

And this is where the connection to writing comes in. A writer creates ideas out of thin air and transmits them via paper or digital representation into someone else’s head. A programmer creates ideas out of thin air that literally fucking DO things on their own (given a general purpose computing hardware substrate)

Post reply on HN