Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

ploy.ai

Post reply on HN
Print view View original

101–110 of 144 posts

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#101

Post by HPsquared »

alasano wrote: post #46
kristianp wrote: post #6

> Numbers like that buy a model a real migration effort. Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style. > But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. The way the L…

I think for a company in AI specifically it's worse. It makes me feel like either 1) you don't use the models enough to know how they write 2) you're not self aware enough to know it matters 3) you're oblivious to the situation overall 4) you don't respect your readers There's no good scenario.

5) You're so immersed in LLM usage that these writing patterns now seem "normal" and you don't even notice them any more.

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#102

Post by fxwin »

throwa356262 wrote: post #99

As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. Wait a moment, did they make the switch based on half a days of playing with Sol? Are these companies ran by teenagers?

I would expect they have production based datasets they evaluate new models against.

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#103

Post by JLO64 »

dannyw wrote: post #69
lcampbell wrote: post #65

> The fix that worked is a schema transform at the provider boundary. For OpenAI-family models only, we rewrite every optional property to be required but nullable, using anyOf: [T, null], which gives the model an explicit way to say “not using this.” I admit, I've only used a bastardized form of MCP, but this smells... wrong? It's not clear to me why the Typescript type definitions would have any influence on (what…

Modern frontier models, including Fable/Opus and 5.6, are often very loose with tool calls, and often don’t follow your schema precisely. For example, see this post for Claude models hallucinating properties for an edit/replace tool call in Pi: https://lucumr.pocoo.org/about/ I suspect some part of this comes from the noticed intelligence degradation when you do constrained decoding. Yes, you’re guaranteed schema val…

I believe the post you were trying to link was this one: https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#104

Post by dominotw »

fxwin wrote: post #102
throwa356262 wrote: post #99

As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. Wait a moment, did they make the switch based on half a days of playing with Sol? Are these companies ran by teenagers?

I would expect they have production based datasets they evaluate new models against.

I would not expect that. They wouldnt have missed mentioning it if that was the case. Its mostly driven by vibes.

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#105

Post by sekai »

throwa356262 wrote: post #99

As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. Wait a moment, did they make the switch based on half a days of playing with Sol? Are these companies ran by teenagers?

AI psychosis is still at all time high

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#106

Post by vessenes »

kristianp wrote: post #6

> Numbers like that buy a model a real migration effort. Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style. > But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. The way the L…

My current claude.md bans the phrase “load-bearing”, and Claude HATES that. It will troll occasionally in comments by saying things like “load-be…most specific”. Like it REALLY loves saying load-bearing. Urgh.

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#107

Post by wqaatwt »

Diti wrote: post #100
yulker wrote: post #83

Earlier quoted context omitted.

this is probably sarcasm, but might actually try this. the visceral negative reaction i have to llm writing makes me instantly want to close the tab

If written sarcasm isn’t properly labeled as sarcasm, per Poe’s law, you may consider that message you replied to be a genuine opinion. I would have downvoted the sarcasm (it doesn’t contribute to the conversation), but I believe it actually is the author’s opinion.

I promise you its not.

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#108

Post by jtrn »

SwtCyber wrote: post #95

Its ironic that under an article with a ton of deep infrastructure insights half the comments are crying about the "forced writing style". What does it matter if claude helped the author clean up the text when inside is a ready-to-use blueprint on how to save 30% of the api budget and fix empty file reads?

One of many reasons I would assume is that people just hate anything related to AI, so they latch on to anything negative they can say.

Another reason is that they mean what they say... That they really, really hate the style of writing, enough to fixate on that.

Personally, I think the people whining about the style are silly. Maybe because I'm terrible at grammar and spelling, but I always just focus on the message, not the delivery. I just care about the concept, facts, the argument, and so forth. The actual grammar and spelling are just trees, while the forest is the point.

Edit: just an infobit: The reason my text isn't full of errors is due to the awesomeness of the dictation and a custom hotkey I have created on my computer, which uses a local LLM to spellcheck any text I have selected and replaces it with the corrected one. Nothing has improved my quality of life and writing more than these two tools!

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#109

Post by avianlyric »

throwa356262 wrote: post #99

As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. Wait a moment, did they make the switch based on half a days of playing with Sol? Are these companies ran by teenagers?

There’s every possibility they got some amount of early access to evaluate GPT5.6, precisely so they could write an article like this.

Re: Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

#110

Post by mkovach »

kristianp wrote: post #6

> Numbers like that buy a model a real migration effort. Such a silly choice of words. I wish the human directing the LLM writing the article put some effort into rewriting the worst examples of LLM style. > But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. The way the L…

> The way the LLMs write (Claude perhaps?) With short phrases separated by colons, commas, or full stops, is so poor and frustrating.

This is exactly why I keep a WRITING.md file alongside AGENTS.md or CLAUDE.md.

Most people spend time telling the model what to build, but very little time telling it how to write. LLMs are surprisingly good at following explicit style guidance if you bother to give it to them.

Mine includes conventions like avoiding unnecessary colons, em dashes, and sentence fragments masquerading as emphasis. Basically, AI-isms and any grammatical errors I tend to make. It also points to writers whose technical prose I admire. Brian Kernighan and Rob Pike are great examples: clear, conversational, and readable without trying to sound important. I've always tried to do two things with documentation: 1) Make it readable and 2) Make people want to read it. A WRITING.md file helps with both points.

If you're generating documentation regularly, it's worth having your agents reference a WRITING.md file. The improvement in readability is often much larger than any gain you'll get from switching to the latest model, and it can keep your documentation consistent between model switches.

Post reply on HN