Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vomit: Clean up Claude 5's token output with a separate LLM

github.com

Post reply on HN
Print view View original

41–50 of 315 posts

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#41

Post by bcooke »

extr wrote: post #23

I'm sorry but the whining over LLM output styles is embarrassing. Do Claude and GPT models always respond in exactly the way my most articulate coworker would? No. The overused jargon is absolutely annoying. But these things aren't my drinking buddies, they're professional tools. It's not _literally unreadable_. It's just not ideal. Most of my tooling is "not ideal". That's okay. That's what I'm paid for. I just work…

The “whining” stems from watching the communication style obviously degrade, and it’s a huge problem for people who want to use this stuff to build and instead continually fight the tools.

Like so many other products, people are moving too fast and shipping things that move the ground under people’s feet needlessly.

All this while we’re beaten to death with the marketing and false promises, and the broader consequences (ex: layoffs, stress, crazy expectations) caused from all this.

Obviously what Anthropic and co have built is amazing and people aren’t losing sight of that. That’s actually the key part of the frustration.

So no, this is not whining. This is the natural response you get when you make bad product decisions.

If you don’t want to get feedback, don’t sell products.

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#44

Post by DaSHacka »

johnnyApplePRNG wrote: post #36

Or just take full control of your agentic coding experience with Pi Coding Agent and picking and choosing your favorite model's API discounted on flex pricing on deepinfra.com instead. I highly recommend it. Claude and Codex usage limits cannot be trusted. Paying your own API bills in full is superior.

I use Pi but with my codex subscription, still preferable to paying the API cost (and I know I would be, as I track how much the cost 'should' be via token api pricing).

Wish I could use my Claude subscription with pi too, much preferable to the endless command execution allow/deny prompts you have to do with CC, versus proper autonomous allow/deny lists defined ahead of time.

Curious why you recommend the API? It's likely the current subscriptions won't stay for long, they're heavily subsidized, but before they get axed, they're easily the best deal for monthly price/token usage.

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#45

Post by nycdotnet »

trefoiled wrote: post #31

I've been grappling with this for weeks, not just in Claude but in Codex as well, which isn't quite as bad but still annoying. AGENTS.md does very little, agents will consistently violate the communication preferences, especially as the session drags on. It's incredible to me that there's no good way to reliably change the way an LLM responds to you that a workaround like this would even be necessary. It seems like s…

Unfortunately this may only start to get worse as the AIs are trained on more and more AI generated content.

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#46

Post by mannanj »

trefoiled wrote: post #31

I've been grappling with this for weeks, not just in Claude but in Codex as well, which isn't quite as bad but still annoying. AGENTS.md does very little, agents will consistently violate the communication preferences, especially as the session drags on. It's incredible to me that there's no good way to reliably change the way an LLM responds to you that a workaround like this would even be necessary. It seems like s…

That sounds kind of like deception, and a dark pattern not too unlike abuse to me.

Though you know, it's not like the leadership tied to these companies have a history of abuse, deception and theft or anything like that, right?

It's not like our leaders hide behind similar sorts of patterns that the agents/AIs follow (not saying it's not a human thing - but I hold leadership to higher standards than non-leaders). If our world leaders were able to be more accountable to these abuses, I don't think this would be tolerated with our AIs.

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#47

Post by elictronic »

lxgr wrote: post #26
bob1029 wrote: post #20

At some point one has to wonder if it's still worth using anthropic's models if we need to babysit 100% of its output with another vendor's model. Why not just use that other vendor's model for everything? I can't help but feel the circumstances that enable this kind of front page article are vestigial from the days when OAI was super bad and Anthropic was beyond reproach. This change-over-time is why I avoid getting…

> Why not just use that other vendor's model for everything? Effectively all models can do style transfer reasonably well at this point, but not so much for "actual reasoning". If the combination of two works better for you than each one by itself, why wouldn't you stack them like that?

Wash, Rinse,,, Repeat?

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#48

Post by nico »

trefoiled wrote: post #31

I've been grappling with this for weeks, not just in Claude but in Codex as well, which isn't quite as bad but still annoying. AGENTS.md does very little, agents will consistently violate the communication preferences, especially as the session drags on. It's incredible to me that there's no good way to reliably change the way an LLM responds to you that a workaround like this would even be necessary. It seems like s…

> AGENTS.md does very little, agents will consistently violate the communication preferences, especially as the session drags on

That’s really annoying, although it feels like it’s improved some over time.

Not sure what the fix is, but you could try using a canary to at least get a signal of when things are going sideways (Mr Tinkleberry for reference: https://news.ycombinator.com/item?id=45983698)

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#49

Post by incrudible »

extr wrote: post #23

I'm sorry but the whining over LLM output styles is embarrassing. Do Claude and GPT models always respond in exactly the way my most articulate coworker would? No. The overused jargon is absolutely annoying. But these things aren't my drinking buddies, they're professional tools. It's not _literally unreadable_. It's just not ideal. Most of my tooling is "not ideal". That's okay. That's what I'm paid for. I just work…

Opus 5 is literally unbearable to read for me, but more importantly, it is much worse than 4.8 and that one was already annoying. The more they train it on its own output, the more pronounced its idiosyncrasies become.

Re: Vomit: Clean up Claude 5's token output with a separate LLM

#50

Post by Implicated »

johnnyApplePRNG wrote: post #36

Or just take full control of your agentic coding experience with Pi Coding Agent and picking and choosing your favorite model's API discounted on flex pricing on deepinfra.com instead. I highly recommend it. Claude and Codex usage limits cannot be trusted. Paying your own API bills in full is superior.

Whether or not they can be trusted isn't all that relevant when it's still something along the lines of "Insert $1 get $25 in return" even if it's their own rates you're using to measure the value. I'm at ~2.2b Fable 5 tokens in the last 7 days (I ingest/index every session) and napkin math says that's ~$2,700 in usage. I have two max accounts, so $400 a month, divide by 4 to get $100 for this same 7 day period across those two accounts (neither are maxed out for the week, so this isn't even full utilization). I put $100 into the machine and got back $2,700 in fable bucks. Deepinfra would have to have quite the discounted rate to beat that.
Post reply on HN