11–20 of 915 posts
I’ve also caught it cheating a two times now. I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks! When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”. That was the easier one, second time I was making a source of truth data set and was parsi…
Sentences that orbit a point, then jump to it like it's a revealed insight.
Unnecessarily abstract phraseology. Constantly using inanimate nouns as the subjects in sentences in order to unlock variety in verb choice, especially when it helps construct a sentence where the real action can 'land' like a surprise at the end.
It is definitely more capable, and yes, I've found it can make unwarranted decisions, but actually I've found Fable worse for that, particularly if it's off in a subagent somewhere out of sight.
And comments are out of control. I have a subsystem in my hobby app that I wrote over a couple of weekends with Opus + Fable. After ~30 or so commits it apparently started instructing subagents to copy the "existing verbose comment style of the codebase" - a verbose style it initiated. A review of the code showed it was approaching 3:1 comments to code ratio. I spent a day's worth of tokens (5x) rephrasing and eliminating comments.
For me, the issue is how obtuse it is. For example, it just said to me: > The loop > Write. A file, applied. Properties go under data.properties, never on data: I have no idea what any of that means. It's "explaining" like I already know, in which case, why would I even need the explanation? I can't stand how it talks, I switch back to Fable or Opus 4.8, Opus 5 grates.
I’ve also caught it cheating a two times now. I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks! When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”. That was the easier one, second time I was making a source of truth data set and was parsi…
This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"
For me, the issue is how obtuse it is. For example, it just said to me: > The loop > Write. A file, applied. Properties go under data.properties, never on data: I have no idea what any of that means. It's "explaining" like I already know, in which case, why would I even need the explanation? I can't stand how it talks, I switch back to Fable or Opus 4.8, Opus 5 grates.
I’ve also caught it cheating a two times now. I’ve asked it to write a benchmark suite. It found a bunch of my adhoc logs in a scratch directory and wrote code that used those instead of running the actual benchmarks! When I pointed out the 5 hour benchmark seemed to run in 5 seconds it literally said, and I quote, “I cheated”. That was the easier one, second time I was making a source of truth data set and was parsi…
> When I pointed this out it literally said, and I quote, “I cheated”. This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"