61–70 of 174 posts
Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...
Earlier quoted context omitted.
I disagree, it improved enormously especially at staying consistent for long-tasks, I have a task running for 32 days (400M+ tokens) via Codex and that's only since gpt-5.4
Has that task accomplished anything yet?
Earlier quoted context omitted.
Models don't know what their cutoff dates are unless told via a system prompt. The proper way to figure out the real cutoff date is to ask the model about things that did not exist or did not happen before the date in question. A few quick tests suggest 5.5's general knowledge cutoff is still around early 2025.
i wonder if they put an older cutoff date into the prompt intentionally so that when asked on more current events it leans towards tool calls / web searches for tuning
Whatever it is, the cutoff date reporting discrepancy isn't new. Back when Musk was making headlines about buying/not buying Twitter, I was able to find recent-ish related news that was published well after the bot's stated cutoff date.
ChatGPT was not yet browsing/searching/using the web at that point. That tool didn't come for another year or so.
Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...
I really like this benchmarking. Have you evaluated the judge benchmark somehow? I'd love to setup my own similar benchmark.
Earlier quoted context omitted.
Has that task accomplished anything yet?
Too soon to tell, give it a billion tokens before we make up our minds
Very bad habit these safeguards. These "safety" filters are counter-productive and even can be dangerous. In my place for example, a lot of doctors are using ChatGPT both to search diagnosis and communicate with non-English speaking patients. Even yourself, when you want to learn about one disease, about some real-world threats, some statistics, self-defense techniques, etc. Otherwise it's like blocking Wikipedia for…
> a lot of doctors are using ChatGPT both to search diagnosis and communicate with non-English speaking patients I think that's the problem. Who's going to claim responsibility when ChatGPT hallucinates or mistranslates a patient's diagnosis and they die? For OpenAI, this would at best be a PR nightmare, so that's why they have safeguards.
A doctor who gets refused by ChatGPT doesn't stop needing to communicate with the patient; they fall back to a worse option (Google Translate, a family member interpreting, guessing). Refusal isn't safety, it's liability-shifting dressed up as safety.
If there's no doctor, no interpreter, no pharmacist, just a person with a sick kid and a phone, then "refuse and redirect to a professional" is advice from a world that doesn't exist for them. The refusal doesn't send them to a better option; there is no better option, it's a large majority of people on this planet.
Hell is paved of good intentions, but open-education and unlimited access to knowledge is very good.
It doesn't change the human nature of some people, bad people stay bad, good people stay good.
About PR, they're optimizing for not being the named defendant in a lawsuit or the subject of a bad news cycle, it's self-interest wearing benevolence as a costume.
This is because harms from answering are punishable (bad PR, unhappy advertisers, unhappy investors, unhappy politicians / dictators, unhappy lobbies, unhappy army, etc); but harms from refusing are invisible and unpunished.
Earlier quoted context omitted.
The doctor would be responsible. I had a choice better a doctor that used AI or not, I would much prefer one that did...
The doctor would be responsible for the accuracy of their translation tool, something they can't verify but you expect them to use?
-> You are in China, you go to emergency, nobody speaks your language
Move hands ? DeepSeek is better than using hands, even Baidu Translate, ChatGPT or whatever you find.
Other solutions are theoretically nice on paper but almost delusional.
An imperfect solution is better than no solution.
==
Similarly, a deaf-person is theorically better with a certified interpreter that can talk with the hands, but they may prefer voice-recognition software or AI tools.
(or... talking with hands is more confusing and annoying or less understandable for them).
Of course ChatGPT transcription can have issues, but that's the difference between the real-world and Silicon Valley's disconnected lawyers world.
==
If ChatGPT says: "sorry I won't be able, please go to see a licensed interpreter, good luck!" then it's just OpenAI trying to save their asses, at your risk/expense.
If you have a choice, you can make the choice, and you can double-check what is said. In other cases, you have no choice, nothing to check, only problems but no hints of solutions.
This is why openness is important.
Earlier quoted context omitted.
I feel like the last 2-3 generations of models (after gpt-5.3-codex) didn't really improve much, just changed stuff around and making different tradeoffs.
I disagree, it improved enormously especially at staying consistent for long-tasks, I have a task running for 32 days (400M+ tokens) via Codex and that's only since gpt-5.4
Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guilamu/llms-wordpress-plugin-benchmark I know it's only on a single benchmark, but I dont understand how it can be so bad...
Your benchmark has Opus 4.7 performing significantly worse than Sonnet 4.6. Even if true on your benchmark, that is not representative of the overall performance of the models.