Notes on OpenAI o3-mini

Well, the post said o3-mini did great in the beginning, so it’s likely something other than reasoning causing the poor performance towards the end.

> The model accepts up to 200,000 tokens of input, an improvement on GPT-4o’s 128,000.

So finally ChatGPT catches up with Claude which has a 200,000 token input limit ever since.

Claude with its projects feature is my go to tool for working on projects that I have to work on for weeks and months. Now I see a possible alternative.

Yikes! Sounds to me like reliable longer form translation is very much not something you can trust to these models. Thanks for sharing.

As I explained in a sister comment, it is not possible to rate translation quality objectively, as opinions and positions about what constitutes a good translation vary. But in my tests of reasoning models since the release of o1-preview, they have not seemed as reliable as the straight nonreasoning versions of ChatGPT, Claude, or Gemini. The translation process itself usually doesn’t seem to require the kind of multistep thinking those reasoning models can be good at.

For more than a year, regular LLMs, when properly prompted, have been able to produce translations that would be indistinguishable from those of some professional translators for some types of translation.

General-purpose LLMs are best for translating straight expository prose without much technical or organization-specific vocabulary. Results are mixed for texts containing slang, dialogue, poetry, archaic language, etc.—partly because people’s tastes differ for how such texts should be translated.

Because most translators are freelancers, it’s hard to get a handle on what impact LLMs have been having on their workloads overall. I have heard reports from experienced translators who have seen work drop off precipitously and have had to change careers, while others report an increase in their workloads over the past two years.

Many translation jobs involve confidential material, and some translators may be hanging onto their jobs because their clients or employers do not allow the use of cloud-based LLMs. That safety net won’t be in place forever, though.

I suspect that those who work directly with translation clients and who are personally known and trusted by their clients will be able to keep working, using LLMs as appropriate to speed up and improve the quality of their work. That’s the position I am fortunate to be in now.

But translators who do piecework through translation agencies or online referrers like Fiverr will have a hard time competing with the much faster and cheaper—and often equally good—LLMs.

I made a few videos about LLMs and translation a couple of years ago. Parts of them are out of date, but my basic thinking hasn’t changed too much since then. If you’re interested:

“Translating with ChatGPT”

https://youtu.be/najKN2bXqCo

“Can GPT-4 translate literature?”

https://youtu.be/5KKDCp3OaMo

“What do translators think about GPT?”

https://www.youtube.com/watch?v=8JUepj7wIl0

I’m planning to make a few more videos on the topic soon, this time focusing on how I use LLMs in my own translation work.

I think there is a misunderstanding here. I wasn't looking to dunk on or shame anyone, only to make an important point. A lot of software engineers are not doing great right now, even though many here may be doing fine. That's a problem bigger than anyone's ego.

And I didn't go back that far, spent about 10 seconds looking.

As for my other comments, at least 80-90% of them have a positive vote ratio, even though I deliberately voice unpopular opinions. Most of the negatives have to do with politics, or Elon Musk, which people here tend to feel strongly about.

Considering that heated debates are not unwelcome here, I don't think that makes me a terrible person. That being said I'll try to be a little more careful before posting in the future, and I broadly agree with the point you were aiming for with your comment. Apologies to the GP if I caused any unwelcome feelings.