Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI o3-mini

simonwillison.net

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Notes on OpenAI o3-mini

#61

Post by Wowfunhappy »

tkgally wrote: post #59
dr_dshiv wrote: post #57

Earlier quoted context omitted.

Im curious if you know of any tools, strategies or papers about this topic. I’ve experienced the same thing with long-form translation. I would expect that chunking the translation (page or paragraph at a time) would fix the missing paragraph problem but potentially the loss of context would cause other problems. This is such a common use case — would love to find resources.

There might be some papers or other guides out there, but their advice will be based on whatever tools happened to be available at the time they were written and on the particular types of translations the authors cared about. The technology is advancing so rapidly that you might be better off just experimenting with various LLMs and prompts for texts and language pairs you are interested in. I started using LLMs for…

> Please note that my translation process above is focused on quality, not on speed. If I needed to translate a large volume of text more quickly, I would write a program to do the translation, checking, and rechecking through API calls, accepting the fact that I would not be able to check and polish the translation manually as I do now.

Would you still expect this to produce a better result than Deepl or other purpose-built translation software?

Re: Notes on OpenAI o3-mini

#62

Post by idunnoman1222 »

tkgally wrote: post #59
dr_dshiv wrote: post #57

Earlier quoted context omitted.

Im curious if you know of any tools, strategies or papers about this topic. I’ve experienced the same thing with long-form translation. I would expect that chunking the translation (page or paragraph at a time) would fix the missing paragraph problem but potentially the loss of context would cause other problems. This is such a common use case — would love to find resources.

There might be some papers or other guides out there, but their advice will be based on whatever tools happened to be available at the time they were written and on the particular types of translations the authors cared about. The technology is advancing so rapidly that you might be better off just experimenting with various LLMs and prompts for texts and language pairs you are interested in. I started using LLMs for…

This doesn’t address OP‘s concern at all about the quality degrading as the number of tokens reaches the maximum memory size or perhaps surpasses it.

Re: Notes on OpenAI o3-mini

#63

Post by tkgally »

Wowfunhappy wrote: post #61
tkgally wrote: post #59

Earlier quoted context omitted.

There might be some papers or other guides out there, but their advice will be based on whatever tools happened to be available at the time they were written and on the particular types of translations the authors cared about. The technology is advancing so rapidly that you might be better off just experimenting with various LLMs and prompts for texts and language pairs you are interested in. I started using LLMs for…

> Please note that my translation process above is focused on quality, not on speed. If I needed to translate a large volume of text more quickly, I would write a program to do the translation, checking, and rechecking through API calls, accepting the fact that I would not be able to check and polish the translation manually as I do now. Would you still expect this to produce a better result than Deepl or other purpo…

I don’t know. I stopped using DeepL sometime last year as I found its inability to be prompted about the purpose of the translation to be too limiting for my purposes. At that time, it also had problems with things like maintaining coherent pronoun reference over multiple paragraphs—problems not seen with LLMs. Perhaps DeepL has gotten better since. In any case, I’m sure they have a lot of smart developers and understand well the problems of translation, so I have no reason to think that I would be able to produce a better fully automated translation system than they have.

Re: Notes on OpenAI o3-mini

#65

Post by dpcpnry »

tkgally wrote: post #59
dr_dshiv wrote: post #57

Earlier quoted context omitted.

Im curious if you know of any tools, strategies or papers about this topic. I’ve experienced the same thing with long-form translation. I would expect that chunking the translation (page or paragraph at a time) would fix the missing paragraph problem but potentially the loss of context would cause other problems. This is such a common use case — would love to find resources.

There might be some papers or other guides out there, but their advice will be based on whatever tools happened to be available at the time they were written and on the particular types of translations the authors cared about. The technology is advancing so rapidly that you might be better off just experimenting with various LLMs and prompts for texts and language pairs you are interested in. I started using LLMs for…

Thanks for sharing the workflow.

I also use many LLMs to assist my translation tasks.

Recently, I have also been using Google AI Studio [1], and I find the its latest models to be smarter.

[1] https://aistudio.google.com/app/prompts/new_chat

Re: Notes on OpenAI o3-mini

#66

Post by simonw »

tkgally wrote: post #59
dr_dshiv wrote: post #57

Earlier quoted context omitted.

Im curious if you know of any tools, strategies or papers about this topic. I’ve experienced the same thing with long-form translation. I would expect that chunking the translation (page or paragraph at a time) would fix the missing paragraph problem but potentially the loss of context would cause other problems. This is such a common use case — would love to find resources.

There might be some papers or other guides out there, but their advice will be based on whatever tools happened to be available at the time they were written and on the particular types of translations the authors cared about. The technology is advancing so rapidly that you might be better off just experimenting with various LLMs and prompts for texts and language pairs you are interested in. I started using LLMs for…

This comment is solid gold! I will definitely be sending people to it.

Would make a great article for your own site, otherwise I'm happy to link to it here instead.

Re: Notes on OpenAI o3-mini

#67

Post by zyklu5 »

maxdo wrote: post #2

How would you rate it against Claude ? Didn’t test it yet, but o1 pro didn’t perform as good

Claude is still better in my opinion.

There's a suite of code-related tasks -- covering a diversity of areas, including dev ops, media manipulation etc., derived from issues I have faced over the years -- I perform for every new release. No model has solved the set of issues solved in one go but Claude still remains the best.

An example of the sort of problems in the suite:

> I have a special problematically encoded mp4 file with a subtle issue (something I ran into a couple of years ago while fixing a bug in a computer vision pipeline). In the question prompt I also pass the output of ffprobe and ask for the ffmpeg command that'll fix it. Only Claude has figured the real underlying issue out (after 4 interactions).

Re: Notes on OpenAI o3-mini

#68

Post by fragmede »

ec109685 wrote: post #41
nycdatasci wrote: post #36

Earlier quoted context omitted.

This is a great anecdote and I hope others can learn from it. R1, o1, and o3-mini work best on problems that have a “correct” answer (as in code that passes unit tests, or math problems). If multiple professional translators are given the same document to translate, is there a single correct translation?

Well, the post said o3-mini did great in the beginning, so it’s likely something other than reasoning causing the poor performance towards the end.

boredom, perhaps?

Re: Notes on OpenAI o3-mini

#69

Post by tkgally »

simonw wrote: post #66
tkgally wrote: post #59

Earlier quoted context omitted.

There might be some papers or other guides out there, but their advice will be based on whatever tools happened to be available at the time they were written and on the particular types of translations the authors cared about. The technology is advancing so rapidly that you might be better off just experimenting with various LLMs and prompts for texts and language pairs you are interested in. I started using LLMs for…

This comment is solid gold! I will definitely be sending people to it. Would make a great article for your own site, otherwise I'm happy to link to it here instead.

Thanks! Feel free to link to the HN comment. That will encourage me to make a video or two for YouTube demonstrating for a wider audience how I use AI for translation. I hope to do that within a few weeks.

Re: Notes on OpenAI o3-mini

#70

Post by learning-tr »

tkgally wrote: post #59
dr_dshiv wrote: post #57

Earlier quoted context omitted.

Im curious if you know of any tools, strategies or papers about this topic. I’ve experienced the same thing with long-form translation. I would expect that chunking the translation (page or paragraph at a time) would fix the missing paragraph problem but potentially the loss of context would cause other problems. This is such a common use case — would love to find resources.

There might be some papers or other guides out there, but their advice will be based on whatever tools happened to be available at the time they were written and on the particular types of translations the authors cared about. The technology is advancing so rapidly that you might be better off just experimenting with various LLMs and prompts for texts and language pairs you are interested in. I started using LLMs for…

In your experience which LLM had the best pronunciation ?
Post reply on HN