So far, it seems like this is the hierarchy o1 > GPT-4o > o3-mini > o1-mini > GPT-4o-mini o3 mini system card: https://cdn.openai.com/o3-mini-system-card.pdf
You cannot compare GPT-4o and o*(-mini) because GPT-4o is not a reasoning model.
OpenAI O3-Mini
801–810 of 944 posts
Re: OpenAI O3-Mini
#802Earlier quoted context omitted.
Maybe they found a need to quantize it further for release, or lobotomise it with more "alignment".
> lobotomise Anyone can write very fast software if you don't mind it sometimes crashing or having weird bugs. Why do people try to meme as if AI is different? It has unexpected outputs sometimes, getting it to not do that is 50% "more alignment" and 50% "hallucinate less". Just today I saw someone get the Amazon bot to roleplay furry erotica. Funny, sure, but it's still obviously a bug that a *sales bot* would do th…
Isn’t that exactly what VCs want?
Re: OpenAI O3-Mini
#803o3-mini is the first one to get it right for me. Transcript:
Me:
I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the wolf together, the goat will eat the wolf. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?
o3-mini
1. Take the wolf across to the right bank.
2. Return alone to the left bank.
3. Take the goat across to the right bank.
4. Bring the wolf back to the left bank.
5. Take the cabbage across to the right bank.
6. Return alone to the left bank.
7. Finally, take the wolf across to the right bank.
Me:
I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the cabbage together, the cabbage will eat the goat. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?
o3-mini
1. Take the cabbage to the right bank.
2. Go back empty to the left bank.
3. Take the wolf to the right bank.
4. Bring the cabbage back to the left bank.
5. Take the goat to the right bank.
6. Go back empty to the left bank.
7. Finally, take the cabbage to the right bank.Re: OpenAI O3-Mini
#804I have to admit I'm kind of surprised by the SWE-bench results. At the highest level of performance o3-mini's CodeForces score is, well, high. I've honestly never really sat down to understand how elo works, all I know is that it scored better than o1, which allegedly as better than ~90% of all competitors on CodeForces. So, you know, o3-mini is pretty good at CodeForces. But it's SWE-bench scores aren't meaningfully…
Real software has a lot more complexity and constraints, as well as ambiguity. Claude nowhere scores so high on codeforces, but crushes o1 on webarena: https://web.lmarena.ai/leaderboard
I also ran a hold out test myself for o3 mini - asked it to implement a function I need for Python 2.5. Claude and O1 get it - o3 mini hard incorrectly believes some functions are available in this certain that aren't. If I correct it, it's revised solution is very hacky (technically works, but I would take Claude's solution over it)
Re: OpenAI O3-Mini
#805Earlier quoted context omitted.
Am I the only one who thinks that R1 is awful at creative writing? I've seen a lot of very credulous posts on twitter that are super excited about excerpts written by DeepSeek that I think are absolutely absymal. Am I alone in this? Maybe people have very different tastes than I do? (I have no formal training in creative writing, though I do read a lot of literature. Not claiming my tastes are superior - genuinely cu…
I mean, do you think this is awful ? https://pastebin.com/Ja14mt6L
I suspect you can update the prompt to make the setting more consistent, but it will still throw in a lot of inappropriate detail. I’m only nitpicking because my initial reaction was that it’s very vivid but feels difficult to understand and I wanted to explain why.
Re: OpenAI O3-Mini
#806O3-mini solved this prompt. DeepSeek R1 had a mental breakdown. The prompt: “Bob is facing forward. To his left is Ann, to his right is Cathy. Ann and Cathy are facing backwards. Who is on Ann’s left?”
Re: OpenAI O3-Mini
#807Earlier quoted context omitted.
How do I avoid the angst about this stuff as a student in computer science? I love this field but frankly I've been at a loss since the rapid development of these models.
LLMs are the new compilers. As a student, you should continue to focus on fundamentals, but also adapt LLMs into your workflow where you can. Skip writing the assembly (now curly braces and semicolons), and focus on what the software you’re building actually does, who it serves, and how it works. Programming is both changing a lot, and not at all. The mechanics may look different, but the purpose is still the same: e…
Re: OpenAI O3-Mini
#808Earlier quoted context omitted.
You need to read the article first to know that. But most people won't.
If the comments say it's worth it I read it, but often (especially on HN) the top comment starts with a summary of errors/inconsistencies in the article and then I don't really feel the need.
Re: OpenAI O3-Mini
#809For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat. o3-mini is the first one to get it right for me. Transcript: Me: I have a wolf, a goat, and a cabbage and a boat. I want to get th…
Re: OpenAI O3-Mini
#810For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat. o3-mini is the first one to get it right for me. Transcript: Me: I have a wolf, a goat, and a cabbage and a boat. I want to get th…
1. The only option for who to take first is the goat.
2. We come back and get the cabbage.
3. We drop off the cabbage and take the goat back
4. We leave the goat and take the wolf to the cabbage
5. We go get the goat and we have all of them
Neither of the transcripts do that. In the first one the goat immediately eats the cabbage. In the second one the wolf eats the goat after you leave them.