o1-pro got turned off? and o3-mini-high is significantly worse than o1-pro :(
OpenAI O3-Mini
821–830 of 944 posts
Re: OpenAI O3-Mini
#822I just had it convert Swift code to Kotlin and was surprised at how the comment was translated. It "knew" the author of the paper and what is was doing!? That is wild. Swift: // // Double Reflection Algorithm from Table I (page 7) // in Section 4 of https://tinyurl.com/yft2674p // for i in 1 .. Kotlin: // Use the Double Reflection Algorithm (from Wang et al.) to compute subsequent frames. for (i in 1 until N) { val X…
Someone else has written this exact code on the internet, OpenAI stole it, and now chatgpt is regurgitating it. Just like it can regurgitate whole articles. You need to stop being wow'd by human intelligence masquerading as AI!
you are so confused it’s not even funny
Re: OpenAI O3-Mini
#823Earlier quoted context omitted.
Good catch. That's likely an artifact of the way I flatten the nested JSON from the comments API. I originally did that to save on tokens but modern models have much larger input windows so I may not need to do that any more.
This is why I keep advocating that JSON should die, or at least no longer be used with LLMs. LLMs (and human brains) are simply not wired up for counting nested curly brackets across long spans of quoted text joined by colons and commas, and it is far too easy for humans to make mistakes when chunking JSON. IMO, (Strict)YAML is a very good alternative, it has even been suggested to me by multiple LLMs when I asked th…
Would you elaborate on why counting braces is different from counting spaces to determine hierarchy? Or is it more about the repetition of higher levels keys in chunks (which could be done in JSON)?
Re: OpenAI O3-Mini
#824For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat. o3-mini is the first one to get it right for me. Transcript: Me: I have a wolf, a goat, and a cabbage and a boat. I want to get th…
Thank you, this is a perfect argument why LLMs are not AI but just statistical models. The original is so overrepresented in the training data that even though they notice this riddle is different, they regress to the statistically more likely solution over the course of generating the response. For example, I tried the first one with Claude and in its 4th step, it said:
> This is safe because the wolf won't eat the cabbage if they're together on the far side.
even though it clearly states the opposite in the question.
It's impressive that just dumb stats can be used to produce something that is very often useful, can help write code and when making it generate intermediate steps, it can often generate a chain of text that happens to be often right. However, it's not actual reasoning, there is no model of the world, no information storage and retrieval, and so on - just statistics between tokens.
Re: OpenAI O3-Mini
#825I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae... For 18,936 input, 2,905 output it cost 3.3612 cents. Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...
Currently on the internet people skip the article and go straight to the comments. Soon people will skip the comments and go striaght to an AI summary reading neither the original article nor the comments.
Have you ever had to give a demo that no meeting attendee actually cared about, just because management demanded it? Standup meetings with 20 people where maybe 2 people cared about? The future might involve AI updates, summarized by AI into weekly reports, summarized again into monthly reports, then into quarterly departmental reports that nobody actually reads.
Re: OpenAI O3-Mini
#826Earlier quoted context omitted.
Currently on the internet people skip the article and go straight to the comments. Soon people will skip the comments and go striaght to an AI summary reading neither the original article nor the comments.
But then there will be no comments to summarize.
You will be given 3 A.I. generated comments to choose as your comment
Re: OpenAI O3-Mini
#827For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat. o3-mini is the first one to get it right for me. Transcript: Me: I have a wolf, a goat, and a cabbage and a boat. I want to get th…
If you’ve talked to OpenAI about this riddle before it’s certain that conversation is now in the training set.
At first I was curious if OP tried arguing with it or telling it the correct answer. If he just asked the question without any further interaction, it would still have no data that it generated the wrong answer.
But as another comment says, many people have tried similar things and posted about it online, that information is siphoned by now and the LLMs weights get updated.
Re: OpenAI O3-Mini
#828Earlier quoted context omitted.
I mean, do you think this is awful ? https://pastebin.com/Ja14mt6L
This looks like my experiments to get R1 to write fiction and I think it’s worse than what you get from openai. For instance, it’s using very colorful language to describe a place that’s both a remote fishing village on the edge of a cliff hours before dawn, and a bustling wharf with chattering laborers and large ships anchored in the distance. It also starts by saying the protagonist wakes up with his mouth tasting…
I agree that it felt hard to read. It also doesn't make sense that they're fishing in a storm. But from a prose perspective I don't think it's cringe, which is an improvement from my expectation. I'd share some of the writings I think are terrible but I don't like to pick on people.
Re: OpenAI O3-Mini
#829For years I've been asking all the models this mixed up version of the classic riddle and they 99% of the time get it wrong and insist on taking the goat across first. Even the other reasoning models would reason about how it was wrong, figure out the answer, and then still conclude goat. o3-mini is the first one to get it right for me. Transcript: Me: I have a wolf, a goat, and a cabbage and a boat. I want to get th…
I tested some cases in Misguided Attention[0]: while many cases now pass, others fail all the same. Given the amount of contamination and the difficulty of finding sufficiently original problems of this nature, I defer to a 20:80 ratio of genuine improvement to recall.
Re: OpenAI O3-Mini
#830Earlier quoted context omitted.
Currently on the internet people skip the article and go straight to the comments. Soon people will skip the comments and go striaght to an AI summary reading neither the original article nor the comments.
Have you ever had to give a demo that no meeting attendee actually cared about, just because management demanded it? Standup meetings with 20 people where maybe 2 people cared about? The future might involve AI updates, summarized by AI into weekly reports, summarized again into monthly reports, then into quarterly departmental reports that nobody actually reads.
Or, the future might be everyone reads summaries, because there are only AI Managers and no human managers, where humans hold occasional meetings and have conversations which are listed to by AI, and we take our lead from the AI summaries. The AI maintains business focus through it monitoring the business performance, updating each summary as needed to maintain performance.
It's a worthwhile experiment for a business school, IMO, automating a layer of bureaucracy.