Earlier quoted context omitted.
Not gonna lie, when I see that someone is banned on X, I assume credibility
Even the 100's of Hamas-affiliated accounts? https://ny1.com/nyc/all-boroughs/technology/2023/10/12/x-say...
frEeDom oF sPeeCh
ChatGPT-4o vs. Math
41–50 of 182 posts
Re: ChatGPT-4o vs. Math
#42This recent article on Hacker News seems to suggest similar inconsistencies. GPT-4 Turbo with Vision is a step backward for coding (aider.chat) https://news.ycombinator.com/item?id=39985596 Without looking deeply at how cross-attention works, I imagine the instruction tuning of the multimodal models to be challenging. Maybe the magic is in synthetically creating this instruct dataset that combines images and text in…
The same guy found 4o to be much better
GPT-4o takes #1 and #2 on the Aider LLM leaderboards https://news.ycombinator.com/item?id=40349655
Subjectively, I've found Aider to be much more useful on 4o. It still makes mistakes applying changes to files occasionally, but not so much to make me give up on it.
Re: ChatGPT-4o vs. Math
#43Earlier quoted context omitted.
This speaks to a deeper issue that LLMs don’t just have statistically-based knowledge, they also have statistically-based reasoning. This means their reasoning process isn’t necessarily based on logic, but what is statistically most probable. As you’ve experienced, their reasoning breaks down in less-common scenarios even if it should be easy to use logic to get the answer.
Does anyone know how far off we are having logical AI? Math seems like low hanging fruit in that regard. But logic as it's used in philosophy feels like it might be a whole different and more difficult beast to tackle. I wonder if LLM's will just get better to the point of being indistinguishable from logic rather than actually achieving logical reasoning. Then again, I keep finding myself wondering if humans actuall…
I think LLMs will need to do what humans do: invent symbolic representations of systems and then "reason" by manipulating those systems according to rules.
Here's a paper working along those lines: https://arxiv.org/abs/2402.03620
Re: ChatGPT-4o vs. Math
#44Earlier quoted context omitted.
This speaks to a deeper issue that LLMs don’t just have statistically-based knowledge, they also have statistically-based reasoning. This means their reasoning process isn’t necessarily based on logic, but what is statistically most probable. As you’ve experienced, their reasoning breaks down in less-common scenarios even if it should be easy to use logic to get the answer.
Does anyone know how far off we are having logical AI? Math seems like low hanging fruit in that regard. But logic as it's used in philosophy feels like it might be a whole different and more difficult beast to tackle. I wonder if LLM's will just get better to the point of being indistinguishable from logic rather than actually achieving logical reasoning. Then again, I keep finding myself wondering if humans actuall…
> Does anyone know how far off we are having logical AI?
Your comment made me think of something. How do we know that logic AI is relevant? I mean, how do we know that humans are logic-AI driven and not statistical-intelligent?
Re: ChatGPT-4o vs. Math
#45Earlier quoted context omitted.
Does anyone know how far off we are having logical AI? Math seems like low hanging fruit in that regard. But logic as it's used in philosophy feels like it might be a whole different and more difficult beast to tackle. I wonder if LLM's will just get better to the point of being indistinguishable from logic rather than actually achieving logical reasoning. Then again, I keep finding myself wondering if humans actuall…
(Not an AI researcher, just someone who likes complexity analysis.) Discrete reasoning is NP-Complete. You can get very close with the stats-based approaches of LLMs and whatnot, but your minima/maxima may always turn out to be local rather than global.
maybe theorem proving could help? ask gpt4o to produce a proof in coq and see if it checks out...or split it into multiple agents -- one produces the proof of the closed formula for the tape roll thickness, and another one verifies it
Re: ChatGPT-4o vs. Math
#46The model's first attempt is impressive (not sure why it's labeled a choke). Unfortunately gpt4o cannot discover calculus on its own.
[dead]
Re: ChatGPT-4o vs. Math
#47I'm fucking sorry but if you gave me that tape math problem i would have given the same answer! I'm so sick of people writing trick questions for AI's and then being like SEEEEEE it failed! And its like no you gave it data and a question and asked it to solve the question, it gave you the best answer it had... Like wtf.
And i'm pretty sure the average person when asked would say the same thing and be like "duh" even though technically based on the minutia it's incorrect.
Re: ChatGPT-4o vs. Math
#48Need to run this experiment on a problem that is t already on its training set.
Not in the training set? The dataset is ALL OF THE INTERNET, i'd love you to find something it hasn't seen before.
Re: ChatGPT-4o vs. Math
#49I'm fucking sorry but if you gave me that tape math problem i would have given the same answer! I'm so sick of people writing trick questions for AI's and then being like SEEEEEE it failed! And its like no you gave it data and a question and asked it to solve the question, it gave you the best answer it had... Like wtf. And i'm pretty sure the average person when asked would say the same thing and be like "duh" even…
But AI is put into places where you wouldn't ask the average person.
It's treated like a genius and that's what it gets measured against.
Re: ChatGPT-4o vs. Math
#50Posted a screenshot from the post in, got it on the first try https://x.com/thisritchie/status/1791153959865852093
Ha nice.
I decided to try the same and it got it incorrect. It's so non-deterministic. It landed on 0.17cm. Tried it another time and it got 0.1697cm. When I asked it to check it's work, it got the right answer 0.00589cm