Live data from Hacker News

ChatGPT-4o vs. Math

sabrina.dev

51–60 of 182 posts

Re: ChatGPT-4o vs. Math

#51
This problem strikes me as relatively simple. What about more complex math problems? Are there good benchmarks for that?

I would dearly love to have an AI tool that I could trust to help with math. What is the state of the art? My math skills are very rusty (the last math class I took was calculus almost 40 years ago), and I find myself wanting to do things which would require a PhD level understanding of computer aided geometric design. If I had the magical AI which really understood a ton of math and/or could be fed the appropriate research papers and could help me, that would be amazing. So far all my attempts with ChatGPT 4 and 4o have been confusing because I don't really trust or fully understand the results.

This simple example and the frequency of wrong answers drives home the fact that I shouldn't trust ChatGPT for math help.

Re: ChatGPT-4o vs. Math

#52
post #51

This problem strikes me as relatively simple. What about more complex math problems? Are there good benchmarks for that? I would dearly love to have an AI tool that I could trust to help with math. What is the state of the art? My math skills are very rusty (the last math class I took was calculus almost 40 years ago), and I find myself wanting to do things which would require a PhD level understanding of computer ai…

Have you tried actual math tools like wolfram alpha?

My response to attempts to use llms to do math is usually, use the right tool for the job.

Re: ChatGPT-4o vs. Math

#53

Earlier quoted context omitted.

(Not an AI researcher, just someone who likes complexity analysis.) Discrete reasoning is NP-Complete. You can get very close with the stats-based approaches of LLMs and whatnot, but your minima/maxima may always turn out to be local rather than global.

maybe theorem proving could help? ask gpt4o to produce a proof in coq and see if it checks out...or split it into multiple agents -- one produces the proof of the closed formula for the tape roll thickness, and another one verifies it

Sure, but those are heuristics and feedback loops. They are not guaranteed to give you a solution. An LLM can never be a SAT solver unless it's an LLM with a SAT solver bolted on.

Re: ChatGPT-4o vs. Math

#54
post #44

Earlier quoted context omitted.

Does anyone know how far off we are having logical AI? Math seems like low hanging fruit in that regard. But logic as it's used in philosophy feels like it might be a whole different and more difficult beast to tackle. I wonder if LLM's will just get better to the point of being indistinguishable from logic rather than actually achieving logical reasoning. Then again, I keep finding myself wondering if humans actuall…

> Does anyone know how far off we are having logical AI? Your comment made me think of something. How do we know that logic AI is relevant? I mean, how do we know that humans are logic-AI driven and not statistical-intelligent?

Humans are really good pattern matchers. We can formalize a problem into a mathematical space, and we have developed lots of tools to help us explore the math space. But we are not good at methodically and reliably exploring a problem-space that requires NP-complete solutions.

Re: ChatGPT-4o vs. Math

#55
post #47

I'm fucking sorry but if you gave me that tape math problem i would have given the same answer! I'm so sick of people writing trick questions for AI's and then being like SEEEEEE it failed! And its like no you gave it data and a question and asked it to solve the question, it gave you the best answer it had... Like wtf. And i'm pretty sure the average person when asked would say the same thing and be like "duh" even…

It actually did really well 3/3 tries correct when given the text prompt and a simple chain of thought appended to the end of the prompt. What's interesting is that combining it with another mode (image) caused confusion, or rather introduced another source of potential errors.

Re: ChatGPT-4o vs. Math

#56
post #43

Earlier quoted context omitted.

Does anyone know how far off we are having logical AI? Math seems like low hanging fruit in that regard. But logic as it's used in philosophy feels like it might be a whole different and more difficult beast to tackle. I wonder if LLM's will just get better to the point of being indistinguishable from logic rather than actually achieving logical reasoning. Then again, I keep finding myself wondering if humans actuall…

I think LLMs will need to do what humans do: invent symbolic representations of systems and then "reason" by manipulating those systems according to rules. Here's a paper working along those lines: https://arxiv.org/abs/2402.03620

Is this what humans do?

Re: ChatGPT-4o vs. Math

#57
post #51

This problem strikes me as relatively simple. What about more complex math problems? Are there good benchmarks for that? I would dearly love to have an AI tool that I could trust to help with math. What is the state of the art? My math skills are very rusty (the last math class I took was calculus almost 40 years ago), and I find myself wanting to do things which would require a PhD level understanding of computer ai…

Have you tried actual math tools like wolfram alpha? My response to attempts to use llms to do math is usually, use the right tool for the job.

"a PhD level understanding of computer aided geometric design"

could you elaborate?

I found chatgpt to pretty useful in high level understanding of where the topics stand in the grand scheme of things

kinda like a pre-google -- it helps me navigate what to google

let's ask gpt4o

  You're an expert math researcher.

  - Give me an overview of topos theory
  - What are the main unsolved problems
  - What is the point of it?
  - What research directions are most promising?
  - What are the math prerequisites to understanding it?
output

  Overview of Topos Theory

  Topos theory is a branch of mathematics that generalizes set theory and category theory, providing a framework for analyzing and understanding mathematical structures. A topos can be seen as a generalized space that behaves much like the category of sets but with more flexible and abstract properties. Topos theory was developed by Alexander Grothendieck and his collaborators in the 1960s, initially to solve problems in algebraic geometry.

  Key concepts in topos theory include:

  Categories: Collections of objects and morphisms (arrows) between them.
  Sheaves: Structures that systematically organize data over a topological space.
  Topoi (plural of topos): Categories that satisfy certain axioms making them similar to the category of sets.

  Main Unsolved Problems
  Some of the main unsolved problems in topos theory involve:

  Classifying all possible topoi: Understanding the full landscape of possible topoi and their interrelationships remains a complex challenge.
  Connections to other areas of mathematics: Fully elucidating the connections between topos theory and other fields such as logic, algebraic geometry, and mathematical physics.
  Homotopy theory in the context of topoi: Developing a deeper understanding of homotopy theory within the framework of higher topos theory.

  The Point of Topos Theory
  The point of topos theory is to provide a unifying and flexible framework for various mathematical concepts and structures. It offers a generalized setting in which both geometric and logical ideas can be expressed and related. This has significant implications for:

  Algebraic geometry: Offering new tools and perspectives.
  Mathematical logic: Providing a categorical framework for logic and set theory.
  Theoretical computer science: Influencing the development of type theory and the semantics of programming languages.
  
  Promising Research Directions
  Some of the most promising research directions in topos theory include:

  Higher Topos Theory: Extending the classical theory to higher categories and understanding its implications in algebraic geometry and homotopy theory.
  Connections with Logic: Exploring further connections between topos theory and logic, particularly in relation to intuitionistic logic and type theory.
  Topos-theoretic Approaches to Physics: Investigating how topos theory can be applied to quantum mechanics and other areas of theoretical physics.
  Descent Theory and Stacks: Further developing the applications of topos theory in descent theory and the study of stacks in algebraic geometry.
  
  Math Prerequisites to Understanding Topos Theory
  To understand topos theory, one should have a solid background in the following areas:

  Category Theory: Fundamental concepts such as categories, functors, natural transformations, limits, and colimits.
  Set Theory: Basic set-theoretic concepts and operations.
  Algebraic Topology: Basic knowledge of topological spaces, sheaves, and cohomology.
  Algebra: Basic group theory, ring theory, and module theory.
  Logic: Understanding of basic logical systems, particularly intuitionistic logic.
  With these prerequisites, one can start delving into the more advanced and abstract concepts of topos theory.
not perfect but good enough to get started

Re: ChatGPT-4o vs. Math

#59
I actually have a contrarian view: being able to do elementary math is not that important in the current stage. Yes, understanding elementary math is a cornerstone for an AI to become more intelligent, but also let's be honest: LLMs are far from being AGIs and does not have common sense nor general ability to deduce or induct. If we accept such limitation of LLM, then focusing the mathematical understanding of an LLM appears to be incredibly boring.

Re: ChatGPT-4o vs. Math

#60
post #44

Earlier quoted context omitted.

Does anyone know how far off we are having logical AI? Math seems like low hanging fruit in that regard. But logic as it's used in philosophy feels like it might be a whole different and more difficult beast to tackle. I wonder if LLM's will just get better to the point of being indistinguishable from logic rather than actually achieving logical reasoning. Then again, I keep finding myself wondering if humans actuall…

> Does anyone know how far off we are having logical AI? Your comment made me think of something. How do we know that logic AI is relevant? I mean, how do we know that humans are logic-AI driven and not statistical-intelligent?

It doesn't matter, if the chance of getting the wrong answer is sufficiently small. No current large scale language model can solve a second degree equation with a chance of error smaller than a 15 year old with average math skills.
Post reply on HN