We had this yesterday (317 points, 133 comments)https://news.ycombinator.com/item?id=48998362
Classic Gabe-bot
Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
361–370 of 681 posts
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#362It’s endlessly fascinating to read the AI transcript of an expert who _really_ knows how to cut to the chase. It just shows how much you can potentially squeeze out of these models. I’m also surprised to see that even Terrence Tao seems to use it in a way that resembles, in progression, how I use llms in my area of expertise (emphasis on progression and usage patterns, not absolute skill, obv I don’t match that): sho…
I encountered something fairly similar working with Claude a few days ago. For a current project I've been fairly hand-wavy with requirements since I was getting good results, but it seemed to be failing hard on some key points, so I started to be more strict with it. Even after the fails were resolved, I've noticed that Claude now behaves differently within that project, carefully checking and rechecking things up f…
The most powerful way to use Fable in particular seems to be to pit it against Sol as a reviewer (or, I suppose, vice versa.) After going back and forth a half-dozen times, the result tends to be pretty much bulletproof.
What's somewhat disturbing is just how much Fable's code really does benefit from the review. It tends to leave a lot of low-hanging fruit, and you can see it getting kind of impatient when repeatedly called on it.
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#363Earlier quoted context omitted.
The years are broad though, the nice thing with AI explanations is that they can go deep quickly, and quite precisely down the path you need for your prior experience.
Oh for sure, I'm not denying the value of AI as a teacher, just saying that it's not going to speed things up much over a quality personal teacher/tutor.
Oh, but it will. Especially in math you depend on definitions and concepts defined earlier, and if you fall behind, perhaps in a class setting, it is hard to catch up, without having somebody who can explain it to you.
But, now, with world class knowledge possessing tutor, you can ask for an explanation of missed concepts, even embarrassingly stupid questions you would never ask a person.
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#364Earlier quoted context omitted.
It's a class with an array of integers in it with .length() == t - 1 and the same methods as Matrix. In lean4, even without mathlib4, TCP/IP is way more code than a Rees algebra. Math uses dense notation that is gigaoverloaded, and the disambiguating context was historically the leisure and proximity to have someone explain what the lexemes even mean. lean4 is proving to be very revealing as an uncorruptible referee…
That's false. Z[n] in rings does not mean "an array of integers of length n", it means the subring generated by Z union with {n}, where n is an element of some other set. For example: Z[ i ], the Gaussian integers, is the subring (of C) generated by Z union { i } where i is the imaginary unit in C, the complex numbers. The Gaussian integers correspond to the integer grid-points of the complex plane, if you want to vi…
Cher Monsieur Laurent, comment pourrions-nous espérer obtenir un soutien de la part de ℤ ⭢ ℤ? Les polynomials! Tout est grande!
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#365Earlier quoted context omitted.
> Basically every academic AI researcher in history was doing what you described. That is not true. Alan Turing did not view things that way, his test would say that a dog has zero intelligence. Neither did any of the MIT Lispers. And neither do Lecun or Sutskever or Sutton! They are all focused on human intelligence. None of them are even slightly concerned about an AI which is intelligent before it learns any langu…
> And neither do Lecun or Sutskever or Sutton! They are all focused on human intelligence. None of them are even slightly concerned about an AI which is intelligent before it learns any language. ??? https://www.youtube.com/watch?v=GvibIstOn_E his arguemtn here is clearly built around using some sort of sensory data to build a model of the world like humans (animals) do. also you clearly decline to mention Lecun who…
In 50 years it will seem very strange that people insisted Mythos was intelligent.
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#366Earlier quoted context omitted.
> just repeatedly saying "keep going" to ChatGPT For posterity, this indeed works for most problems where an agent might give up. LLMs don't inherently know something is impossible. The phrase I tend to use in my harder prompts to automate this with a sane loop breaker: > **REPEAT THIS PROCESS UNTIL CONVERGENCE AND YOU ARE OUT OF OPTIMIZATION IDEAS.** You have permission to keep iterating.
I have a pi extension that just runs the same prompt in a loop 25 times. I tried giving it a loop breaker but I found that it'd give up too readily. When the task is actually complete each iteration didn't do a lot of work so it was efficient enough. I suppose another way is to call out to a separate context to check if the task is complete?
Commands like /goal and similar are the more complex version of this; you write a prompt like it was a singular iteration, it runs one iteration, then runs an "evaluator" to determine if the goal has been reached, then runs the prompt again with a little extra to make it go again - and so on. The evaluator is just an LLM with most of the result or context looking at the original goal and the state and answering the question "has the goal been met".
The interesting part of this: while some leading implementations use the same LLM and context for the evaluator, some call out to a different context, some to a tuned LLM and different context; so which is better? many blog-scale benchmarks are calling it a toss-up that is highly dependent on the primary model.
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#367What was most remarkable to me from this transcript, was how strong of an equal the AI agent comes across compared to the user (Tao). And Tao is one of the top mathematicians of modern times. Yes, Tao is guiding it to where he wants to go. But also, Tao is actively learning from it and relying on its explaining, analysis, and inference abilities. You can easily imagine this conversation having taken place between Tao…
I dunno. To me, this seems like kind of a counterexample (pun intended?) to that thesis. Would a conversation with another AI have been as fruitful as this conversation with Tao? Certainly not! Will that change in a year? I dunno, but it seems like those goal posts keep moving, and I'm a bit skeptical.
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#368Is ChatGPT's interface always this atrociously jittery? Or is it just because this page is getting an HN hug-of-death right now? Every time I try to scroll the whole page goes blank for a few seconds and then re-renders.
I've found it's had unacceptable scroll performance in long contexts for a while now. In this case, it takes me 12 seconds to see content when first opening the link, and about 18 to re-render content when scrolling.
So, the Jacobian Conjecture was done, via an counterexample. Can we now put the models to work on an even more difficult problem: Scrolling 100k of text, on a 128GB 24 core processor, smoothly in a browser?
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#369Earlier quoted context omitted.
This is also true for almost every other field, even within computer science. The only difference is that a lot of people operate at a very surface level without realizing just how much background knowledge they have accumulated. Think about the number of keywords your average SWE is expected to know. It is rather insane. Cache, stack, heap, process, thread, socket, file, tcp, http, tls, websocks, socks, soc2???, dea…
> https://en.wikipedia.org/wiki/Transmission_Control_Protocol compare to > https://en.wikipedia.org/wiki/Rees_algebra Most people, especially non-tech technical people, could crash through the TCP article and come out the other side with at least a high level understanding of it. Most people, even technical ones, could not even get through the first line of the rees article, heck the first statement of the article. A…
This is perhaps more a comment on Wikipedia's coverage of mathematics. They have some general guidelines in their manual of style, but it's really hard to write math articles in a way such that something like the Rees algebra without defining 1000 thinks beforehand.
To understand the definition of the Rees algebra, you would need to define, mostly in order: sets, groups, abelian groups, rings, ideals of rings, algebras over rings, direct sums of rings, adjoining things to rings, etc.
This is just to understand the definition; to understand its significance in algebraic geometry (which I have no idea of), there are a thousand more definitions.
The issue with trying to understand a concept in math is there is a massive directed acyclic graph of prerequisites leading to these concepts, and one needs to traverse this graph in the right order. Unfortunately, knowing the right order is almost tantamount to understanding the concept itself.
Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample
#370Earlier quoted context omitted.
One thing I've repeatedly told people is that chatbots are often the most patient teachers we'll ever get (especially when explaining "stupid" questions) — compared to what we've encountered on StackOverflow or Reddit.
They lack the empathy to understand where and why you're struggling. I've given private math lessons and seen students struggle with ai, even though ai gave the right answers. My intuition is that humans spot xy problems easier when teaching (user ask x but really needs y), whereas llms will oblige writing about x.
That's interesting, thanks for sharing. Can you please share a specific example of what you describe? Does the student stop asking why after a while?