Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

chatgpt.com

Post reply on HN
Print view View original

641–650 of 681 posts

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#641

Post by lanstin »

theplumber wrote: post #634
lanstin wrote: post #602

Earlier quoted context omitted.

That seems only partially true. I have no doubt it could crank out a UI like Facebook, but having it work for high volume is less well represented in the training data. Things that seems obvious to us, like sharding, using asynchronous messaging systems to provide more durability at the cost of instant consistency, traffic shedding, intelligent clients, conscious decisions on CAP properties in the light of user exper…

I call it BS. When you are hitting real scalability issues that the AI can no longer help you are already a successful business. The non technical founder will not know about async message passing but will ask the AI why the site is loading so slowly and sure enough the AI is able to implement caching and various mechanism to improve the performance and observability. You are hitting a wall when you do novel stuff (I…

I thought the model was the one person company with enough usage to be worth 1 billion USD. Badly designed systems can hit bottle necks at like a few thousand DB calls per second which a well designed system can handle on one container. Getting those new features adding to table width or SQL length (or document nesting if you are going that route) with each new feature, I feel the company with a few experts will be at an advantage. Like the old beating the averages essay on using lisp. More smart flexibility wins in the marketplace.

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#642

Post by eitally »

raz32dust wrote: post #424
lukebuehler wrote: post #8

It’s endlessly fascinating to read the AI transcript of an expert who _really_ knows how to cut to the chase. It just shows how much you can potentially squeeze out of these models. I’m also surprised to see that even Terrence Tao seems to use it in a way that resembles, in progression, how I use llms in my area of expertise (emphasis on progression and usage patterns, not absolute skill, obv I don’t match that): sho…

The top comment has a counter example though https://x.com/DmitryRybin1/status/2079904005652893709 I think it's not even about the ability to steer the AI. Just the ability to ask the right questions

I am not impressed with that steering of the AI. In the case of Terry Tao, he clearly held a solid understanding of the problem domain and could also understand the outputs of the LLM at an expert level. He asked astute, topical questions that specifically probed areas of the problem that were either unclear or insufficiently specified, and ultimately ended up at a place where (presumably) he both understood the Jacobian Conjecture Counterexample exhaustively, and how it was derived. My impression is that through his learnings during the chat, he became sufficiently well-educated in this subdomain to apply some of these lessons to other mathematical pursuits.

In your counterexample, the human just repeatedly asked the model to "try harder". Not remotely the same.

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#643

Post by theplumber »

bdcravens wrote: post #594
theplumber wrote: post #464

Earlier quoted context omitted.

Indeed that’s the case but most of the programming problems are solved problems and unless you care how things are done someone with less/no knowledge can still make great use of programming to do what it needs to do. Let’s not pretend you need to be programmer to use AI for programming like mathematicians need to solve math problems. Programming (most of the time)solves real problems rather than abstract constructs.…

True, but I think it's the difference between doing it for a single $200 subscription vs spending $200 a day in tokens.

I think 5 x max subscriptions would suffice.

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#644

Post by davesque »

blargey wrote: post #215
davesque wrote: post #195

Earlier quoted context omitted.

Without any more context, "keep going" seems to be doing a lot of work. The user is placing a lot of faith in the LLM to not make subtle logic mistakes and to take good approaches to each problem. In my experience, even frontier models (such as Fable) are quite capable of getting confused during even simple technical work I've done in the dev ops world. For example: LLM: This package hasn't made it to production. ME:…

There may be something(s) about mathematics (proofs) that makes it particularly amenable to LLM reasoning - highly inductive from facts that are explicitly within-context/associative space? Being an unusually well documented discipline in general, with less influence from tacit knowledge or idiosyncratic “it works however the opinionated human made it work +- bugs” processes? Something about simulating even the small…

I disagree. In fact, I think the field of dev ops gives a clean analogy with mathematical proofs. My point was that my work often requires that I figure out a consistent way to prove to myself what the condition of a system is by asking the right probing questions about it. What I've seen is that even the best LLMs lack a good intuition about what questions they should be asking and instead reach for the quickest and most obvious checks that leave edge cases uncovered. Maybe it is something about the domain of the problem; I don't know. But it makes it hard for me to imagine that an LLM wouldn't make similar errors in other cases, especially when generating mathematical proofs that will soon be too dense for humans to review.

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#645

Post by computably »

abstractbill wrote: post #71
computably wrote: post #61

Earlier quoted context omitted.

A term that gets tossed around in math is "mathematical maturity." It's similar to what you see in other fields - e.g. learning how to program, learning how to make music, learning how to cook - that involves many "aha" moments and reshapes your perspective. Math is full of such steps, moreso than most other endeavors, probably because the main limit is the abstract reasoning itself.

Math is full of such steps, moreso than most other endeavors, probably because the main limit is the abstract reasoning itself. That, as well as how long we've been doing it (thousands of years!) and so how much of the more accessible parts we've explored very thoroughly.

Of the examples I listed, only programming is a modern invention. Humans have certainly been making music and cooking for thousands of years.

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#647

Post by chorizo »

Kon5ole wrote: post #550
larsfaye wrote: post #436

Earlier quoted context omitted.

Correctamundo. That's EXACTLY what I just wrote about. And how we'll create the next generation of people who know how to evaluate responses when we're creating a generation of people who are increasingly reliant on LLMs to do the work makes for a strange paradox. https://larsfaye.com/articles/ai-coding-will-prevent-experti...

Good writeup, and certainly food for thought. I wonder if this key point actually holds though: >The skills to do so, however, are a function of someone who has experienced the friction and challenges over time that culminate in "good taste". It's certainly sometimes true, but I don't think it's a general rule. Sometimes friction is just friction and sometimes you spend 1000 hours learning something that disappears a…

> Everything you know about 6800 assembly will not make your note-taking app any better.

For most apps, it makes sense to profile your app for expected use cases. Put even minimal thought into making the hot codepaths faster by moving unnecessary operations out of it.

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#648

Post by tstrimple »

Geof25 wrote: post #440
bredren wrote: post #209

Earlier quoted context omitted.

It is ~a meme on subreddits that developers struggling to get good results out of any given model is a "skills issue." But I think your comment drives at some authentic take on this. Skill with AI is not only crafting iterative prompts the agent will understand, but also very high domain-specific knowledge of what the prompts explore. One without the other can result in frustration or worse.

It is a skill issue. Such developers have to take a problem, isolate it and explain it to someone/something else with enough context to be able to work with it. It is perplexing how many developers lack this basic skill, some of them borderline lack theory of mind and are incapable to understand that other people can't see the unspoken part in their heads.

I wonder how much this overlaps with the seeming inability of so many developers to ask good questions. They want to jump right into screen sharing and talking about the issue. They cannot sit down and summarize the problem accurately and describe the things they have tried already. It's always "let's hop on a call real quick and walk through it". Just tell me what the damn problem is!

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#649

Post by danielezell »

smsm42 wrote: post #627
Kon5ole wrote: post #550

Earlier quoted context omitted.

Good writeup, and certainly food for thought. I wonder if this key point actually holds though: >The skills to do so, however, are a function of someone who has experienced the friction and challenges over time that culminate in "good taste". It's certainly sometimes true, but I don't think it's a general rule. Sometimes friction is just friction and sometimes you spend 1000 hours learning something that disappears a…

> Everything you know about 6800 assembly will not make your note-taking app any better. It could. It could make a simple note-taking app not take gigabytes of memory and take visible delay on each click. Most people don't bother of course because simple note-taking app is not worth the effort. It's possible to do better, it's just often not practical.

Question from the peanut gallery here. Why not just train an LLM to write in Assembly? Or bare metal? Validate by compiling. Or binary, no compilation needed. Can't a frontier LLM find the bugs better than we? As long as the program is built from small, verifiable subunits, the pragmatic problem of creating an efficient program is in every careful planner's reach. Seems the friction is nearly gone. I have no skin in the game here, but this is a developing (no pun intended) expectation for me.

Re: Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

#650

Post by com2kid »

jameshart wrote: post #412
com2kid wrote: post #389

Earlier quoted context omitted.

A byte is quite real. On old machines you could open them up and literally point to where bytes were physically stored in memory. Now days of course the chips are small so you have to point to where the multiple gigabyte chips are at. But they are still quite physically. Heck a C pointer points to an actual physical location on your machine, if you ignore the MMU.

Wow and TCP sends a stream of those microscopic memory cells?

Over a wire it sends pulses of electricity which you can see with an oscilloscope.

Heck there is a standard for tcp/ip over short wave if you want to hear the bytes being transmitted. More widespread, those of us familiar with dial up modems are also aware that network traffic can be carried as actual sound.

Or fiber optics where network traffic is flashes of light.

Or IR transmissions, use your phone camera and you can see data being sent over the air. Not tcp/up but physical blinking lights sending digital data.

Post reply on HN