Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Terence Tao on O1

mathstodon.xyz

Post reply on HN
Print view View original

311–320 of 527 posts

Re: Terence Tao on O1

#311

Post by lifeformed »

skydhash wrote: post #68
WhatIsDukkha wrote: post #26

Earlier quoted context omitted.

I entirely agree about their utility. HN, and the internet in general, have become just an ocean of reactionary sandbagging and blather about how "useless" LLMs are. Meanwhile, in the real world, I've found that I haven't written a line of code in weeks. Just paragraphs of text that specify what I want and then guidance through and around pitfalls in a simple iterative loop of useful working code. It's entirely a lea…

> Much Much more productive world by just knuckling down and learning how to do the work. The fact everyone that say they've become more productive with LLMs won't say how exactly. I can talk about how VIM have make it more enjoyable to edit code (keybinding and motions), how Emacs is a good environment around text tooling (lisp machine), how I use technical books to further my learning (so many great books out here)…

I think people who are successfully using it to write code are just chaining APIs together to make the same web apps you see everywhere.

Re: Terence Tao on O1

#312

Post by wokwokwok »

Kiro wrote: post #290
wokwokwok wrote: post #256

Earlier quoted context omitted.

> I've found that I haven't written a line of code in weeks Please post a video of your workflow. It’s incredibly valuable for people to see this in action, otherwise they, quite legitimately, will simply think this is not true.

Who cares what they think? In fact, the fewer who uses this the better for the ones that do. It's not in my self-interest to convert anyone and I obviously don't need to convince myself when I have the result right in front of me. Whether you believe it or not does not make me less productive.

The obvious answer is you’ll get called a liar and shrill.

I’m not saying you are; I think there are a lot of legitimate AI workflows people use.

…but, there are a lot of people trying to sell AI, and that makes them say things about it which are just flat out false.

/shrug

But you know; freedom of speech; you can say whatever you want if you don’t care what people think of you.

My take on it is showing people things (videos, blogs, repos, workbooks like Terence posted) moves the conversation from “I don’t believe you” to “let’s talk about the actual content”. Wow, what an interesting workflow, maybe I’ll try that…

If you don’t want to talk to people or have a discussion that extends beyond meaningless trivia like “does AI actually have any value” (obviously flame bait opinions only comment threads)… why are you even here?

If you don’t care, then fine. Maybe someone else will and they’ll post an interesting video.

Isn’t that the point of reading HN threads? What do you win by telling people not to post examples of their workflow?

It’s incredibly selfish.

Re: Terence Tao on O1

#313

Post by perihelions »

SOTGO wrote: post #88
perihelions wrote: post #30

Earlier quoted context omitted.

Because I'm verifying everything by hand, as is the whole point of studying pure mathematics.

How can you verify a proof though? Pure math isn't really about computations, and it can be very hard to spot subtle errors in a proof that an LLM might introduce, especially since they seem better at sounding convincing rather than being right.

The same way I verify my own proofs of textbook exercises: very cautiously. Subtle errors are a feature of the problem domain, not a new novelty.

Re: Terence Tao on O1

#314

Post by [deleted] »

CJefferson wrote: post #307
wenc wrote: post #5

Once GPT is tuned more heavily on Lean (proof assistant) -- the way it is on Python -- I expect its usefulness for research level math to increase. I work in a field related to operations research (OR), and ChatGPT 4o has ingested enough of the OR literature that it's able to spit out very useful Mixed Integer Programming (MIP) formulations for many "problem shapes". For instance, I can give it a logic problem like "…

I'm currently teaching a course on MIP, and out of interest I tried asking 4o about some questions I ask students. It could give the 'basic building blocks' (How to do x!=y, how to do a knapsack), but as soon as I asked it a vaguely interesting question that wasn't "bookwork", I don't think any of it's models were right. I'm interested on how you seem to be getting better answers than me (or, maybe I just discard the…

[deleted]

Re: Terence Tao on O1

#315

Post by atomic128 »

idiotsecant wrote: post #86
calmworm wrote: post #73

Earlier quoted context omitted.

I would take that bet.

Me too.

Somebody tested people on Hacker News to evaluate programming competency.

This was part of a larger evaluation comparing the Hacker News population to people on Reddit programming subreddits.

Here is a very heated discussion of the result:

https://news.ycombinator.com/item?id=33293522

It appears that Hacker News is perhaps NOT populated by the programming elite. In contrast, there are real wizards on Reddit.

Surprising, I know.

Re: Terence Tao on O1

#316

Post by teaearlgraycold »

asdasjhG wrote: post #202
teaearlgraycold wrote: post #188

Earlier quoted context omitted.

Or can everyone now lead research projects and build businesses?

No, almost everyone who gets funding for a business already belongs to the monied royalty and gets it either directly from his family, via friends of the family or laundered through a VC. There are exceptions of course, but that's how the bulk of businesses, especially those with stupid ideas are funded. In the latter category success does not even matter, the trust fund baby just has to have the appearance of a lead…

There is truth to this, but you’re overstating it. If AI is cheap and can replace grunt workers then we’ll have a massive wave of new businesses solving problems that previously required a massive capital investment.

Re: Terence Tao on O1

#317

Post by p-e-w »

rvnx wrote: post #147
cjbgkagh wrote: post #114

Earlier quoted context omitted.

In my view these models produce above average code which is good enough for most jobs. But the hacker news sampling could be biased towards the top tier of coders - so their personal account of it not being good enough can also be true. For me the quality isn't anywhere close to good enough for my purposes, all of my easy code is already done so I'm only left working on gnarly niche stuff which the LLMs are not yet h…

It is also a problem of ego. It is difficult if you have been told all your life that you are the best, to accept the fact that a computer or even other people might be better than you. It requires lot of self-reflection. Real top-tiers programmers actually don’t feel threatened by LLMs. For them it is just one more tool in the toolbox like syntax highlighting or code completion. They choose to use these tools based…

> Real top-tiers programmers actually don’t feel threatened by LLMs.

They should, because LLMs are coming for them also, just maybe 2-3 years later than for programmers that aren't "real top-tier".

The idea that human intellect is something especially difficult to replicate is just delusional. There is no reason to assume so, considering that we have gone from hole card programming to LLMs competing with humans in a single human lifetime.

I still remember when elite chessplayers were boasting "sure, chess computers may beat amateurs, but they will never beat a human grandmaster". That was just a few short years before the Deep Blue match.

The difference is that nobody will pay programmers to keep programming once LLMs outperform them. Programmers will simply become as obsolete as horse-drawn carriages, essentially overnight.

Re: Terence Tao on O1

#318

Post by l33t7332273 »

wenc wrote: post #217
l33t7332273 wrote: post #182

Earlier quoted context omitted.

I an also working in OR and I have had the complete opposite experience with respect to MILP optimization(and the research actually agrees; there was a big survey paper published earlier this year showing LLMs were mostly correct on textbook problems but got more and more useless as complexity and novelty increased.) The results are boiler plate at best, but misleading and insidious at worst, especially when you get…

Can you point me to that paper? What version of the model were they using? Have you tried again with the latest LLMs? ChatGPT4 actually (correctly) explains what each constraint does in English -- it doesn't just provide the constraint when you ask it for the formulation. Also, not sure if CPLEX should be involved at all -- I usually just ask it for mathematical formulations, not CPLEX calling code (I don't use CPLEX…

I was referring to section 4 of A Survey for Solving Mixed Integer Programming via Machine Learning(2024): https://arxiv.org/pdf/2401.03244.

I’ve heard (but not so much observed) that there is substantial difference between recent models, so it’s possible that they are better than when this was written.

Anyways, CPLEX has an associated modeling language that features syntactic sugar which has the effect of providing opaqueness to the underlying MILP that it solves. I find LLMs essentially unable to even make an attempt at determining the MILP from that language.

PS: How is Xpress? Is there some reason to prefer it to Gurobi or Mosek?

Re: Terence Tao on O1

#319

Post by wenc »

CJefferson wrote: post #307
wenc wrote: post #5

Once GPT is tuned more heavily on Lean (proof assistant) -- the way it is on Python -- I expect its usefulness for research level math to increase. I work in a field related to operations research (OR), and ChatGPT 4o has ingested enough of the OR literature that it's able to spit out very useful Mixed Integer Programming (MIP) formulations for many "problem shapes". For instance, I can give it a logic problem like "…

I'm currently teaching a course on MIP, and out of interest I tried asking 4o about some questions I ask students. It could give the 'basic building blocks' (How to do x!=y, how to do a knapsack), but as soon as I asked it a vaguely interesting question that wasn't "bookwork", I don't think any of it's models were right. I'm interested on how you seem to be getting better answers than me (or, maybe I just discard the…

I had to prompt it correctly (tell it to exclude x=y case in the x≠y formulation), but ChatGPT seems to have arrived at the correct answer:

https://chatgpt.com/share/66e652e1-8e2c-800c-abaa-92e29e0550...

Re: Terence Tao on O1

#320

Post by afro88 »

The o1 model is hit and miss for me. On one hand it has solved the NYT Connections game [0] each day I've tried it [1]. Other models, including Claude Sonnet 3.5 cannot.

But on the other hand it misses important detail and hallucinates, just like GPT-4o. And can need a lot of hand holding and correction to get to the right answer, so much so that sometimes you wonder if it would have been easier to just do it yourself. Only this time it's worse because you're waiting 20-60 seconds for an answer.

I wonder if what it excels at is just the stuff that I don't need it for. I'm not in classic STEM, I'm in software engineering, and o1 isn't so much better that it justifies the wait time (yet).

One area I haven't explored is using it to plan implementation or architectural changes. I feel like it might be better for this, but need the right problems to throw at it.

[0] https://www.nytimes.com/games/connections

[1] https://chatgpt.com/share/66e40d64-6f70-8004-9fe5-83dd3653a5...

Post reply on HN