Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A misalignment of AI in mathematics

mathandai.org

Post reply on HN
Print view View original

881–890 of 1001 posts

Re: A misalignment of AI in mathematics

#881

Post by DoctorOetker »

zozbot234 wrote: post #823
freehorse wrote: post #817

Earlier quoted context omitted.

> his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo th…

The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the word 'misalignment' would imply in this context): they are merely working with frontier tools that have a very limited capability for human-like understanding. It's left to human mathema…

I wouldn't call the situation settled, nor do I expect we will ever find out what truly happened:

- a rare low likelihood sequence of coincidences

- deliberate frontier lab behavior, setting intellectual interrupts on LLM "aha"-moments, so that when some mathematician explains for the umpteenth time the approach they want to take "stop paraphrasing my approach, start the actual calculation!" and when the chatbot eventually groks it, they can scoop in, possibly with live dash-board and interrupt priority levels ranging from "ensure this sample gets into the new dataset" to "Millenium Prize Scoop Opportunity Imminent, call Sam"...

There is always an alignment problem, incentives to ideals, behavior to incentives, ...

Re: A misalignment of AI in mathematics

#882

Post by int_19h »

gdsimoes wrote: post #677
ThalesX wrote: post #243

Earlier quoted context omitted.

It boggles my mind. Why cure cancer? The cancer researchers will be out of a job and their kids are going to starve!

Why not cure cancer and take care of the researchers’ kids?

Why not indeed?

AI should be the commons.

Re: A misalignment of AI in mathematics

#883

Post by 40four »

GPerson wrote: post #709
tmhn2 wrote: post #198

As a mathematician maybe I am a little more optimistic than this declaration. I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH. Well, what is the result? In the Mochizuki case, it was a lot of skeptici…

It’s frustrating that this comment is at the top because it, along with lots of the replies it inspired, absolutely misrepresents the actual declaration. The declaration is not making any statements about not using any AI in mathematics. The entire point is to push the use of the technology in a direction which is compatible with positive pre-existing features of the math community, and to make it better known what s…

I don’t see why the two are mutually exclusive to be honest. I’m not a mathematician, but I’m a software engineer and whether I like it or not, AI has changed my field dramatically forever. I understand the distaste for the way AI might be affecting the field of mathematics, but the genie is out of the bottle, and it will never be the same. It just took a little longer for it to hit the math community. Software engineers have been debating and coming to terms with the new reality for a few years now.

Re: A misalignment of AI in mathematics

#884

Post by laughingcurve »

int_19h wrote: post #840
nmeagent wrote: post #713

Earlier quoted context omitted.

Yep. Perhaps humanity would be better off if we instituted and enforced the notion of "Thou shalt not make a machine in the likeness of a human mind." It's worth thinking about; just because we can build AI systems doesn't mean we should.

I would remind you that the fictional society that did this was also described as a feudal monarchy in which a tiny class of nobles exploits the economic output of trillions of people reduced to serfs by controlling all means of interstellar trade. Oh, and they also have legal slavery.

“B-b-b-but muh pop culture trope”

Thank you for commenting this. I'm tired of hearing about Dune like its some kind of utopia

Re: A misalignment of AI in mathematics

#885

Post by int_19h »

noisy_boy wrote: post #781

We have an intellectual structure that has been largely the same irrespective of the level of technological advancement. The brightest among us think about hard problems. Even though all of us may not understand those advances, we all appreciate and celebrate them to varying degrees. Those who are able to comprehend have time to absorb, generate more discussion. In time, we honor those achievements that withstand the…

> The process is as much a human experience as is the result.

Sure, but in your scenario, the process is the experience for a small group of people, while the result benefits everyone (perhaps not directly, but ultimately via a chain of applications). Which one is more important?

It's not at all a new dilemma either. Jacquard looms destroyed the "human experience" of manual weaving for craftsmen who spent decades to perfect their skill. They also made clothing orders of magnitude cheaper for everyone else.

Re: A misalignment of AI in mathematics

#886

Post by looofooo0 »

40four wrote: post #883
GPerson wrote: post #709

Earlier quoted context omitted.

It’s frustrating that this comment is at the top because it, along with lots of the replies it inspired, absolutely misrepresents the actual declaration. The declaration is not making any statements about not using any AI in mathematics. The entire point is to push the use of the technology in a direction which is compatible with positive pre-existing features of the math community, and to make it better known what s…

I don’t see why the two are mutually exclusive to be honest. I’m not a mathematician, but I’m a software engineer and whether I like it or not, AI has changed my field dramatically forever. I understand the distaste for the way AI might be affecting the field of mathematics, but the genie is out of the bottle, and it will never be the same. It just took a little longer for it to hit the math community. Software engin…

Soon AI mathematics will be 10x or 100x or 1000x more productive than humans in the proofing theorems business. Mathematics of old style is dead, they should develop ideas how to deal with that as a math community.

Re: A misalignment of AI in mathematics

#887

Post by tim333 »

MelonUsk wrote: post #874
tim333 wrote: post #864

It reminds me of Frédéric Bastiat's Candle Makers' Petition. It satirised protectionism in the 1800s: >We candlemakers are suffering from the unfair competition of a foreign rival. This for­eign manufacturer of light has such an advantage over us that he floods our domestic markets with his product. And he offers it at a fantastically low price. The moment this foreigner appears in our country, all our customers de­s…

This is a straw men argument The petition is about a much broader problem: if we’ll demotivate people from understanding math, vibe math in a rush without making it understandable - math becomes useless Basically with pic generation at least we all understand what is on a pic - some cat or something With math it is trivial to generate a lot of stuff that is half-assed and just gives a plain yes/no but doesn’t get abs…

Fair enough - maybe I didn't fully read the statement. Though you can ask AI questions about how things worked sometimes. Like here's me asking the google one how you derive E=mc2 https://share.google/aimode/dtHKAO9Nr0EYWq7JE

It answered about the Navier Stokes too but not at a very deep level https://share.google/aimode/JD69g15GIQiYo9L65

Re: A misalignment of AI in mathematics

#888

Post by Forgeties79 »

int_19h wrote: post #860
Forgeties79 wrote: post #416

Earlier quoted context omitted.

This would be valid if we didn’t live in under capitalism, which reduces the way we spend our time down to dollars and cents.

Correct. So burn down capitalism, not AI?

Or be honest about what’s happening and interrogate both?

Re: A misalignment of AI in mathematics

#889

Post by Forgeties79 »

onethought wrote: post #472
Forgeties79 wrote: post #223

Earlier quoted context omitted.

It’s not a parallel. No one ever claimed photography to be the same as painting or some other medium - it was a new medium that wasn’t respected. AI is being treated and pushed as a replacement for every medium.

It literally was a replacement for portraits.

Portraits are not an artistic medium in the same sense, they’re something one creates with the tools of a particular medium. Painting is a medium, photography is a medium, both are capable of producing portraits.

An LLM can produce a portrait of you as well by generating a facsimile of you based on existing material (or functionally a “guess” if you describe yourself I guess).

Re: A misalignment of AI in mathematics

#890

Post by int_19h »

trillobyte wrote: post #290
td2 wrote: post #233

Earlier quoted context omitted.

But its normal, and good, that technologal progress creates, and destroys some jobs. Imagin a cheap, 100% reliable, self driving car would be released. Death from Traffic incidence fall by orders of magnitude Would you argue it didnt benefit humanity, because taxi/bus drivers are nolonger required

I'm actually a AI optimist. I think it'd be great to have everybody getting around in self driving vehicles. If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy. If the magnitude of changes that OpenAI and Athropic believe will be delivered with in…

> If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?

It cannot. The problem is that everything gets automated, so there's nowhere for people to "transition" to.

More fundamentally, the problem is that capitalism is simply incompatible with this. Specifically: capitalism is a system of property rights which in the long run always results in concentration of capital in a few hands. What made it tolerable so far is that capital cannot produce wealth by itself - you need people to use it to that end. Which in turn means that you need to pay for labor, at least enough for it to sustain itself. So even if you pocket most of the wealth that is generated in the process, the workers still get the crumbs at least.

AI is not like automation in the past because its end goal is doing everything that humans can do. Taken to its logical conclusion, you get capital that doesn't need labor at all: those who own the robots live in a personal post-scarcity utopia, everyone else starves. A slightly better case is when everyone else gets a meager pension, just high enough that people have something to lose and don't try to burn everything down - this is why you hear so much about UBI from the likes of Sam Altman (this isn't to say that UBI is inherently a bad thing! but make no mistake, what they actually want would look a lot more like The Expanse and a lot less like The Culture). It "minimizes disruption" in a sense that there are no riots, and it might even be a material improvement for a lot of the globe, but it sure isn't an improvement for most people living in developed countries today.

So I would argue that, if anything, we need more disruption here, not less. All the way to the top.

Post reply on HN