Problems the AI industry is not addressing adequately • Hacker News Board

Re: Problems the AI industry is not addressing adequately

#81

Post by deadbabe » Sat, Jul 05, 2025, 3:28 PM UTC

imiric wrote: post #75

matusp wrote: post #8

Observe what the AI companies are doing, not what they are saying. If they would expect to achieve AGI soon, their behaviour would be completely different. Why bother developing chatbots or doing sales, when you will be operating AGI in a few short years? Surely, all resources should go towards that goal, as it is supposed to usher the humanity into a new prosperous age (somehow).

Related to your point: if these tools are close to having super-human intelligence, and they make humans so much more productive, why aren't we seeing improvements at a much faster rate than we are now? Why aren't inherent problems like hallucination already solved, or at least less of an issue? Surely the smartest researchers and engineers money can buy would be dogfooding, no? This is the main point that proves to…

Data from the future is tunneling into the past to mess up our weights and ensure we never achieve AGI.

Re: Problems the AI industry is not addressing adequately

#82

Post by darth_aardvark » Sat, Jul 05, 2025, 3:34 PM UTC

impossiblefork wrote: post #71

blululu wrote: post #45

Earlier quoted context omitted.

Kind of a sucker move here since you personally will 100% be forgotten. We are only going to remember one or two people who did any of this. Say Sam Altman and Ilya Sttsveker. Everyone else will be forgotten. The authors or the Transformer paper are unlikely to make it into the history books or even popular imagination. Think about the Manhattan Project. We recently made a movie remembering that one guy who did somet…

Personally I think the ones who will be remembered will be the ones who publish useful methods first, not the ones who succeed commercially. It'll be Vaswani and the others for the transformer, then maybe Zelikman and those on that paper for thought tokens, then maybe some of the RNN people and word embedding people will be cited as pioneers. Sutskever will definitely be remembered for GPT-1 though, being first to re…

Is "we" the particular set of scientists who know those last four people? Surely you realize they're nowhere near as famous as the Wright brothers, right? This is giving strong https://xkcd.com/2501/ feelings.

Re: Problems the AI industry is not addressing adequately

#83

Post by impossiblefork » Sat, Jul 05, 2025, 3:39 PM UTC

darth_aardvark wrote: post #82

impossiblefork wrote: post #71

Earlier quoted context omitted.

Personally I think the ones who will be remembered will be the ones who publish useful methods first, not the ones who succeed commercially. It'll be Vaswani and the others for the transformer, then maybe Zelikman and those on that paper for thought tokens, then maybe some of the RNN people and word embedding people will be cited as pioneers. Sutskever will definitely be remembered for GPT-1 though, being first to re…

Is "we" the particular set of scientists who know those last four people? Surely you realize they're nowhere near as famous as the Wright brothers, right? This is giving strong https://xkcd.com/2501/ feelings.

Yes, that is indeed the 'we', but I think more people are knowledgeable than is obvious.

I'm not an aerodynamicist, and I know about those guys, so they can't be infinitely obscure. I imagine every French person knows about Bleriot at least.

Re: Problems the AI industry is not addressing adequately

#84

Post by hamburga » Sat, Jul 05, 2025, 3:41 PM UTC

> This reminds me of a paradox: The AI industry is concerned with the alignment problem (how to make a super smart AI adhere to human values and goals) while failing to align between and within organizations and with the broader world. The bar they’ve set for themselves is simply too high for the performance they’re putting out.

My argument is that it’s our job as consumers to align the AIs to our values (which are not all the same) via selection pressure: https://muldoon.cloud/2025/05/22/alignment.html

Re: Problems the AI industry is not addressing adequately

#85

Post by archeantus » Sat, Jul 05, 2025, 3:44 PM UTC

bombcar wrote: post #36

A_D_E_P_T wrote: post #4

> "This is purely an observation: You only jump ship in the middle of a conquest if either all ships are arriving at the same time (unlikely) or neither is arriving at all. This means that no AI lab is close to AGI." The central claim here is illogical. The way I see it, if you believe that AGI is imminent, and if your personal efforts are not entirely crucial to bringing AGI about (just about all engineers are in th…

>your best move is to do whatever is most profitable in the near-term Unless you’re a significant shareholder, that’s almost always the best move, anyway. Companies have no loyalty to you and you need to watch out for yourself and why you’re living.

I read that most of the crazy comp Zuck is offering is in stock. So in a way, going to the place where they have lots of stock reflects their belief about where AGI is going to happen first.

Re: Problems the AI industry is not addressing adequately

#86

Post by eMPee584 » Sat, Jul 05, 2025, 3:53 PM UTC

sys_64738 wrote: post #41

rvz wrote: post #9

Are we finally realizing that the term "AGI" is not only hijacked to become meaningless, but achieving it has always been nothing but a complete scam as I was saying before? [0] If you were in a "pioneering" AI lab that claims to be in the lead in achieving "AGI", why move to another lab that is behind other than offering $10M a year. Snap out of the "AGI" BS. [0] https://news.ycombinator.com/item?id=37438154

I don't pay too close attention to AI as it always felt like man behind the curtain syndrome. But where did this "AGI" term even come from? The original term AI is meant to be AGI so when did "AI" get bastardized into what abomination it is meant to refer to now.

capitalism all the way down..

Re: Problems the AI industry is not addressing adequately

#87

Post by SoftTalker » Sat, Jul 05, 2025, 3:55 PM UTC

sys_64738 wrote: post #41

rvz wrote: post #9

Are we finally realizing that the term "AGI" is not only hijacked to become meaningless, but achieving it has always been nothing but a complete scam as I was saying before? [0] If you were in a "pioneering" AI lab that claims to be in the lead in achieving "AGI", why move to another lab that is behind other than offering $10M a year. Snap out of the "AGI" BS. [0] https://news.ycombinator.com/item?id=37438154

I don't pay too close attention to AI as it always felt like man behind the curtain syndrome. But where did this "AGI" term even come from? The original term AI is meant to be AGI so when did "AI" get bastardized into what abomination it is meant to refer to now.

See the history of Tesla and "full self-driving" for the explanation. In short: for sales.

Re: Problems the AI industry is not addressing adequately

#88

Post by Davidzheng » Sat, Jul 05, 2025, 3:55 PM UTC

GolfPopper wrote: post #54

Davidzheng wrote: post #23

Earlier quoted context omitted.

what's your definition? AGI original definition is median human across almost all fields which I believe is basically achieved. If superhuman (better than best expert) I expect <2030 for all nonrobotic tasks and <2035 for all tasks

A "median human" can run a web search and report back on what they found without making stuff up, something I've yet to find an LLM capable of doing reliably.

I bet you median humans make up a nontrivial amount of things. Humans misremember all the time. If you ask for only quotes, LLMs can also do this without problems (I use o3 for search over google)

Re: Problems the AI industry is not addressing adequately

#89

Post by Davidzheng » Sat, Jul 05, 2025, 4:00 PM UTC

gnz11 wrote: post #60

Davidzheng wrote: post #23

Earlier quoted context omitted.

what's your definition? AGI original definition is median human across almost all fields which I believe is basically achieved. If superhuman (better than best expert) I expect <2030 for all nonrobotic tasks and <2035 for all tasks

How are you coming to the conclusion that "median human" is "basically achieved"? Current AI has no means of understanding and synthesizing new ideas the way a human would. It's all generative.

synthesizing new ideas: in order to express the idea in our language it basically means you have some new combinations of existing building blocks, just sometimes the building blocks are low level enough and the combination is esoteric enough. It's a spectrum again. I think current models are in fact quite capable of combining existing ideas and building blocks in new ways (this is how human innovation also happens). Most of my evidence comes from asking newer models o3/gemini-2.5-pro for research-level mathematics questions which do not appear in existing literature but is of course connected with them.

so these arguments by fundamental distinctions I believe all cannot work--the question is how new are the AI contributions. Nowadays there's of course still no theoretical breakthroughs in mathematics from AI (though biology could be close!). Also I think the AIs have understanding--but tbf the only thing we can test is through testing on tricky questions which I think support my side. Though of course some of these questions have interpretations which are not testable--so I don't want to argue about those.

Re: Problems the AI industry is not addressing adequately

#90

Post by bluGill » Sat, Jul 05, 2025, 4:01 PM UTC

matusp wrote: post #8

Observe what the AI companies are doing, not what they are saying. If they would expect to achieve AGI soon, their behaviour would be completely different. Why bother developing chatbots or doing sales, when you will be operating AGI in a few short years? Surely, all resources should go towards that goal, as it is supposed to usher the humanity into a new prosperous age (somehow).

The people who make the money in gold rushes sold shovels, not mined the gold. Sure some random people found gold and made a lot of money, but many others didn't strike it rich.

As such even if there is a lot of money AI will make, it can still be the right decision to sell tools to others who will figure out how to use it. And of course if it turns out another pointless fad with no real value you still make money. (I'd predict the answer is in between - we are not going to get some AGI that takes over the world, but there will be niches where it is a big help and those niches will be worth selling tools into)