Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ilya Sutskever: We're moving from the age of scaling to the age of research

dwarkesh.com

Post reply on HN
Print view View original

361–370 of 374 posts

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#361

Post by camillomiller »

throwaway2037 wrote: post #146
camillomiller wrote: post #22

Earlier quoted context omitted.

Fridman is a morally broken grifter, who just built a persona and a brand on proven lies, claiming an association with MIT that was de facto non-existent. Not wanting to give the guy recognition is not a matter of being liberal or conservative, but just interested in truthfulness.

> claiming an association with MIT that was de facto non-existent Google search: "lex fridman and mit" Second hit: https://cces.mit.edu/team/lex-fridman/ > Lex conducts research in AI, human-robot interaction, autonomous vehicles, and machine learning at MIT.

He does not. He taught one IAP course, which is a joke. He is also basically the only one in that page with a one-liner description.

Here's more information on why he's a massive fraud:

https://medium.com/the-pub/is-lex-fridman-a-fraud-722a82b6ec...

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#362

Post by illwrks »

In the “teenagers learn to drive in 10 hours” part… that’s active learning, but they have spent countless hours in their life in a car, on a bus or other forms of transport, even watching the shows and movies featuring driving, playing with toys and computer games etc. There is years of passive information absorbed already before that 10 hours of active learning begins.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#363

Post by Schlagbohrer »

simianparrot wrote: post #234
selectodude wrote: post #210

Earlier quoted context omitted.

> the capacity goes with it. Sort of. The GPUs exist. Maybe LLM subs can’t pay for electricity plus $50,000 GPUs, but I bet after some people get wiped out, there’s a market there.

Datacenter GPU's have a lifespan of 1-3 years depending on use. So yes they exist, but not for long, unless they go entirely unused. But then they also deprecate in efficiency compared to new hardware extremely fast as well, so their shelf life is severely limited either way.

This is why I find the business case of putting datacenters in orbit to be so stupid. And yet there are several startups saying they are gonna do just that.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#364

Post by aswegs8 »

HarHarVeryFunny wrote: post #321
joelthelion wrote: post #227

> When do you expect that impact? I think the models seem smarter than their economic impact would imply. > Yeah. This is one of the very confusing things about the models right now. As someone who's been integrating "AI" and algorithms into people's workflows for twenty years, the answer is actually simple. It takes time to figure out how exactly to use these tools, and integrate them into existing tooling and workf…

No doubt LLMs and tooling will continue to improve, and best use cases for them better understood, but what Ilya seems to be referring to is the massive disconnect between the headline-grabbing benchmarks such as "AI performs at PhD level on math", etc, and the real-world stupidity of these models such as his example of a coding agent toggling between generating bug #1 vs bug #2, which in fact largely explains why th…

There is actually an interesting scenario in this disconnect that we are experiencing. Maybe "real" AGI in the sense of intelligence that self-corrects effectively like a human is still a long way. Maybe we will be stuck with this kind of ever-improving but still kind of deficient LLM intelligence we have right now.

There are tons of use cases even for such a limited type of intelligence. No, it is not a million math PhDs at your disposal. It is a narrow intelligence that is still hugely useful and businesses will need a few years to adapt. The impact on topics like customer service with LLM+RAG+triggering actions is very close already and should transform the industry in the next years.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#365

Post by HarHarVeryFunny »

aswegs8 wrote: post #364
HarHarVeryFunny wrote: post #321

Earlier quoted context omitted.

No doubt LLMs and tooling will continue to improve, and best use cases for them better understood, but what Ilya seems to be referring to is the massive disconnect between the headline-grabbing benchmarks such as "AI performs at PhD level on math", etc, and the real-world stupidity of these models such as his example of a coding agent toggling between generating bug #1 vs bug #2, which in fact largely explains why th…

There is actually an interesting scenario in this disconnect that we are experiencing. Maybe "real" AGI in the sense of intelligence that self-corrects effectively like a human is still a long way. Maybe we will be stuck with this kind of ever-improving but still kind of deficient LLM intelligence we have right now. There are tons of use cases even for such a limited type of intelligence. No, it is not a million math…

Yes - LLMs are useful, even if auto-regressively trained GPTs aren't the answer to human intelligence, and outside of software development (maybe there too) it seems we're still very early in companies trying to figure out what they can and can not usefully be used for.

It seems the LLM companies generating all the hype (mostly OpenAI & Anthropic) may be shooting themselves in the foot a bit here, raising false expectations of what LLMs can do, or soon will be able to do, and therefore encouraging all the misapplication and failed corporate projects that are currently happening. Anthropic are talkiing out of both sides of their mouth here, saying that AGI is imminent, about to replace developers and remote workers, yet acknowledging that the technology and use case selection is so fickle that corporations aren't likely to be successful without 1-on-1 guidance from Anthropic.

The mythical AGI, an artificial human, will presumably be transformative if/when it ever arrives, but even if we're still early days in LLM adoption it's not clear if that (LLMs) really will be. Developers get a new tool to use, consumers get a new frustrating AI customer service to deal with, corporate e-mails, marketing literature and powerpoints become enshittified LLM-generated AI slop, etc. Maybe the biggest "transformative" (widely felt) impact of LLMs is potentially chatbots and AI-search, but it seems people are just taking that in their stride, and not obvious that the experience and impact from that is going to change much going forwards.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#366

Post by mrwrong »

fireant wrote: post #353
magicalist wrote: post #309

Earlier quoted context omitted.

> In medicine, we're already seeing productivity gains from AI charting leading to an expectation that providers will see more patients per hour. And not, of course, an expectation of more minutes of contact per patient, which would be the better outcome optimization for both provider and patient. Gotta pump those numbers until everyone but the execs are an assembly line worker in activity and pay.

I don't think that more minutes of contact is better for anybody. As a patient, I want to spend as little time with a doctor as possible and still receive maximally useful treatment. As a doctor, I would want to extract maximal comp from insurance which I don't think is tied time spent with the patient, rather to a number of different treatments given. Also please note that in most western world medical personnel is…

> leading to an expectation that providers will see more patients per hour

> reducing their overall workload

what?

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#368

Post by nateglims »

riku_iki wrote: post #336
HarHarVeryFunny wrote: post #321

Earlier quoted context omitted.

No doubt LLMs and tooling will continue to improve, and best use cases for them better understood, but what Ilya seems to be referring to is the massive disconnect between the headline-grabbing benchmarks such as "AI performs at PhD level on math", etc, and the real-world stupidity of these models such as his example of a coding agent toggling between generating bug #1 vs bug #2, which in fact largely explains why th…

> the real-world stupidity of these models such as his example of a coding agent toggling between generating bug #1 vs bug #2, which in fact largely explains why the current economic and visible impact is much less than if the "AI is PhD level" benchmark narrative was actually true. this could be true in the past, but in recent weeks I started more and more trust top AI models and less PhDs I work with. Quality jump…

Are you a mathematician? I’m not an expert on the math field but it seems like they are hitting the same issues everyone else has: current LLMs still more or less need to be supervised by an expert and struggle to do something actually novel or build out a complicated proof correctly.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#369

Post by riku_iki »

nateglims wrote: post #368
riku_iki wrote: post #336

Earlier quoted context omitted.

> the real-world stupidity of these models such as his example of a coding agent toggling between generating bug #1 vs bug #2, which in fact largely explains why the current economic and visible impact is much less than if the "AI is PhD level" benchmark narrative was actually true. this could be true in the past, but in recent weeks I started more and more trust top AI models and less PhDs I work with. Quality jump…

Are you a mathematician? I’m not an expert on the math field but it seems like they are hitting the same issues everyone else has: current LLMs still more or less need to be supervised by an expert and struggle to do something actually novel or build out a complicated proof correctly.

I work in math heavy applied setting. Randomly hired PhDs are also need to be supervised, end results being monitored, code be reviewed or they will make lots of mistakes, and my view is if you throw some problem like: build optimization model for this kind of problem on this kind of data, LLMs may produce better results.

Re: Ilya Sutskever: We're moving from the age of scaling to the age of research

#370

Post by HarHarVeryFunny »

nateglims wrote: post #368
riku_iki wrote: post #336

Earlier quoted context omitted.

> the real-world stupidity of these models such as his example of a coding agent toggling between generating bug #1 vs bug #2, which in fact largely explains why the current economic and visible impact is much less than if the "AI is PhD level" benchmark narrative was actually true. this could be true in the past, but in recent weeks I started more and more trust top AI models and less PhDs I work with. Quality jump…

Are you a mathematician? I’m not an expert on the math field but it seems like they are hitting the same issues everyone else has: current LLMs still more or less need to be supervised by an expert and struggle to do something actually novel or build out a complicated proof correctly.

There's a limit to how much novelty you're going to get from an LLM, especially in areas like programming and math where they've been heavily RL'd NOT to be novel, even to extent that the base model supports, and instead generate much narrower more proscribed outputs.

The limit to the novelty you are going to get from an LLM is essentially the "deductive/generative closure" of the training data. To be truly novel and move past the limits of your own past experience requires things like curiosity, continual learning, and the autonomy/agency to explore and learn.

Post reply on HN