Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unpredictable abilities emerging from large AI models

quantamagazine.org

Post reply on HN
Print view View original

211–220 of 326 posts

Re: Unpredictable abilities emerging from large AI models

#211

Post by felipeerias »

mpalmer wrote: post #59

I'd like to see posts on LLMs written from a different perspective. For me, the surprise comes not from the sudden emergent capability of language models, but that the understanding (and synthesis!) of ideas encoded in language has succumbed to literally nothing more than statistical analysis. Or at least come that much closer to doing so. That it bears so close a resemblance to actual thinking says more about the im…

Some anthropologists suggest that our main evolutionary advantage was not so much our individual ability for reasoning, but our collective capacity to accumulate, evolve and transmit cultural knowledge over the centuries and millennia.

Skills like fire or language, for example, had a major influence in the development of our species and are mainly culturally transmitted: trying to reason your way into creating one or the other from scratch is a surprisingly difficult task.

If that point of view is true, then it shouldn’t be surprising that a large part of what we consider human-like behaviours should be tractable simply by analysing large amounts of data. AI systems are not modelling cognition, but culture.

Re: Unpredictable abilities emerging from large AI models

#212

Post by Izkata »

WC3w6pXxgGd wrote: post #197
olalonde wrote: post #195

Earlier quoted context omitted.

Indeed, it's fascinating to witness a sizable segment of the HN community and distinguished intellectuals like Chomsky displaying absolute denial. I've started bookmarking the comments so I can look back at them in 5 years and have a good laugh. Some from a few months back are already aging badly[0]. [0] https://news.ycombinator.com/item?id=34197033

Has Chomsky been right about anything?

My first introduction to him was when one uncontacted tribe or anothers' language refuted something he thought was fundamental to humans.

In the decade(s) since I kinda think "Chomsky was wrong" is pretty much the only reason people bring him up.

Re: Unpredictable abilities emerging from large AI models

#214

Post by civilized »

coldtea wrote: post #206
HervalFreire wrote: post #92

Earlier quoted context omitted.

What they said 10 years ago was correct. It did hit society like a sledge hammer. Machine learning basically took over the AI space and penetrated the consumer space with applications that were all but impossible in the previous decade. There's AI chips in smart phones now. What you're seeing here with LLMs is sledge hammer number 2. It's understandable how most people don't notice the sledge hammer. The decade prior…

> What they said 10 years ago was correct. It did hit society like a sledge hammer. Machine learning basically took over the AI space and penetrated the consumer space with applications that were all but impossible in the previous decade. There's AI chips in smart phones now. And still almost all of these applications are not really impactful or that important compared to actually society and life changing developmen…

I feel like the only form of "AI" I regularly benefit from is when I type "n" in the address bar and it autocompletes to news.ycombinator.com.

Oh, browser, you know me so well.

Re: Unpredictable abilities emerging from large AI models

#215

Post by HyperSane »

WC3w6pXxgGd wrote: post #197
olalonde wrote: post #195

Earlier quoted context omitted.

Indeed, it's fascinating to witness a sizable segment of the HN community and distinguished intellectuals like Chomsky displaying absolute denial. I've started bookmarking the comments so I can look back at them in 5 years and have a good laugh. Some from a few months back are already aging badly[0]. [0] https://news.ycombinator.com/item?id=34197033

Has Chomsky been right about anything?

About foreign policy? No.

Re: Unpredictable abilities emerging from large AI models

#216

Post by warkdarrior »

Ambolia wrote: post #163
datpiff wrote: post #93

Earlier quoted context omitted.

> According to NVidia, LLM sizes have been increasing 10X per year for the last few years. Clearly this cannot continue, as the training costs will exceed all the compute capacity in existence. The other limit is training data, eventually you run out of cheap sources.

They may also run out of data if they already consumed most of the internet. Or start producing so much of the internet's content that LLMs start consuming what they write in a closed loop.

Sure, but they do not need to grow infinitely more, they just need to grow sufficiently to perform better than 90% of humans.

Re: Unpredictable abilities emerging from large AI models

#217

Post by bentcorner »

ip26 wrote: post #210
anon7725 wrote: post #152

Earlier quoted context omitted.

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career. I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to da…

Somewhat similar boat; I just hope it winds up helping me do my job better, instead of me trying to teach it how to do my job. Setting aside professional anxieties, after using chatGPT for a bit I quickly realized I am completely uninterested in prompt engineering. But as a tool, like with an IDE, it could be a big leap forward.

I literally just finished writing some documentation and had Copilot running, sometimes it'd autocomplete something with garbage (and I would just type what I wanted anyway), more than once it autocompleted with several lines of exactly what I wanted to write. This is way better than the phone-type autocomplete I've seen in gmail where it just guesses the next word.

Re: Unpredictable abilities emerging from large AI models

#218

Post by fnordpiglet »

bentcorner wrote: post #217
ip26 wrote: post #210

Earlier quoted context omitted.

Somewhat similar boat; I just hope it winds up helping me do my job better, instead of me trying to teach it how to do my job. Setting aside professional anxieties, after using chatGPT for a bit I quickly realized I am completely uninterested in prompt engineering. But as a tool, like with an IDE, it could be a big leap forward.

I literally just finished writing some documentation and had Copilot running, sometimes it'd autocomplete something with garbage (and I would just type what I wanted anyway), more than once it autocompleted with several lines of exactly what I wanted to write. This is way better than the phone-type autocomplete I've seen in gmail where it just guesses the next word.

I ducking agree

Re: Unpredictable abilities emerging from large AI models

#219

Post by enord »

blurbleblurble wrote: post #202
enord wrote: post #182

Earlier quoted context omitted.

«Computationally universal»? Are you quite sure? Hoo boy pass the nitroglycerin hooooo aaah ohh after 80 years Turing was proven right after all ghasp

Well I don't understand the nitroglycerin reference and of course "computationally universal" doesn't mean "sentient" but the point is when you add the external memory they (Flan-U-PaLM 540B to be specific) have been demonstrated to be capable of simulating a specific, well defined Turing machine without special training. There are some other papers out there arguing this from a theoretical angle too, but this is the…

5 thermostats in the right comfiguration can be a universal computer. Approximation, in the computational sense, is relative to some well defined function, which gives the approximation an ontology. The feasability of approximation is not disputed, but the content and nature of the ontology is a battlefield of motivated rhetoric.

Yes, this is arguing semantics but in this particular case, semantics is the whole point.

Re: Unpredictable abilities emerging from large AI models

#220

Post by simonw »

UncleOxidant wrote: post #177
roflyear wrote: post #165

Earlier quoted context omitted.

I think it'll end at using chatgpt as a great tool. And that seems fine. I think much of the anxiety here is in the unknown. You have to learn your IDE, but it doesn't make programming harder. It makes it more fun actually. Chatgpt is going to require that devs are even better IMO because you have to catch its errors and know how to work with it.

Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?

I can already feel it dramatically increasing my personal productivity. I expect it's going to eventually lead to a 2x or 3x time productivity increase for software engineering generally (if not more).

Will that result in less demand for software engineers? I doubt it. I think it will lead to companies and organizations doing way more software engineering.

I was going to use the old "ATMs resulted in more jobs for bank tellers example", but apparently that particular example is flawed: https://www.vox.com/2017/5/8/15584268/eric-schmidt-alphabet-...

Post reply on HN