Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unpredictable abilities emerging from large AI models

quantamagazine.org

Post reply on HN
Print view View original

241–250 of 326 posts

Re: Unpredictable abilities emerging from large AI models

#241

Post by majormajor »

tarruda wrote: post #80
throwbadubadu wrote: post #65

Earlier quoted context omitted.

Oh man... if it will be as stupid as us humans we are completely doomed. For now lets assume what it most likely is: not emerging anything, just a very good simulation of that emergeance. I mean we already did stupid chat bots in the 90s with convoluted if-else-spaghettis that imitated Eliza. That could already hold for some sentences... GPT-4 now holds much longer, but still you can easily drive it to the point wher…

I have very limited understanding of how these systems are created, but as far as I know they are not programmed to simulate anything. It is just a system that outputs the most likely next word based on a really large dataset of training samples it was given. Yet you are still able to "chat" with these systems. Here's a session I just ran on llama.cpp (7 billion parameter version). Everything above "====" line is the…

The characterization of HN as a "question and answer" website is a fun one.

Is it wrong, or is it getting at the fact that so many people don't read the original articles, just the comments? ;)

Re: Unpredictable abilities emerging from large AI models

#242

Post by lxgr »

sebzim4500 wrote: post #36
legulere wrote: post #19

Earlier quoted context omitted.

The same was said 10 years ago. It's astonishing what can be done, but you can already see fundamental limits. I think it will raise productivity for some tasks, but not fundamentally change society.

The number of people saying it now is many orders of magnitude more than the number of people saying it 10 years ago. Not saying that means it will happen, but it isn't the same situation.

> The number of people saying it

Popular sentiment is a pretty meaningless metric for predicting the future.

Re: Unpredictable abilities emerging from large AI models

#243

Post by Animats »

simonw wrote: post #220
UncleOxidant wrote: post #177

Earlier quoted context omitted.

Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?

I can already feel it dramatically increasing my personal productivity. I expect it's going to eventually lead to a 2x or 3x time productivity increase for software engineering generally (if not more). Will that result in less demand for software engineers? I doubt it. I think it will lead to companies and organizations doing way more software engineering. I was going to use the old "ATMs resulted in more jobs for ba…

> apparently that particular example is flawed

Um, yes. Been in a bank branch lately? Had to visit a bank branch for any reason?

Re: Unpredictable abilities emerging from large AI models

#244

Post by jmvoodoo »

timdiggerm wrote: post #130
jmvoodoo wrote: post #89

Earlier quoted context omitted.

What's most interesting to me, is that this is how I would expect a human to approach the problem if presented with the code and asked for the output. The LLM didn't run the code, it tried to predict the output based on its knowledge of python and primes.

Sorry, did it try to predict the output, or did it only predict the output based on what it has seen people say about similar code online?

I think its knowledge of python can only come from comments about code, since it can't execute the python or learn from trying code like a human would.

That doesn't change the fact that its predicting the output based on its knowledge of python though.

So, yes you're right, but at the same time it doesn't change anything IMO.

Re: Unpredictable abilities emerging from large AI models

#245

Post by Animats »

guns wrote: post #91
mpalmer wrote: post #59

I'd like to see posts on LLMs written from a different perspective. For me, the surprise comes not from the sudden emergent capability of language models, but that the understanding (and synthesis!) of ideas encoded in language has succumbed to literally nothing more than statistical analysis. Or at least come that much closer to doing so. That it bears so close a resemblance to actual thinking says more about the im…

This is what Stephen Wolfram concludes in a recent article about ChatGPT: > The specific engineering of ChatGPT has made it quite compelling. But ultimately (at least until it can use outside tools) ChatGPT is “merely” pulling out some “coherent thread of text” from the “statistics of conventional wisdom” that it’s accumulated. But it’s amazing how human-like the results are. And as I’ve discussed, this suggests some…

> (at least until it can use outside tools)

This is key. ChatGPT/GPT-4 alone are limited to reformulating what they know from their training data. Linked to search engines, databases, and computational tools such as Wolfram Alpha, they acquire much more capability. We're already seeing that with Microsoft Bing.

(Update: what happens as large language models learn Excel? Especially since Microsoft is already connecting them to Excel.)

What's striking is how fast this field is advancing. Huge advances over months, not years or decades.

We now have a much better idea of how intelligence evolved. It's mostly just more neurons. One of the great philosophical questions has, inadvertently, been answered.

Is the Singularity happening right now?

Re: Unpredictable abilities emerging from large AI models

#246

Post by alexthehurst »

fnordpiglet wrote: post #221
bcrosby95 wrote: post #178

Earlier quoted context omitted.

Your opinion on this probably depends on what your job currently looks like. Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me. But if you work at BigCo and mostly sit on your hands already, then, yeah. I don't know.

Every BigCo I’ve worked at (and I’ve worked at places that put the B in Big) people aren’t sitting on their hands because there isn’t way more to do than they can feasibly do. It’s usually because they’re fundamentally illsuited for the job and sitting on their hands is better than being homeless, or the people who are illsuited for their jobs feel anxious about being homeless so force everyone into meetings and team…

> No AI will ever host a breakout session at an offsite to discuss strategy for getting people to work in the horrific hotel seating dehumanized office

Well, let’s hope not.

Re: Unpredictable abilities emerging from large AI models

#247

Post by kefabean »

drdec wrote: post #201
UncleOxidant wrote: post #175

Earlier quoted context omitted.

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs wil…

> Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. OTOH, software development will become significantly cheaper. That means the business case for using software in more places just tilted in favor of throwing some software at…

Kind of like a Jevon’s paradox for software engineering!

Re: Unpredictable abilities emerging from large AI models

#248

Post by UncleOxidant »

drdec wrote: post #201
UncleOxidant wrote: post #175

Earlier quoted context omitted.

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs wil…

> Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. OTOH, software development will become significantly cheaper. That means the business case for using software in more places just tilted in favor of throwing some software at…

> You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses.

And in a lot of cases they're not going to need to hire a software developer to do develop it.

Re: Unpredictable abilities emerging from large AI models

#249

Post by saulpw »

layer8 wrote: post #82
ishvanl wrote: post #33

There is an idea bouncing around my brain for a pipe-dream science fiction book I could write, about the rise of the first truly sentient (or sentient-appearing) AI & its effects on the world. It is beginning to get to the point where I am considering whether I need to actually start writing this book before this "sci-fi" concept becomes a mundane reality.

I’m looking forward to the “alternative history” SF novels of the future where AGI never materialized.

I wonder what kind of __punk this will be called..

Re: Unpredictable abilities emerging from large AI models

#250

Post by FormulatedEdits »

pedrovhb wrote: post #10

This caught my attention as I found it implausible: > One DeepMind engineer even reported being able to convince ChatGPT that it was a Linux terminal and getting it to run some simple mathematical code to compute the first 10 prime numbers. Remarkably, it could finish the task faster than the same code running on a real Linux machine. Following the link, there's a screenshot to a screenshot [0] of a code-golf solutio…

Its speed as compared to the laptop - I assume it was running online, on OpenAI’s hardware. That hardware is quite extensive, I thought. I believe I’ve seen the cost of one (not ‘search’, what’s the word here, ‘output’?) is something like 1-10 cents.
Post reply on HN