Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bear case: My predictions regarding AI progress

lesswrong.com

Post reply on HN
Print view View original

211–220 of 220 posts

Re: A bear case: My predictions regarding AI progress

#211

Post by jmholla »

aaronbaugher wrote: post #108
cglace wrote: post #32

The thing I can't wrap my head around is that I work on extremely complex AI agents every day and I know how far they are from actually replacing anyone. But then I step away from my work and I'm constantly bombarded with “agents will replace us”. I wasted a few days trying to incorporate aider and other tools into my workflow. I had a simple screen I was working on for configuring an AI Agent. I gave screenshots of…

It kind of reminds me of the Y2K scare. Leading up to that, there were a lot of people in groups like comp.software.year-2000 who claimed to be doing Y2K fixes at places like the IRS and big corporations. They said they were just doing triage on the most critical systems, and that most things wouldn't get fixed, so there would be all sorts of failures. The "experts" who were closest to the situation, working on it in…

I just want to pile on here. Y2K was avoided due to a Herculean effort across the world to update systems. It was not an imaginary problem. You'll see it again in the lead up to 2038 [0].

[0]: https://en.wikipedia.org/wiki/Year_2038_problem

Re: A bear case: My predictions regarding AI progress

#212

Post by namaria »

fragmede wrote: post #208
namaria wrote: post #199

Earlier quoted context omitted.

How are you surprise that getting an LLM to spit out a clone of a very common starter project is evidence of it being able to generate non trivial and valuable code - as in not a clone of overabundant codebases - on demand?

because in actually doing the exercise, and not just talking about it, you'd come up with your own tweak on the Todo app that couldn't be directly be in the training data. you, as a smart human, could come up with a creative feature for your Todo app to have, that no one else would make, showing that these things can compose between the things in their training data and produce a unique combination that didn't exist…

I think you're getting too distracted with the analogies. LLMs have been shown to spit out easily code for patterns that are very common such as todo apps. That isn't very impressive in itself.

LLMs are cool and show some interesting emergent properties, but knowing why they work getting them to make a very common type of app doesn't show a great deal of emergent ability beyond searching a compressed space and reproducing a common pattern found in it.

It's easy to underestimate the amount and variability of the training data used. There are possibly tens of thousands of variations on todo apps available on github alone.

Re: A bear case: My predictions regarding AI progress

#213

Post by disgruntledphd2 »

fragmede wrote: post #207
disgruntledphd2 wrote: post #205

Earlier quoted context omitted.

Because if they're not covering their costs now, then eventually they will which either means service degradation ( cough ads cough ) or price increases. I applaud the GP for thinking about this before it becomes an issue.

Which, as we know, is what killed YouTube and no one uses that anymore.

Youtube has billions of dollars in ads revenue, and the word on the street is that just covers their costs.

Re: A bear case: My predictions regarding AI progress

#214

Post by fragmede »

namaria wrote: post #212
fragmede wrote: post #208

Earlier quoted context omitted.

because in actually doing the exercise, and not just talking about it, you'd come up with your own tweak on the Todo app that couldn't be directly be in the training data. you, as a smart human, could come up with a creative feature for your Todo app to have, that no one else would make, showing that these things can compose between the things in their training data and produce a unique combination that didn't exist…

I think you're getting too distracted with the analogies. LLMs have been shown to spit out easily code for patterns that are very common such as todo apps. That isn't very impressive in itself. LLMs are cool and show some interesting emergent properties, but knowing why they work getting them to make a very common type of app doesn't show a great deal of emergent ability beyond searching a compressed space and reprod…

So be creative and ask it to make something that isn't a Todo app with thousands examples on github. Knowing that it's really really really advanced autocomplete based on matrix math that does have some funny edge cases because of a numbering subreddit is an interesting degenerate case, but the sheer vastness of the training data it has picked up makes it able to dig into a simultaneously deep and shallow Lego box. it does fall over and go in circles, and having learned to program before the Internet was mainstream, I'm able to go into the code a fix it manually, doesn't impeach it's ability to get that far in the first place.

if it's only able to do the first 90% of the work, and I have to do the last 90% of the work, it's still saved me from doing that first 90% of the work.

Re: A bear case: My predictions regarding AI progress

#215

Post by namaria »

fragmede wrote: post #214
namaria wrote: post #212

Earlier quoted context omitted.

I think you're getting too distracted with the analogies. LLMs have been shown to spit out easily code for patterns that are very common such as todo apps. That isn't very impressive in itself. LLMs are cool and show some interesting emergent properties, but knowing why they work getting them to make a very common type of app doesn't show a great deal of emergent ability beyond searching a compressed space and reprod…

So be creative and ask it to make something that isn't a Todo app with thousands examples on github. Knowing that it's really really really advanced autocomplete based on matrix math that does have some funny edge cases because of a numbering subreddit is an interesting degenerate case, but the sheer vastness of the training data it has picked up makes it able to dig into a simultaneously deep and shallow Lego box. i…

I may be wrong but I coming to the conclusion that the promise that a piece of software can generate any piece of software you can possibly desire and describe is basically saying that P=NP and the halting problem can be solved by a Turing machine that approaches infinite speed.

Re: A bear case: My predictions regarding AI progress

#216

Post by bdangubic »

csomar wrote: post #22

> LLMs still seem as terrible at this as they'd been in the GPT-3.5 age. Software agents break down once the codebase becomes complex enough, game-playing agents get stuck in loops out of which they break out only by accident, etc. This has been my observation. I got into Github Copilot as early as it launched back when GPT-3 was the model. By that time (late 2021) copilot can already write tests for my Rust function…

If you have a revolutionary intelligence product, why is it not working for me?

This is kind of like if I said when the first dumbbell was invented “why don’t I look like arnold schwarzenegger…

Re: A bear case: My predictions regarding AI progress

#217

Post by fragmede »

namaria wrote: post #215
fragmede wrote: post #214

Earlier quoted context omitted.

So be creative and ask it to make something that isn't a Todo app with thousands examples on github. Knowing that it's really really really advanced autocomplete based on matrix math that does have some funny edge cases because of a numbering subreddit is an interesting degenerate case, but the sheer vastness of the training data it has picked up makes it able to dig into a simultaneously deep and shallow Lego box. i…

I may be wrong but I coming to the conclusion that the promise that a piece of software can generate any piece of software you can possibly desire and describe is basically saying that P=NP and the halting problem can be solved by a Turing machine that approaches infinite speed.

I'm not claiming that it can generate any piece of software you can possibly desire, but that there are enough examples of quite a lot of pieces of them, and that it can compose them into something that it hasn't directly seen before. Like an astronaut on a horse that was popular for Stablediffusion. It didn't have that directly in the training data but it had enough of the pieces that it's able to create a reasonable looking version of one. That it produces too many fingers on hands and has no concept of writing is a glaring obvious shortcoming, just like LLMs generating buggy inefficient code is another one. Catching it's fuckups is missing the point.

My point is that it's such a game changer that you ignore it at your own peril. Just go into a side project, half-cocked and get shit built. Yes the code will be ugly but it got built. Maybe. It has its limits, as does the operator's patience, so it's entirely possible you'll run into a bug it can't fix. But a smart operator knows when to stop it and dig into the problem and fix it manually.

Funnily enough though, if you give it some toy code that doesn't ever complete, like a Fibonacci number generator and ask it if it will halt, it's able to point out that it won't. That, of course, is because those are in the training data, but it's cute nonetheless.

Re: A bear case: My predictions regarding AI progress

#218

Post by namaria »

fragmede wrote: post #217
namaria wrote: post #215

Earlier quoted context omitted.

I may be wrong but I coming to the conclusion that the promise that a piece of software can generate any piece of software you can possibly desire and describe is basically saying that P=NP and the halting problem can be solved by a Turing machine that approaches infinite speed.

I'm not claiming that it can generate any piece of software you can possibly desire, but that there are enough examples of quite a lot of pieces of them, and that it can compose them into something that it hasn't directly seen before. Like an astronaut on a horse that was popular for Stablediffusion. It didn't have that directly in the training data but it had enough of the pieces that it's able to create a reasonabl…

Thanks for the advice, I mean it without irony.

But I think if you know how to code well you can use code generators, whereas if you know how to use code generators you don't necessarily learn how to code. And being able to review generated code stems from mastering code, not from fiddling with prompts.

More generally, I think the premise that using code generators is a skill is empty. They don't require special skill beyond understanding programming well so it is probably a better use of my time to keep on coding and if someone wants to give me money to use code generators I am in a better position having spent lots of time writing code.

And if code generators become so good that it requires no skill to create software, then all this is moot. But of course I think that is nonsense because it seems to me that depends on the foundations of computer science being wrong.

Re: A bear case: My predictions regarding AI progress

#219

Post by fragmede »

namaria wrote: post #218
fragmede wrote: post #217

Earlier quoted context omitted.

I'm not claiming that it can generate any piece of software you can possibly desire, but that there are enough examples of quite a lot of pieces of them, and that it can compose them into something that it hasn't directly seen before. Like an astronaut on a horse that was popular for Stablediffusion. It didn't have that directly in the training data but it had enough of the pieces that it's able to create a reasonabl…

Thanks for the advice, I mean it without irony. But I think if you know how to code well you can use code generators, whereas if you know how to use code generators you don't necessarily learn how to code. And being able to review generated code stems from mastering code, not from fiddling with prompts. More generally, I think the premise that using code generators is a skill is empty. They don't require special skil…

Hm, that's fair. I suppose I should be more transparent as to my bias. I learned to program as the Internet was coming online and before Google, Stack Overflow, and now, ChatGPT. Not trying to brag, merely trying to show that I know how to code.

I can't say how I'd actually feel if I was just starting out, but your position isn't unreasonable. I will disagree and say there is a certain skill to prompting, even though calling prompt engineering is maybe ridiculous. Given the way the industry is trending, getting code generators into a corner; having it run around in circles, and then having in skill to fix it by hand, quickly, is the skill to hone. Whether that comes from practicing writing code from scratch, or from using code generation and then fixing the bugs it creates though, is up for debate. I obviously think the latter but that's just a random person who's not in your shoes' opinion.

Best of luck to you in your career! Hopefully it goes well, no matter which direction it takes.

Re: A bear case: My predictions regarding AI progress

#220

Post by Breza »

roenxi wrote: post #60
roncesvalles wrote: post #56

Earlier quoted context omitted.

The last line has been my experience as well. I only trust what I've verified firsthand now because the Internet is just so rife with people trying to influence your thoughts in a way that benefits them, over a good faith sharing of the truth. I just recently heard this quote from a clip of Jeff Bezos: "When the data and the anecdotes disagree, the anecdotes are usually right.", and I was like... wow. That quote is t…

> I knew Uber, Netflix, Spotify were revolutionary the first time I used them. Maybe re-tune your revolution sensor. None of those are revolutionary companies. Profitable and well executed, sure, but those turn up all the time. Uber's entire business model was running over the legal system so quickly that taxi licenses didn't have time to catch up. Other than that it was a pretty obvious idea. It is a taxi service. T…

I strongly disagree about Netflix. It came out when I was in high school without a car. Being able to get whatever DVD I wanted without having to bum a ride from my parents--and also never have to pay late fees--was a major game changer.
Post reply on HN