Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big Tech Has Suddenly Flipped on the AI Jobs Wipeout Scenario

wsj.com

Post reply on HN
Print view View original

101–106 of 106 posts

Re: Big Tech Has Suddenly Flipped on the AI Jobs Wipeout Scenario

#101

Post by intended »

foldr wrote: post #87
taurath wrote: post #79

Earlier quoted context omitted.

I find it to not be acceptable if AI's trend is to degrade performance of both AI and humans at the same time, that's kinda not the goal, right? Why are we spending money to make us dumber?

AI isn't going to go away, and AI-generated content isn't going to go away. So while it's an open question the extent to which AI training will be hampered by the proliferation of AI-generated content, I think that the existence of such content is a reality that we'll have to accept, whether you like it or not.

> while it's an open question

I think theres research that shows that models collapse after being trained on their own content.

https://www.nature.com/articles/s41586-024-07566-y

Re: Big Tech Has Suddenly Flipped on the AI Jobs Wipeout Scenario

#102

Post by intended »

skybrian wrote: post #27
gortok wrote: post #26

Earlier quoted context omitted.

> nowadays AI chatbots and coding agents routinely assume they need to get up-to-date information in other ways, via web searches and other tool calling. So I don’t see accuracy declining at least for programming. How do those chat bots discern that the ‘web searches’ they’re using are returning human generated information only that’s been vetted instead of LLM output?

It’s true that they are only as good as their input data, but the same is true if you do your own web searches.

Not quite.

The difference is in rate of generation.

Today, the ratio of good data to slop has dropped. Previously, you couldn't mass produce websites and have content ready to go at a click.

So you had better hit ratios before, and you have worse hit ratios now. The more labour intensive to create content, you lived in a better world.

Hmm, The better the ratio of creating content to verifying content, the better the environment we live in.

Re: Big Tech Has Suddenly Flipped on the AI Jobs Wipeout Scenario

#103

Post by foldr »

intended wrote: post #101
foldr wrote: post #87

Earlier quoted context omitted.

AI isn't going to go away, and AI-generated content isn't going to go away. So while it's an open question the extent to which AI training will be hampered by the proliferation of AI-generated content, I think that the existence of such content is a reality that we'll have to accept, whether you like it or not.

> while it's an open question I think theres research that shows that models collapse after being trained on their own content. https://www.nature.com/articles/s41586-024-07566-y

Right, but we’re not talking about a worst case scenario where models are trained primarily on their own output. We’re talking about what will happen in the long run, as future models are trained on a realistic mix of content. It’s surely an open question what will happen.

Re: Big Tech Has Suddenly Flipped on the AI Jobs Wipeout Scenario

#104

Post by y0eswddl »

kristiandupont wrote: post #57
gortok wrote: post #11

> The only thing that matters is if LLMs with sufficient scaling can become frontier AI researchers kicking off the exponential. Everything else is transient As long as the term “AI” means by-and-large LLMs with additional features sprinkled on top, the answer is no. More likely (without careful vetting by the folks aggregating these models) is that the quality will go down as more and more AI-generated output gets s…

>LLMs-as-AI can only give us probabilistic outputs based on inputs I am not completely sure what you are saying here, but it sounds like a variation of the "it's just a stochastic parrot" argument, which is reductionist. The human brain is also just a bunch neurons firing.

"i am not sure what you're saying..."

"...but I'm gonna argue against it anyway."

Re: Big Tech Has Suddenly Flipped on the AI Jobs Wipeout Scenario

#105

Post by gortok »

kristiandupont wrote: post #57
gortok wrote: post #11

> The only thing that matters is if LLMs with sufficient scaling can become frontier AI researchers kicking off the exponential. Everything else is transient As long as the term “AI” means by-and-large LLMs with additional features sprinkled on top, the answer is no. More likely (without careful vetting by the folks aggregating these models) is that the quality will go down as more and more AI-generated output gets s…

>LLMs-as-AI can only give us probabilistic outputs based on inputs I am not completely sure what you are saying here, but it sounds like a variation of the "it's just a stochastic parrot" argument, which is reductionist. The human brain is also just a bunch neurons firing.

I’m using the term “stochastic parrot” exactly as the author of the paper did, and incidentally an interview with her was on HN yesterday. https://news.ycombinator.com/item?id=48805401

To wit:

> Another one is that “stochastic parrot” got picked up and interpreted by other people as a minimization or an insult. It was not meant that way. Other people might be using it that way, but that’s not how I intended it, because it’s just a description of what these systems actually are. To see it as an insult requires either the belief that the large language model is the kind of thing that can take offense, which it isn’t, or that these large language models should be understood as steps toward this grand ideal that I don’t hold of artificial intelligence.

> What I have been doing in many places—the octopus thought experiment, stochastic parrots, the phrase “synthetic text-extruding machines”—it’s all about trying to make vivid to people who aren’t in the business of building language technology what these systems actually do, which is not the same thing as insulting the systems or insulting the people who like the systems.

Re: Big Tech Has Suddenly Flipped on the AI Jobs Wipeout Scenario

#106

Post by danaris »

openquery wrote: post #28
gortok wrote: post #11

> The only thing that matters is if LLMs with sufficient scaling can become frontier AI researchers kicking off the exponential. Everything else is transient As long as the term “AI” means by-and-large LLMs with additional features sprinkled on top, the answer is no. More likely (without careful vetting by the folks aggregating these models) is that the quality will go down as more and more AI-generated output gets s…

> More likely (without careful vetting by the folks aggregating these models) is that the quality will go down as more and more AI-generated output gets subsumed into these models. This assumes that there aren't algorithmic breakthroughs which reduce training/inference costs by several OOMs. How much do these models need to do before people throw their hands in the air and say, ok this is happening. The Erdos unit di…

> This assumes that there aren't algorithmic breakthroughs which reduce training/inference costs by several OOMs.

Of course it does.

You cannot operate based on the assumption that you will hit a breakthrough in any given span of time. Breakthroughs are, by their very nature, unpredictable—both on when you will reach them, and on whether they exist to reach at all.

It's also assuming that there isn't a hardware breakthrough that will put us another several OOMs ahead on compute.

It's also assuming that there isn't a military "breakthrough" that leaves all our supply lines either bombed or hostile to us.

When you're forecasting, you can only operate based on what is known and knowable now.

Breakthroughs are unknowable. You cannot rely on them.

Post reply on HN