Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where does next-token prediction leave us?

pop.rdi.sh

Post reply on HN
Print view View original

191–200 of 221 posts

Re: Where does next-token prediction leave us?

#191

Post by dyauspitr »

overgard wrote: post #185
dyauspitr wrote: post #72

Earlier quoted context omitted.

Social media offered no value. If LLMs get about 5x better at what they do they offer a lot in terms of medical discoveries, governance, dirt cheap labor, fully automated raw material to end product pipelines, automated farming and the list goes on and on.

You should read a book on how LLMs actually work on a technical level. Saying they just need to get "5x better" and suddenly they're performing miracles is just nonsense hyperbole

I’m very aware of how they work and pretty much everything they do is emergent.

Re: Where does next-token prediction leave us?

#192

Post by giardini »

xtracto wrote: post #169
shinycode wrote: post #114

Earlier quoted context omitted.

That’s exactly part of the feelings I have. I always loved to learn, dig subjects, debug, create. Now I feel something has been taken away and has no value. I feel indolence and apathy. In my company the CEO explicitly says that code quality does not matter. He doesn’t care as long as we ship fast and iterate. I am genuinely sad and feel I’m losing something and if I do everything like I used to do, I am pressured th…

I recently at around 42yo learned how to darn/mend my socks. My mom taught me how to do it. I have enough money that I could just throw my socks with holes to throw trash and buy new ones inconsequentialy, but mending them by hand gives me something, it is kind of therapeutic and a sense of accomplishment. Im sure there are machines that could do it in a second, or a patch I could stick on it as well. Point being tha…

I too enjoy mending my socks! I wear mostly cotton athletic socks, so sewing them up is enough. I can't darn worth a darn.

Patches I tried didn't last and I haven't found a machine that works. And it feels bad to buy a new pair of socks only b/c of a hole in the heel.

Re: Where does next-token prediction leave us?

#193

Post by thrance »

red75prime wrote: post #132
overgard wrote: post #40

I have a more depressing theory. I think class is part of it, but I'm starting to suspect that a shockingly large number of people lack the critical thinking skills needed to think out the implications of this stuff. I say that because I've met so many people of the managerial class that seem to think it's going to replace the annoying people they have to pay, but somehow they don't think it's going to come for them…

Correct me if I wrong, up to 40% of Roman citizens were on "welfare". This, at least, shows that it's not impossible to have a functioning society with a large percentage of people who only contribute political activity.

These people weren't completely idle for one, and Rome was subsidized by the rest of the Empire, from which it extracted wealth and resources.

Re: Where does next-token prediction leave us?

#194

Post by thrance »

overgard wrote: post #40

I have a more depressing theory. I think class is part of it, but I'm starting to suspect that a shockingly large number of people lack the critical thinking skills needed to think out the implications of this stuff. I say that because I've met so many people of the managerial class that seem to think it's going to replace the annoying people they have to pay, but somehow they don't think it's going to come for them…

I believe you're simply (correctly) identifying the lack of class consciousness that is rotting our civilization. If everyone believes they are smarter than everyone else, and will be able to come up ahead in the rat race, then there's no one left to organize and take the collective actions needed to smoothen the AI transition.

Re: Where does next-token prediction leave us?

#195

Post by jononor »

imtringued wrote: post #167
altmanaltman wrote: post #32

Earlier quoted context omitted.

Coding was always "democratized". You just had to put in the effort of learning and understanding. Finding free resources is very easy. AI doesn't make that learning and understanding easy but just allows people to skip it. That doesn't mean democratization at all.

>You just had to put in the effort of learning and understanding. Yeah it's always so weird when people anthropomorphize the harsh physical reality of learning a skill that is accessible to everyone as some sort of secret boys club, where the existing club members do not let you in, when the only difference between the members and outsiders is whether they've paid the entry fee or not.

A 10'000 hour entry fee does rule out a fair bunch of people though, in practice. While there are few artificial barriers to learning to code, there still are some natural ones, like time.

Re: Where does next-token prediction leave us?

#196

Post by thrance »

seanmcdirmid wrote: post #10
mxkopy wrote: post #4

Earlier quoted context omitted.

The attitudes aren’t 1-1 comparable. China is on a winning streak in terms of socioeconomic development, and AI is likely seen as merely a new technology in the context of the social contract. The US is going the opposite way, and people here view AI through the lens of oligarchy more often than not. I wouldn’t say that a lot of people feel as optimistic, even if they are actually more economically secure.

Do you have direct experience with this? From what I understand China has huge youth employment issues right now, and the 35 and out (at even non tech companies) meme has some basis in reality. China historically has had a poor social safety net, but made up for it with a more dynamic labor market (well, we could say the same about the USA vs Europe).

China is still riding the high of its economic miracle, that lifted hundreds of millions out of poverty, which is why the population is so complacent toward their authoritarian government. For now, at least.

Re: Where does next-token prediction leave us?

#197

Post by thrance »

jstummbillig wrote: post #180
customguy wrote: post #162

Earlier quoted context omitted.

In what way is it similar?

So, where does electricity leave us? In a perpetual loop of rent-seeking for something made with humanity’s collective output of centuries. It is not a good place for an individual to be in, regardless of class.

What? People made electricity manually for centuries, collectively? And who were the electricity rent-seekers? Electricity has almost always been a public service.

Re: Where does next-token prediction leave us?

#198

Post by seanmcdirmid »

thrance wrote: post #196
seanmcdirmid wrote: post #10

Earlier quoted context omitted.

Do you have direct experience with this? From what I understand China has huge youth employment issues right now, and the 35 and out (at even non tech companies) meme has some basis in reality. China historically has had a poor social safety net, but made up for it with a more dynamic labor market (well, we could say the same about the USA vs Europe).

China is still riding the high of its economic miracle, that lifted hundreds of millions out of poverty, which is why the population is so complacent toward their authoritarian government. For now, at least.

That's a pretty generic answer that would have been correct 10 or 20 years ago as well. But where are the recent trends in youth employment? Is automation reducing the need for workers going to cause (or is causing) social instability?

I saw a tiktok a few days ago about the proliferation of autonomous food delivery taking away jobs from delivery bike riders (the ones pretty ubiquitous in China today), and this kind of job was sort of considered a safety net if you lost your normal job (in that you can can always deliver food). It was Chinese authored and I doubt anyone who has never been to China would have understood it anyways.

Re: Where does next-token prediction leave us?

#199

Post by red75prime »

killingtime74 wrote: post #155
red75prime wrote: post #132

Earlier quoted context omitted.

Correct me if I wrong, up to 40% of Roman citizens were on "welfare". This, at least, shows that it's not impossible to have a functioning society with a large percentage of people who only contribute political activity.

You mean they were slaves right

Nah. It was welfare for Roman citizens: cura annonae.

Re: Where does next-token prediction leave us?

#200

Post by yorwba »

overgard wrote: post #177
yorwba wrote: post #126

Earlier quoted context omitted.

Yes, widespread automation of knowledge work is unlikely to decrease total production, so all the goods and services people currently demand will still be provided, but the power dynamic of who is consuming and who is providing might flip around. So a bunch of formerly upwardly-mobile people could end up at the bottom of the social hierarchy while others whom they used to look down upon will be able to afford servant…

I see little evidence that any social role reversal is going to occur. This technology is soon going to price out average to poor people when they have to pay true token costs. It might be the case that only the rich and powerful have access to the powerful models.

Poor people with hard-to-automate occupations don't necessarily need to be able to afford token costs for social role reversal to occur. They only need to be able to hire an even poorer person who used to earn a salary that exceeded even those token costs and who was laid off as a result.
Post reply on HN