Live data from Hacker News

Where does next-token prediction leave us?

pop.rdi.sh

181–190 of 221 posts

Re: Where does next-token prediction leave us?

#181
post #39
post #25

Earlier quoted context omitted.

How accessible were covid vaccines?

It costs between 4k and 10k if you break your leg and call an ambulance in the US. Forget medical advances, it costs you a car to get a cast

Not if you're homeless in, say, San Francisco.

Re: Where does next-token prediction leave us?

#182

I have a more depressing theory. I think class is part of it, but I'm starting to suspect that a shockingly large number of people lack the critical thinking skills needed to think out the implications of this stuff. I say that because I've met so many people of the managerial class that seem to think it's going to replace the annoying people they have to pay, but somehow they don't think it's going to come for them…

[flagged]

"want to see where it goes". Ok, what if it goes somewhere awful? A "paradigm shift" could easily just mean "corporate feudalism". It's not "small minded" to consider that not all change is good, and skepticism isn't a bad personality trait.

Re: Where does next-token prediction leave us?

#183
post #127

The title is a little misleading as it sounds like it might hint at the end state of next token prediction - i.e. will we reach AGI with it and so on. Instead, this is entirely about "means of production" and "class war" stuff. While that may be coming, it isn't clear that this is the most likely outcome, its just the most pessimistic one. The author might as well have included that it will "be the end of the human r…

Would you like to elaborate on why you think the outcome described isn't among the more likely?

It requires a nascent analogous architecture to biology (neural nets) and a discovery (next token prediction) to exactly match evolution. I think that is unlikely. Instead it will be a tool that is capable of doing certain things that machines were not capable of previously.

Re: Where does next-token prediction leave us?

#184

Earlier quoted context omitted.

That’s exactly part of the feelings I have. I always loved to learn, dig subjects, debug, create. Now I feel something has been taken away and has no value. I feel indolence and apathy. In my company the CEO explicitly says that code quality does not matter. He doesn’t care as long as we ship fast and iterate. I am genuinely sad and feel I’m losing something and if I do everything like I used to do, I am pressured th…

I recently at around 42yo learned how to darn/mend my socks. My mom taught me how to do it. I have enough money that I could just throw my socks with holes to throw trash and buy new ones inconsequentialy, but mending them by hand gives me something, it is kind of therapeutic and a sense of accomplishment. Im sure there are machines that could do it in a second, or a patch I could stick on it as well. Point being tha…

Do you use a washing machine or do you manually wash your clothes in a stream?

Much of modern life is automating away the boring useless bits.

AI, on the other hand, changes the conversation on what’s boring or useless.

Re: Where does next-token prediction leave us?

#185

Earlier quoted context omitted.

One would hope progress directly benefits more than the tech CEOs and investors. They're the ones going around heralding fully automated societies and god-level AIs. Open minded people also recognize the dangers of technological disruption. Let us not forget how social media went from being viewed as utopian to dystopian in a decade.

Social media offered no value. If LLMs get about 5x better at what they do they offer a lot in terms of medical discoveries, governance, dirt cheap labor, fully automated raw material to end product pipelines, automated farming and the list goes on and on.

You should read a book on how LLMs actually work on a technical level. Saying they just need to get "5x better" and suddenly they're performing miracles is just nonsense hyperbole

Re: Where does next-token prediction leave us?

#186
post #136

Earlier quoted context omitted.

This is exactly the missing long-term perspective. All AI creators are focused either on the projected wins or the shiny technology, ignoring societal effects. Not that they should, they are engineers or MBAs after all, but somebody should. Somebody like us, or better somebody with better reach and better knowledge, should figure out a way to offer a future also to former clerks.

I don't get how this didn't become yet a major election topic in most democracies. It's business as usual mostly.

Because humans are bad at reacting to things that might happen; we’re far better once there is a literal crisis.

Re: Where does next-token prediction leave us?

#187
post #83

Earlier quoted context omitted.

Or average joe becomes DIY plumber with help of chatGPT and there is a lot less demand for plumbers then.

How does average joe afford the tools and parts they need with no job employing them?

Where I live, just the call out fee for a plumber would cover the cost of tools and materials with enough left over for beers afterwards. If you’re relatively handy and willing to thing things through, the main obstacle to doing all your own plumbing (again, at least where I live) is regulatory capture.

Re: Where does next-token prediction leave us?

#188

Earlier quoted context omitted.

No they do not - sheltering make sense only if there is anything worth waiting for on the other side.

Yeah, seems like a big downgrade to go from being a billionaire in society to a nobody in a bunker with no society waiting for you outside the bunker.

Oh imagine some society will be outside the bunker, waiting.

Re: Where does next-token prediction leave us?

#189
post #69

I have a more depressing theory. I think class is part of it, but I'm starting to suspect that a shockingly large number of people lack the critical thinking skills needed to think out the implications of this stuff. I say that because I've met so many people of the managerial class that seem to think it's going to replace the annoying people they have to pay, but somehow they don't think it's going to come for them…

I don't think its entirely class. I've met many people who have convinced themselves they have some special sauce--even in software! And even those who are insulated like doctors and plumbers don't realize they're still at risk from second order effects. What happens when half of their customers are broke? That possibility is not an exaggeration. Even if everyone re-skills over night, people will default on their mor…

On the flip side, people think AI is magic: too much magic.

Even with large advances in reasoning and actionability, it still makes insane mistakes and it's wildly prone to prompt attacks. That is a fundamental flaw in the technology.

Even an AGI will lack the millennia of self-preservation evolution that will cause it to do crazy shit. So much of what humans do is governed by a desire to be alive and to be liked.

So, I'm not particularly worried about these changes happening rapid fire. I'm more concerned about the ladder pulling effect and misinformation.

Re: Where does next-token prediction leave us?

#190

LLMs reveal an important truth that we were already deeply aware of. With this cognitive offloading, the interests that drive us lose their meaning. If friction disappears, there is no longer any desire, pleasure, longing, obstacle, or demand. It is indolence, and it goes by the name of apathy. All these promises without even reaching an ideal! Could it simply be that this ideal is becoming impoverished? The prevaili…

> All these promises without even reaching an ideal! Could it simply be that this ideal is becoming impoverished? What ideal? > The prevailing view is that « AI makes us stupid », so it’s actually a good thing if only the wealthy were still capable of using it. You mean it would be a good thing? > I doubt that [..] we will ever fail to overcome entropy What does this mean? > What is given without effort does not tran…

> What ideal?

What is ideal for one person may not for another, therefore I cannot define this ideal without making a mistake. But if I had to try, I would say: answers for the scientist, paradise for the believer. As for the impoverished ideal, some of the fundamental goals that serve as a pretext for these technologies have not even been achieved and yet we are already, gradually and methodically, being stripped of using them.

> You mean it would be a good thing?

It would be a good thing, yes. Not the most desirable option, to be sure. But proportionally less harmful to all those who would then be forced to do without it and adapt without necessarily conforming. I admit it's a divisive issue and that it's unlikely to happen that way. But we won't be able to reason with those who see only progress or magic in it, nor will we be able to close this Pandora's box. Whether rightly or wrongly.

> What does this mean?

This means that, as far as we know, it is impossible to prevent the universe from deteriorating, aging, or descending into chaos, because decay and disorganization are fundamental laws. Perfection is not of this world, and thankfully so, for beings as imperfect as we are would quickly corrupt it.

> the best things in life are free.

That’s an interesting thought experiment, indeed. There are indeed things that are given to us, and for which we can be grateful. Antoine Lavoisier said, “Nothing is lost, nothing is created; everything is transformed.” That’s kind of the problem with quotes, they sound nice but don’t always take the complexity of the world into account.

It was a pleasure exchanging with you.

Post reply on HN