Live data from Hacker News

Where does next-token prediction leave us?

pop.rdi.sh

201–210 of 221 posts

Re: Where does next-token prediction leave us?

#201

Earlier quoted context omitted.

So, where does electricity leave us? In a perpetual loop of rent-seeking for something made with humanity’s collective output of centuries. It is not a good place for an individual to be in, regardless of class.

What? People made electricity manually for centuries, collectively? And who were the electricity rent-seekers? Electricity has almost always been a public service.

People also don't make data centers, GPU, or do AI research collectively. The part that is made, collectively, are the layers of knowledge required to provide either.

> Electricity has almost always been a public service.

I have no clue where that would be true, if anywhere. Not where I live. I am paying a private company, and the electricity that they are selling me is generated by other private companies.

Re: Where does next-token prediction leave us?

#202

Earlier quoted context omitted.

> All these promises without even reaching an ideal! Could it simply be that this ideal is becoming impoverished? What ideal? > The prevailing view is that « AI makes us stupid », so it’s actually a good thing if only the wealthy were still capable of using it. You mean it would be a good thing? > I doubt that [..] we will ever fail to overcome entropy What does this mean? > What is given without effort does not tran…

> What ideal? What is ideal for one person may not for another, therefore I cannot define this ideal without making a mistake. But if I had to try, I would say: answers for the scientist, paradise for the believer. As for the impoverished ideal, some of the fundamental goals that serve as a pretext for these technologies have not even been achieved and yet we are already, gradually and methodically, being stripped of…

> What is ideal for one person may not for another, therefore I cannot define this ideal without making a mistake. But if I had to try, I would say: answers for the scientist, paradise for the believer.

Ah I see, I thought you had a specific ideal in mind, but you rather mean having any ideal at all, right? I agree, I'm dismayed how many view any idealism as "immature" or "unrealistic". It's very tragic IMO because we're not so passive because the world would be so hard to make better, the world is so hard to make better because we're so passive, and think so small.

> with those who see only progress or magic in it, nor will we be able to close this Pandora's box. Whether rightly or wrongly.

Hey, didn't you just deplore lack of idealism? Of course we can't reason with them, not with that attitude! But knowing that other humans just want the best for themselves and those they love, surely they shall see the good sense of our arguments, if only we bring them forth with the right cheerful spirit! I'm only half-joking, and of course you also have a point.

If "AI" does real work and gets better and better, they'll price us out, stomp us underfoot like ants. A giant blob of consumers saying "go away, batin!" until it bursts and dries out, leaving nothing but some Moloch monolith. The upside of that would be that we don't have to worry about arguments, or beat ourselves up for not having tried hard enough. We'd be like other early humans species that curiously are no longer around.

BUT if we're right, and it's really just putting everything people made into a blender, to produce a brief powder flash and then confusion and regret, we'd do well to hold on to our ideals, whatever they may be, and to caring and thinking things through as best as we can; in short, to being human. There's a new drug, a lot of people got addicted or kinda lost their minds, but a lot of them will come around, too, and should know they're always welcome back.

> This means that, as far as we know, it is impossible to prevent the universe from deteriorating, aging, or descending into chaos

Okay, so you meant we will never overcome entropy, not, as you originally wrote, that we will never fail to overcome it. You can see how that would be confusing.

> decay and disorganization are fundamental laws. Perfection is not of this world, and thankfully so, for beings as imperfect as we are would quickly corrupt it.

Not that I would call us or anything perfect (gotta keep it around as an ideal :), but that's what life does, struggle against entropy, introduce and proliferate some order, for a little while. The idea that with just enough power and/or knowledge we can somehow make a leap into a different category (one that lasts) is probably just wishful thinking, if there's even any thought involved, so the best we can do is try to be excellent, and excellent to each other.

(At least, that'd be my ideal, someone else might say we must fight as brutally as we can so at the end of time, some super stronk warrior gets to turn out the light, but it's so easy to show such things up as stupid so that's what we should keep doing)

Re: Where does next-token prediction leave us?

#203

Earlier quoted context omitted.

> What ideal? What is ideal for one person may not for another, therefore I cannot define this ideal without making a mistake. But if I had to try, I would say: answers for the scientist, paradise for the believer. As for the impoverished ideal, some of the fundamental goals that serve as a pretext for these technologies have not even been achieved and yet we are already, gradually and methodically, being stripped of…

> What is ideal for one person may not for another, therefore I cannot define this ideal without making a mistake. But if I had to try, I would say: answers for the scientist, paradise for the believer. Ah I see, I thought you had a specific ideal in mind, but you rather mean having any ideal at all, right? I agree, I'm dismayed how many view any idealism as "immature" or "unrealistic". It's very tragic IMO because w…

[dead]

Re: Where does next-token prediction leave us?

#204

Earlier quoted context omitted.

One would hope progress directly benefits more than the tech CEOs and investors. They're the ones going around heralding fully automated societies and god-level AIs. Open minded people also recognize the dangers of technological disruption. Let us not forget how social media went from being viewed as utopian to dystopian in a decade.

Social media offered no value. If LLMs get about 5x better at what they do they offer a lot in terms of medical discoveries, governance, dirt cheap labor, fully automated raw material to end product pipelines, automated farming and the list goes on and on.

Dirt cheap labor doesn't sound good for the working class. At best they get a livable UBI, while the capital class becomes even richer and more powerful.

I don't know where you're getting governance and manufacturing stuff that sounds like advanced robotics out of LLMs.

Re: Where does next-token prediction leave us?

#205
post #200

Earlier quoted context omitted.

I see little evidence that any social role reversal is going to occur. This technology is soon going to price out average to poor people when they have to pay true token costs. It might be the case that only the rich and powerful have access to the powerful models.

Poor people with hard-to-automate occupations don't necessarily need to be able to afford token costs for social role reversal to occur. They only need to be able to hire an even poorer person who used to earn a salary that exceeded even those token costs and who was laid off as a result.

Um, what poor person is hiring ANYONE? If you're able to hire a software dev, even a laid off one, you are not poor.

Re: Where does next-token prediction leave us?

#206

Earlier quoted context omitted.

That’s exactly part of the feelings I have. I always loved to learn, dig subjects, debug, create. Now I feel something has been taken away and has no value. I feel indolence and apathy. In my company the CEO explicitly says that code quality does not matter. He doesn’t care as long as we ship fast and iterate. I am genuinely sad and feel I’m losing something and if I do everything like I used to do, I am pressured th…

> Now I feel something has been taken away and has no value. Did you feel similar when Wikipedia was created or Google or when you first got access to the internet? All of these tools have made digging into subjects and learning easier. I find the same with AI. I love it when a random thought pops into my head and I can explore it with an AI such as Gemini. Then ask it for the sources it used so that I can read furth…

> AI is just a tool

You are forgetting that AI can also use tools.

And those AIs can be used as tools as well by other AIs.

The moment you start working on something, someone else is already automating the exact thing you are doing now.

Re: Where does next-token prediction leave us?

#207

Earlier quoted context omitted.

I recently at around 42yo learned how to darn/mend my socks. My mom taught me how to do it. I have enough money that I could just throw my socks with holes to throw trash and buy new ones inconsequentialy, but mending them by hand gives me something, it is kind of therapeutic and a sense of accomplishment. Im sure there are machines that could do it in a second, or a patch I could stick on it as well. Point being tha…

Do you use a washing machine or do you manually wash your clothes in a stream? Much of modern life is automating away the boring useless bits. AI, on the other hand, changes the conversation on what’s boring or useless.

>Much of modern life is automating away the boring useless bits.

>AI, on the other hand, changes the conversation on what’s boring or useless.

A compression algorithm finds and removes redundancy. Simple automation is like simple compression algorithms (RLE like). AI (which even internally - at least the encoding to the representation parts - looks like compression) just like a much more sophisticated compression algorithm which finds and removes redundancy where we thought the creativity and originality were.

It does look like our civilization has accumulated a lot of cruft masquerading as creative/original/intellectual activities ("Bullshit jobs" comes to mind, also all that talk about stagnation in science, and all those huge collective collaborations - where collective there is a stagnation - and now with AI individual scientists will again be able to wield all the bleeding edge across the wide fields while digging deep in desired target research direction), and the AI is the vacuum cleaner for that cruft.

Re: Where does next-token prediction leave us?

#208

Earlier quoted context omitted.

ChatGPT is probably less useful than youtube for this kind of stuff.

Might help to find the right video though

These days I find Gemini often recommends me a youtube video that's just an AI voice reading out a reddit post that was chat gpt generated full of emoji.

Re: Where does next-token prediction leave us?

#209
post #31

Earlier quoted context omitted.

Were the vaccines accessible because of the sympathy towards the people at the bottom, or so that they can go back to work as soon as possible? Medicine is subsidised when there is a global pandemic, but not when individual lives are at stake.

I don’t really find it fruitful to speculate about motives in complex systems. The claim was that average people don’t benefit from medical advances, which seems clearly untrue. Now, if we’re shifting to why advances are often generally available, sure, there are lots of plausible theories that may or may be true, in whole or part, and which we’ll never prove one way or the other. Believe what makes you happy, or ang…

> Now, if we’re shifting to why advances are often generally available

But it's not me who is shifting, right? The claim was:

> Science and technology, I feel, has always had a certain apathy towards the plight [of] the people at the bottom rungs.

not

> The claim was that average people don’t benefit from medical advances

My point is to reject the claim that the high availability of vaccines is a counter-example. It's a political decision, not a science/technology related one.

Re: Where does next-token prediction leave us?

#210
post #200

Earlier quoted context omitted.

Poor people with hard-to-automate occupations don't necessarily need to be able to afford token costs for social role reversal to occur. They only need to be able to hire an even poorer person who used to earn a salary that exceeded even those token costs and who was laid off as a result.

Um, what poor person is hiring ANYONE? If you're able to hire a software dev, even a laid off one, you are not poor.

After going without food for a few days, even a software dev is going to have to swallow their pride and consider alternative employment options. You might think they could just take a poor person's existing job, but why would for example a meatpacking plant hire a software dev with zero meatpacking experience when they already have lots of experienced meatpackers and meat demand hasn't gone up? Meanwhile, the meatpackers might like it if they could have someone babysit their children, cook for them and clean their now much larger house (which used to belong to a software dev who fell on hard times and had to move out). And a steady supply of desolate characters holding up cardboard signs saying "I used to be a software dev, but now I would do literally anything for a meal" could put such luxury into their affordable range.
Post reply on HN