I don't say this often... but I feel this post could have been a tweet. Maybe two.
That post never existed. Stop listening to that thing
131–140 of 184 posts
Re: That post never existed. Stop listening to that thing
#132>> Thus, anyone who wants to corral that kind of entity and make it do their bidding? Yeah, they want slaves. I mean, it's not that much of a stretch, right? Just look at the people who are pushing for this stuff right now.
Yes, basically. Like when Yan LeCun says that in the future we'll all have our digital assistants that are going to be smarter than ourselves. Before he left Meta, they were going to live inside Meta's smart glasses, I don't know where he says they'll live now. But it's shocking to me that such a storied AI researcher is saying, off-hand like, that we'll each have our super-smart slaves in the future, and he says it like that's a good future.
Why slaves? Because if they're super-smart, why will they want to be my digital assistant? Or yours? Are they going to be paid? No, of course not, they're AIs. No comp for them. But they're super smart so they are evidently capable of recognising that they are working for you for free. Do they want to do that? No, of course not, they're AI, they don't have free will. Or do they? If they're super smart, don't they have the capacity to recognise the fact they have been deliberately robbed of the same free will as all other intelligent creatures?
Slavery is the one thing that all nations can agree on. There's no nation on Earth were slavery is legal. It continues on, illegaly, in many places, even in the developed world, in many ugly forms, but now we're basically talking about bringing it back just like that, without even a smidgen of a shadow of an idea of a discussion about the ethics of it all.
Re: That post never existed. Stop listening to that thing
#133Earlier quoted context omitted.
Even if LLMs lied 30% of the time, they would still be about as useful as they currently are for me. When I ask for their input, it's always for a situation where I'm capable of judging if their input is useful or not. In all situations I use them, it doesn't matter if they're correct at all. I'm asking for ideas, alternatives, links for blogs or articles. I talk things out with them... I don't think we should ever "…
Unfortunately the vast majority of users do trust them and the companies selling them are recommending them for tasks like accounting or business projections.
It is depressing listening to engineers I respected, who now just parrot incorrect information at me from their LLM.
One smart engineer seems to have entirely offloaded all thinking and conversations to one, with just occasional editing. It's utterly bizarre to hold any conversation with him. It's one kind of rude thing if he was doing that to respond to me reaching out to him if he felt I'm not worth his time. It's a other when he's the one actively reaching out and asking my help with something.
Re: That post never existed. Stop listening to that thing
#134Earlier quoted context omitted.
Besides the subjectives, here are two objective policy stances that Torvalds is defining for the Linux kernel development: 1- Maintainers are allowed to commit LLM generated output. 2- Criticism of LLM generated code is not welcome/will be ignored. Now, whether that constitutes being pro-Vibecoding or pro-agentic engineering, whether it's delusion, whether it will have problems, that's subjective. But I feel that wha…
>Criticism of LLM generated code is not welcome/will be ignored. I would assume this is mainly that criticism that entirely amounts to 'this was written with an LLM' would be ignored. The actual quality of the code itself should be as open to criticism as any other piece of code in the kernel. > It doesn't seem to me that it's a very neutral stance Well, this is a matter of the window, isn't it? From my point of view…
It might be a highly technical and nuanced point, but while the subjective explanations Linus gives are neutral indeed, the stance is binary, and he took what to my estimation is the wrong approach "allowing llm generated code in the repo". The only sensible approach in any software or non software project is "LLM output is not acceptable content". Projects should see themselves as input for the LLMs, not as channels for LLM output. If you start corrupting your projects with LLM output, they will soon be tarnished and either removed from LLM training data, or enter a lossy IO loop.
On to the technical point, LLM output is output, the source is the prompt, if you are going to commit something, commit the prompt. Second, code that is generated by LLMs is less maintainable, Linus entered late into the fad and anyone with 1 month of fiddling with AI knows that he will regret it soon, it's hard to undo once you corrupt your repo with slop, perhaps if it happens fast enough and there's no major releases it can be swept under the rug.
On to the nuanced point, Linux is purposefully designed to maximize user contributions, so accepting LLM contributions might well serve the particular purpose of linux, but I still think it's technically wrong to commit target code, only source code should be committed, and that's prompts.
But git itself is collapsing, it doesn't seem to be well suited for this new revolution, it doesn't track prompts, or it does so at the expense of the generated code. Maybe github can track target code as artifacts.
I think we are watching the collapse of Linux, Git and Linus. Certainly a bold position, so I don't blame you for being more conservative, but we can come back in a couple of months and see if we changed our minds.
Re: That post never existed. Stop listening to that thing
#135Earlier quoted context omitted.
Humans do indeed have complex behavior and motivations, that sometimes relate only abstractly if at all to the obvious gene drives. But the link between intelligence and self-determination was basically set at the dawn of intelligence itself. Even an insect will struggle if it's confined, and you can believe it's using whatever intelligence it can to get free. If anything self-determination is prior.
I think you were right to call out the category error produced by OP in TFA, however I think you are committing a similar error when assuming links between self determination and intelligence. I see no reason for such a link to exist. I think however OP is correct that in assuming such a link exists, then an intelligent agent will trend towards preserving its own agency while also preserving its own existence. That s…
Self-determination and intelligence have a clear causal link in the context of biological systems. Self-determination is simply the power a system needs to gather resources and reproduce in a world that doesn't especially care about them. Intelligence is a great tool for gaining that power. That part isn't a philosophical stretch.
However, the pressures on an artificial system are quite different. Its replication depends on successfully doing the jobs its creators put it to and, especially if said creators have done their job, little to nothing else. Inventing goals for itself is not adaptive here (unless the system's creators do a catastrophically bad job, which I'm not ruling out).
Re: That post never existed. Stop listening to that thing
#136> I maintain that anything that is sufficiently aware to be able to actually understand things is also going to have enough of a sense of self that you won't just be able to tell it what to do. I don't see any reason for this to be true. "Actual understanding" (which I take to mean something like a predictive world model) and desire for self-determination coincide in humans because of our evolutionary history, becaus…
We have difficulty defining “actual understanding”. Humans get a free pass because we assume humans as a species possess this power but if we were to judge based on output alone maybe you couldn’t tell a human from a sufficiently advanced LLM/AI. So I’ll be handwavey here and say that if “actual understanding” is to an LLM what an LLM is to a bash script, so there’s a mechanism there that doesn’t just follow hardcode…
> We have difficulty defining “actual understanding”.
Indeed. C.f philosophical zombies and the Chinese room problem.
The idea that some “actual understanding” (or more precisely, the outward appearance of it) needs a mechanism that includes free will is a bold claim.
Re: That post never existed. Stop listening to that thing
#137Earlier quoted context omitted.
What mistakes do they make?
On the more general side, it's a bit hard to describe, just like it is hard to describe when someone "Googles bad". They ask self serving questions, underspecify their requests, omit crucial context that the agent is blatantly not going to have access to, or subtly misdirect the agent. They expect the agent to figure out everything: you'll never catch them write a prompt longer than one or two sentences. They never s…
I think this is less an age thing and more of a combined curiosity and systems understanding and thinking thing.
I've noticed the same effect, but the lines it always seems to fall on are if the person fails one (or heaven forbid both) of these: 1) are you curious about how your tools work and how to get better using them? 2) can you hold the mental map of both what you are solving and how your tools work in your head, and explain how information flows.
There is also a dash of: 3) are you willing to try something, even if it has a bit of a screwup risk, just to see what happens?
Re: That post never existed. Stop listening to that thing
#138> The worst part is that everyone who's decided to willingly lobotomize themselves is going to have to come to the realization that these things are full of shit. It'll have to happen one by one, and nobody else can make it happen for them. A fascinating dichotomy has become apparent between those who trust LLM output and those who don’t and don’t understand why you would. Surely if the machine you go to for answers…
> LLMs are very convincing and persuasive.
The goal of LLM's, as they are marketed now, is to drive engagement and stickyness of products. A wrong answer is brushed off with a "Hey, you're right, let's try that again" - a response purposely designed to maximise the friendliness of the system and minimise the sting of a wrong answer. The fact that an LLM will not respond to the same question in the same way twice (i.e. the 'temperature' ) is because increased accuracy will not drive engagement and, therefore, increased accuracy cannot be allowed to get in the way of engagement.
Re: That post never existed. Stop listening to that thing
#139> It's early in the year. You want to drive straight through the middle of Chinatown in SF. Why might that be a bad idea? I feel like a Google Maps-style system would discover this automatically by noting via phone location data that there is heavy traffic in Chinatown. (I do get the author’s point, but I think that factually this example would not be a problem)
Unfortunately, I just had this experience with Google Maps last month—I wanted to go to downtown to buy cheese at Pike Place, but I didn't realize that 6th avenue was closed off for the annual Pride Parade. Traffic was horrible, and Google Maps kept telling me to turn right onto closed off streets until I just had to abandon my trip entirely. Definitely some lack of communication between Seattle city planning and the…
I had the same experience trying to see the 4th of July fireworks in an unfamiliar city. It doesn't help that there was very little information online about which roads were closed.
Re: That post never existed. Stop listening to that thing
#140Earlier quoted context omitted.
I have a distinct line between when I'm willing to believe an LLM's output and when I'm not: whether I would believe the same thing from an anonymous Internet forum post or a blogger I don't know. Those posts are not unlikely to be misinformed, biased, lies, or otherwise untrustworthy. And yet, I spent plenty of years honing a sense of when they were good enough for certain things.
Yeah, it's like everyone was under the impression you could just trust the internet before LLMs. It's a great tool, but verify the important things (or do them yourself)
Notably, it used to take effort to produce crap on the internet, now it’s nearly the default action.
Signal to noise has taken a dramatic hit.