If AI scaling is to be shut down, let it be for a coherent reason • Hacker News Board

Re: If AI scaling is to be shut down, let it be for a coherent reason

#391

Post by [deleted] » Fri, Mar 31, 2023, 8:44 PM UTC

realce wrote: post #77

Godel_unicode wrote: post #48

Earlier quoted context omitted.

Huh? How does someone else’s possession of a LLM threaten your personal freedom? How does training your own counteract that? They’re not Pokémon…

It's not someone else's possession - it's someone else's proprietary possession. A government's proprietary possession of computational power is analogous to it having weapons barred from public ownership, meaning they control a vector of violence than could be used against their citizens without counter-balance. If someone else has the ability to weaponize information against you, your ability to understand reality…

[deleted]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#392

Post by _4bjt » Fri, Mar 31, 2023, 8:53 PM UTC

ur-whale wrote: post #195

prompt: what is the third word in this sentence gpt4: third why are we afraid of this thing exactly? sure, it will improve, especially with the plugin story, but the fundamental shortcomings that underpins how it actually works won't go away anytime soon. Many people are spooked, because for the first time ever a computer can somewhat coherently understand and output natural language. As Stephen Wolfram pointed out,…

ChatGPT-4 answered correctly when I entered your prompt.

---

Prompt: What is the third word in this sentence?

GPT-4: In the question you provided, "What is the third word in this sentence?", the third word is "the".

Re: If AI scaling is to be shut down, let it be for a coherent reason

#393

Post by lukev » Fri, Mar 31, 2023, 8:59 PM UTC

famouswaffles wrote: post #374

lukev wrote: post #333

Earlier quoted context omitted.

We apparently have misaligned understandings of what we mean by "how they work." I agree, we don't know how to interpret the weight structure that the model learns during training. But we do know exactly what happens mechanically during training and inference; what gets multiplied by what, what the inputs and outputs are, how data moves around the system. These are not some mysterious agents that could theoretically…

It is all things considered pretty easy to set up GPT such that it runs on its own input forever while being able to interact with users/other systems. add an inner monologue/react and reflexion and you have a very powerful system. embody it with some physical locomotive machine and oh boy. no one has really put this all together yet but everything i've said has been done to some degree. The individual pieces are her…

I agree that LLMs could be one module in a future AGI system.

I disagree that LLMs are good at simulation. They're good at prediction. They can only simulate to the degree that the thing they're simulating is present in their training data.

Also, if you were trying to build an AGI, why would you NOT run it slowly at first so you could preserve and observe the logs? And if you wanted to build it to run full speed, why would you not build other single-purpose dumber AIs to watch it in case its thought stream diverged from expected behavior?

There's a lot of escape hatches here.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#394

Post by nradov » Fri, Mar 31, 2023, 9:02 PM UTC

1attice wrote: post #388

nradov wrote: post #335

Earlier quoted context omitted.

The mRNA vaccines were a wonderful innovation but the scientific data clearly shows that even before vaccines arrived the risk to me (and the vast majority of other people) was always minor. There was certainly never any valid justification for the cruel and destructive policies imposed by irrational authoritarians such as lockdowns, school closures, and mandates. https://nypost.com/2023/02/27/10-myths-told-by-covid-…

I don't dispute your broader point about humans and novel risk, I dispute that COVID is a valid example of this. In fact, I rather think we didn't react swiftly or strongly enough. Masks, in particular, should have been mandated (at specific levels of quality, such as N95 or, failing that, KN95) and distributed by the state. https://www.twincities.com/2023/03/16/zeynep-tufekci-why-the... There was an era wherein libe…

The actual science never supported mask mandates.

https://doi.org/10.1002/14651858.CD006207.pub6

When you don't know the proper course of action it's better to gather more data instead of taking swift but wrong actions.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#395

Post by bennysonething » Fri, Mar 31, 2023, 9:14 PM UTC

I get the feeling that this is people over hyping their field to boost their own status. It's amazing technology, but I doubt there's any emergency here.

In another way this reminds me of Roark's court room speech in the fountainhead

"Thousands of years ago, the first man discovered how to make fire. He was probably burned at the stake he had taught his brothers to light. He was considered an evildoer who had dealt with a demon mankind dreaded. But thereafter men had fire to keep them warm, to cook their food, to light their caves. He had left them a gift they had not conceived and he had lifted darkness off the earth. Centuries later, the first man invented the wheel. He was probably torn on the rack he had taught his brothers to build. He was considered a transgressor who ventured into forbidden territory. But thereafter, men could travel past any horizon. He had left them a gift they had not conceived and he had opened the roads of the world. "

Re: If AI scaling is to be shut down, let it be for a coherent reason

#396

Post by bick_nyers » Fri, Mar 31, 2023, 9:17 PM UTC

benlivengood wrote: post #343

echelon wrote: post #320

Earlier quoted context omitted.

Why do these arguments not tell us how it will happen? Show us the steps the AI will take to turn the earth into a playground. Give us a plausible play by play so that we might know what to look for. Does it gain access to nukes? How does it keep the power on? How does it mine for coal? How does it break into these systems? How do we not notice an AI taking even one step towards that end? Has ChatGPT started to fiddl…

Yudkowski's default plausible story is that the slightly superhuman AI understands physics well enough to design sufficient nanotechnology for self-realization and bootstrap it from existing biochemistry. It uses the Internet to contact people who are willing (maybe it just runs phishing scams to steal money to pay them off) to order genetically engineered organisms from existing biotech labs that when combined with…

The idea that Super Intelligence wouldn't even think a thought until it has the ability to execute that thought at a specified capability is very interesting.

One interpretation I have is that it can think ideas/strategy in the shadows, exploiting specific properties about how ideas interact with each other to think about something via proxy. Similar to the Homicidal Chauffer problem, which pits a driver trying to run a person over as a proxy for missile defense applications.

The other interpretation is much more mind-boggling, that it somehow doesn't need to model/simulate a future state in its thinking whatsoever.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#397

Post by gspencley » Fri, Mar 31, 2023, 9:25 PM UTC

JohnFen wrote: post #353

gspencley wrote: post #219

Earlier quoted context omitted.

> It matters a lot to me because the whole point of commenting here (or anywhere) is to talk to other humans, not just to talk to myself. That's fair, but one of my points was that even prior to ChatGPT the ability existed for you to be "sucked into" a relationship with another under false pretenses. LLMs might make it easier to put this sort of thing on "autopilot", but if what you seek is a guarantee of talking to…

> one of my points was that even prior to ChatGPT the ability existed for you to be "sucked into" a relationship with another under false pretenses That's true, of course, but it's still interacting with a real human being. An adverse interaction, but at least a human. > I don't see how, in a post-LLM-world, that can't be done. I don't see how it can be done without losing much of the value of online interactions. >…

> That's true, of course, but it's still interacting with a real human being. An adverse interaction, but at least a human.

Actually, not entirely. Some of the stories that really made me raise an eyebrow were people who claimed that they were video-chatting with "the girl." An important piece of context is that these men reached out to me because they found pictures of the woman they believed they were in a relationship with on my website. They wanted to know if the woman was employed by me or if we could verify certain details about her to try and make sense of what they had gone through.

Of course there were people driving this interaction. But a video chat? Obviously it was faked. What I think that AI advancement is going to allow these scammers to do in the future is possibly have extremely convincing voice chats, because when I probed about these video chat claims often times the scammers would have excuses about the microphone not working etc. so they were clearly just feeding pre-recorded video.

Anyway I've gotten the sense by your reply that you are under the impression that we are having some sort of debate or argument. I'm just making conversation and sharing my point of view and experiences. In my opinion I'm not sure the Internet "should" be anything in particular.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#398

Post by CatWChainsaw » Fri, Mar 31, 2023, 9:35 PM UTC

JohnFen wrote: post #389

CatWChainsaw wrote: post #385

Earlier quoted context omitted.

I begin to feel this way now. The whole drive is to make something that can do everything humans can, but better. For vague reasons. To create some sort of utopia. To upload our minds and go to the stars and live forever. We're unlikely to get human utopia or transhumanism, but we are likely to get extremely competent NAIs. Maybe they can't be stapled together as a GAI and that's a limit we reach, but it means that w…

> It's very nihilistic and depressing. Indeed. The common reactions here to people who are scared of what LLMs might bring have gone far to increase my worries. An extreme lack of empathy and even expressions of outright contempt for people is very common from those who are enthusiastic about this technology. Instead of scorn, anger, and mocking, people who think that LLMs are a great thing should be working on actua…

The contempt is another symptom of what social media shoved into overdrive (extreme polarization). Hatred has become easier than empathy. But it also reads like a very different sort of doomsday cult that worships a borg, only instead of the cult being the weird fringe movement, they're the ones driving the bus.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#399

Post by rhn_mk1 » Fri, Mar 31, 2023, 9:47 PM UTC

lukev wrote: post #322

rhn_mk1 wrote: post #198

Earlier quoted context omitted.

You're the one who put state in the conversation. State is part of the whole, and not the whole. It's not enough to understand state if you want to understand why they work. I feel like you're trying to muddy the waters by redirecting the problem to be about the state - it isn't.

On one hand I hate to belabor this point, but on the other I think it's actually super important. Both of things things are true: 1. The relationships between parameter weights are mysterious, non-evident, and we don't know precisely why it is so effective at token generation. 2. An agent built on top of a LLM cannot have any thought, intent, consideration, agenda, or idea that is not readable in plain english. Becau…

I'm not going to argue whether that's correct or not. In the end, adding state to a LLM is trivial. Bing chat has enough state to converse without forgetting the context. Google put an LLM on a physical robot, which has state even if narrowly understood as the position in space. Go further and you might realize that we have systems with part LLM, part other state (LLM + a stateful human on the other side of the chat).

So we have ever-more-powerful seemingly-intelligent LLMs, attached to state with no obvious limit to the growth of either. I don't see why in the extreme this shouldn't extrapolate to godlike intelligence, even with the state caveat.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#400

Post by hollerith » Fri, Mar 31, 2023, 9:51 PM UTC

carapace wrote: post #345

kypro wrote: post #253

My whole life I have been terrified of AI. It all started with watching the Iron Giant when I was 10, and realising, holy crap, we could probably build a crazy monster like this in the near-future. Obviously as I got older and went on to learn about AI and neural nets in university my opinions on this subject matured. Instead of worrying about killer robots my thoughts became more nuanced, however what didn't change…

There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. GAI will feed on information. Humans are the densest source of information in the known Universe. (The rest of the Universe is a poem in physics, in information space humans shine as the sole star in a dark universe.) Ergo, from first principles, hyper-intelligent GAI…

>There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist.

We are made of atoms that the GAI can use for something else. Ditto our farms and cities. Moreover, unlike most of the atoms in the universe, the GAI doesn't have to spend time traveling to get to us. If we could arrange for the GAI to appear in some distant galaxy, then yeah, by the time it gets to us, it'd already have control over so many atom that it might just leave us alone (because we are so different from most of the atoms).

The GAI will know about the prime directive because it will have been trained on internet conversations about the prime directive, but there is no particular reason to hope that exposure to moral arguments will alter the GAI's objectives similar to what tends to happen with young human beings: instead it will have whatever objectives its creator gave it, which (given the deplorable state of most AI research) is unlikely to be the objectives that its creator thought it was giving it. (By "creator" I mean of course a team of human researchers.)

Your poetical imagery might make you feel better, but won't save us.

>Humans are the densest source of information in the known Universe.

You feel that way about humans because evolution made you that way. It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done.