Are language models deprived of electric sleep? • Hacker News Board

Re: Are language models deprived of electric sleep?

#31

Post by benreesman » Sat, Jul 23, 2022, 3:56 PM UTC

legalcorrection wrote: post #29

benreesman wrote: post #16

Earlier quoted context omitted.

Exactly right. But extremely serious scientists, very smart people, are still drawing epicycles on blackboards studying “consciousness”.

Pretending consciousness doesn't exist or that it has no function I think reflects poorly on someone who studies human behavior.

I’m saying that the idea of a bright line between the emergent behavior of a dolphin and a human is very pre-Copernicus.

Studying, even measuring the capabilities of an animal is science.

Justifying a soul is the purview of spirituality, not science. (Nothing against spirituality, I have a spiritual life, I just don’t confuse it with science).

Re: Are language models deprived of electric sleep?

#32

Post by wodenokoto » Sat, Jul 23, 2022, 3:57 PM UTC

This is the Copenhagen Business School (one of the largest universities in Denmark) library homepage and presumably the library blog. Which makes it weird that the author uses "I", when the author name is not shown.

Re: Are language models deprived of electric sleep?

#33

Post by bitL » Sat, Jul 23, 2022, 3:58 PM UTC

benreesman wrote: post #2

I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet. People read too much into this. The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sen…

It seems like the bigger transformer you have, the less the generated text feels like a speech of somebody with Wernicke's encepalopathy and more like a normal person, but there is still the feeling of some "brain lesion" in there somewhere.

Re: Are language models deprived of electric sleep?

#34

Post by boredemployee » Sat, Jul 23, 2022, 3:59 PM UTC

skybrian wrote: post #25

boredemployee wrote: post #8

Just a side note, I was showing some friends gpt3 and asking some trivial questions, trying to impress them, when one of them asked me: "Is it any different from like asking google for the same information?" I mean, In a pure naive and practical point of view she wasn't that wrong.

Yes, of course it's different. Google can point you to a real Wikipedia article while GPT-3 will generate links to fake ones (or you could ask it to generate a fake article.) It's the difference between truth and fiction. Or at least, a first-level approximation. Wikipedia articles can be wrong.

Well google also can point me to fake links with no real content at all as well. But I get your point.

Re: Are language models deprived of electric sleep?

#35

Post by legalcorrection » Sat, Jul 23, 2022, 4:05 PM UTC

benreesman wrote: post #31

legalcorrection wrote: post #29

Earlier quoted context omitted.

Pretending consciousness doesn't exist or that it has no function I think reflects poorly on someone who studies human behavior.

I’m saying that the idea of a bright line between the emergent behavior of a dolphin and a human is very pre-Copernicus. Studying, even measuring the capabilities of an animal is science. Justifying a soul is the purview of spirituality, not science. (Nothing against spirituality, I have a spiritual life, I just don’t confuse it with science).

I strongly disagree. Pure behaviorism is just willful blindness. Consciousness is a real phenomenon, as any person not philosophically committed to denying its existence can tell you. It's front and center of our experience of human cognition. It would be quite strange for it to serve no function in the human mind.

Yeah, it's hard to quantify and isolate and experiment on, but that just speaks to either current limitations of human science, or possibly to limitations that cannot be surpassed. Given how much mileage certain philosophical movements have gotten out the common intuition that emerged during the Elightenment that everything is scientifically tractable, I understand the resistance to accepting these limitations and opening the door to all of the philosophical consequences of that intuition failing. But sorry, reality doesn't care about your philosophical attachments.

Re: Are language models deprived of electric sleep?

#36

Post by sitkack » Sat, Jul 23, 2022, 4:06 PM UTC

benreesman wrote: post #10

benreesman wrote: post #7

Earlier quoted context omitted.

I tend to agree that the math gets a lot simpler if the sun doesn’t rotate around a mystical axiom that humans are “special”.

High-energy physicists are about as serious as it gets, but they’re still dragging their feet around many worlds because it has uncomfortable implications for consciousness and free will. Homo Sapiens has a serious agenda around being something other than the smartest chimpanzee.

I think of humans as a lens, mirror or piece of turbulence in a high dimensional space that information about the universe flows through. Maybe the smartest chimp, but not any more or less alive than anything else. Somewhat fortunate in the ways that we get to experience existence.

Re: Are language models deprived of electric sleep?

#37

Post by ncallaway » Sat, Jul 23, 2022, 4:09 PM UTC

IdealeZahlen wrote: post #30

benreesman wrote: post #15

Earlier quoted context omitted.

Superposition collapse is impossible to test by definition of not including the observer in the system. If we just admit the possibility that the math was right all along we win, the math works, and we lose “I am a unique, distinct entity.” I’m on team math.

That's math;not physics. You need experiments in physics.

Can you explain the experiments and physics that justifies physicists beliefs in the Copenhagen interpretation of QM?

Re: Are language models deprived of electric sleep?

#38

Post by bitxbitxbitcoin » Sat, Jul 23, 2022, 4:16 PM UTC

Nice title. Many levels to the play on the 1968 “Do Androids Dream of Electric Sheep?”[0]

[0] https://en.m.wikipedia.org/wiki/Do_Androids_Dream_of_Electri...

Re: Are language models deprived of electric sleep?

#39

Post by tempe1h1 » Sat, Jul 23, 2022, 4:16 PM UTC

benreesman wrote: post #15

alar44 wrote: post #12

Earlier quoted context omitted.

No, they're "dragging their feet" because it's impossible to test, it's unfalsifiable.

Superposition collapse is impossible to test by definition of not including the observer in the system. If we just admit the possibility that the math was right all along we win, the math works, and we lose “I am a unique, distinct entity.” I’m on team math.

> we lose “I am a unique, distinct entity.

I agree with this sentiment. quantum mechanics (stemming out of statistics mechanics) start from regarding all particles as indistinguishable.

IMO, this theory is more mathematics than it is physics.

Given a wide enough application, humans end up ourselves as particles, essentially we are commoditifiying our individuality.

Re: Are language models deprived of electric sleep?

#40

Post by Swizec » Sat, Jul 23, 2022, 4:16 PM UTC

benreesman wrote: post #27

skybrian wrote: post #22

Earlier quoted context omitted.

What this experiment shows is that making stuff up off the top of your head, using memory alone, isn't as hard as it looks. But when doing actual research, we check our work against the real world. For example, that's how you get a list of real references rather than fake ones. Suppose we played a guessing game: given a title, does the Wikipedia article exist or not? You could fairly confidently say that "Apple" exis…

I sometimes fine tune new language models on the HN comment corpus for fun. It generates comments arguing that ML models are just parrots.

As a bird owner, they are not “just parrots”. There is clear intent to communicate. Even species who can’t say human words use distinct vocalizations for specific purposes and situations.