Why This AI Moment May Be the Real Deal • Hacker News Board

Re: Why This AI Moment May Be the Real Deal

#201

Post by YeGoblynQueenne » Tue, Aug 01, 2023, 9:40 PM UTC

>> 2. It can understand natural language.

That entire section is one big confused contradiction. The author argues that CYC (Doug Lenat's project to hand-code a gigantic database of logic rules encoding common sense knowledge) "can do deep reasoning" because it can answer this question correctly:

  How tall was the president when JFK was born?

Unlike Google and WolframAlpha who can only find the closest association (the elevation of the town were JFK was born). Still, the author says, CYC would never "challenge humanity’s unique rational status, any more than computers that could solve equations did" because it doesn't understand natural language, only logic formulae. And yet, the author seems to be arguing that ChatGPT does understand natural language because... it can answer the same question as CYC; albeit in natural language, unlike CYC.

I think the author has confused the ability to return results in natural language with the ability to understand natural language. By that token, ELIZA (mentioned at the start of the article) must have also been able to understand natural language, even if it didn't have CYC's ability for "deep reasoning", just because it could parrot its user's input in natural language.

That doesn't make sense. Understanding is clearly something that needs to happen before utterances are formed. Just looking at the output of a system doesn't tell you anything about the internal workings of the system, that's an error that many people keep making in this entire discussion about "AI"s. Look at the sky: the sun looks like it's turning around the Earth. Well, we know it isn't. What we see on the surface is rarely enough to explain what is going on "inside" (or outside, as the case may be, for the sun).

Re: Why This AI Moment May Be the Real Deal

#202

Post by notahacker » Tue, Aug 01, 2023, 9:43 PM UTC

mhink wrote: post #168

notahacker wrote: post #107

Earlier quoted context omitted.

Yes, you're misreading it. "the president" is not a label that applies solely to JFK, or one that applies at all to JFK in the context "when JFK was born". If someone gave me the question "who was the president when JFK was born?", the answer is clearly not JFK, so it isn't correct to infer that JFK is the president whose height, party affiliation or essay writing skill is being asked about "when JFK was born". If so…

> If someone gave me the headline "was the president strong at writing essays when JFK was a high school freshman?" I would assume that either they were trying to trick me or that they were a non-native English speaker or computer program that didn't understand how English syntax or the concept of presidency worked. I don't think it's as complicated as all that. Most English speakers wouldn't refer to a single person…

> Most English speakers wouldn't refer to a single person by two distinct proper nouns in a sentence like this, so I'd assume they were referring to two separate people.

I agree with this (though elegant and inelegant variation is a thing). But nor would any English speaker be likely to ask about the skill of an anonymous holder of a position with reference to an ambiguous date range associated with a time in which a later holder of that position would be extensively practicing that skill, which is why I classed it the sort of sentence with no natural meaning you'd only conceive in order to be deliberately ambiguous (or because you don't understand how to write English properly).

The original sentence isn't ideally constructed, but at least the president when JFK was born is a natural and easily identified sentence object, and it would be odd to ignore that in favour of an alternative interpretation that also involved interpreting "how tall" as being more likely to refer to the [probably-not-recorded-for-posterity] length of a newborn infant (who wouldn't be idiomatically referred to as "tall" or the president in the context of his birth) than the height of a person who was president in the relevant time period

Re: Why This AI Moment May Be the Real Deal

#203

Post by ben_w » Tue, Aug 01, 2023, 9:52 PM UTC

nonameiguess wrote: post #178

ben_w wrote: post #64

Earlier quoted context omitted.

> But it is not conscious How can we tell? We don't have a single working definition of what "consciousness" is, rather we have at least half a dozen wildly different definitions, some of which aren't testable[0], some of which include VHS players[1], some of which exclude many humans most of the time[2]. If consciousness exists purely in working memory and not short/long term, then LLMs may have it within any given…

Consciousness seems like a red herring. OP can speak for themselves, but what I took is the feature preventing singularity is statelessness. Whether or not a thinking system experiences qualia, it seemingly needs some amount of persistence and continued ability to learn. As it stands, the training phase of a GPT might meet these requirements, but the inference phase is just a stored program. People talk of "ChatGPT"…

> Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning.

Depends on the size of the context window.

Humans don't, and can't, learn everything from the ground up — we seem to have started with peer learning (it predates humans), which became culture; then we got (complex) language and could share more complex discoveries about the world; then writing and the dead could share their thoughts with the not-yet born centuries later…

…and, apparently, with machines. (Imagine explaining to Plato that we'd made metal from sand and used amber to make it discuss his philosophies).

If the context window is enough to create part of something new, then yes, I'd say an LLM can likely invent (or at least contribute to inventing) ${interesting novel tech}.

That said, I don't buy the "everything all at once" meme of the technological singularity, as I think any model with infinities in it is wrong — my preferred analogy is the technological event horizon, which is more like "you can't make good predictions past this point", akin to the event horizon around singularities in relativity.

https://kitsunesoftware.wordpress.com/2022/09/20/not-a-singu...

Re: Why This AI Moment May Be the Real Deal

#204

Post by YeGoblynQueenne » Tue, Aug 01, 2023, 10:19 PM UTC

rtkwe wrote: post #58

jasfi wrote: post #23

Earlier quoted context omitted.

The cost to compute these language models should eventually lower. Will OpenAI then release more frequently, or release larger LLMs? They'll probably to try achieve both goals in some proportion.

A big issue with constant retraining is going to be the self referential consumption of it's own generated material as training material. There's already been the studies that these models quickly break down when they're fed their own generated data as training data. It won't immediately degrade them as it's a small percentage but there's already a lot of people using these models to generated spam junk out there and…

>> There's already been the studies that these models quickly break down when they're fed their own generated data as training data

Not "these models" if you're referring to large language models pretrained and fine-tuned with Transformers. The only study I am aware of that explored the effect you discuss [1] had to make do with very different models as proxies: Variational Autoencoders, Mixture of Gaussians and a smaller LLM with 125 million parameters that hardly qualifies as one. It is not safe to extrapolate those results to really Large-LMs trained with Transformers, as it is not safe to extrapolate any results on one neural net architecture on another.

Note the Mixtures of Gaussians aren't even neural nets at all, but a completely different statistical model.

The question of what will happen once a few generations of LLMs have trained on their ancestors' outputs is an interesting one that has not yet been clearly answered. In the long run, future developments may even mean it's not really a valid question at all because such feedback retraining can't happen in practice- who knows?

I for one remain curious to see how this goes.

______________________

[1] The Curse of Recursion: Training on Generated Data Makes Models Forget https://arxiv.org/abs/2305.17493

Re: Why This AI Moment May Be the Real Deal

#205

Post by notahacker » Tue, Aug 01, 2023, 10:25 PM UTC

balnaphone wrote: post #180

notahacker wrote: post #107

Earlier quoted context omitted.

Yes, you're misreading it. "the president" is not a label that applies solely to JFK, or one that applies at all to JFK in the context "when JFK was born". If someone gave me the question "who was the president when JFK was born?", the answer is clearly not JFK, so it isn't correct to infer that JFK is the president whose height, party affiliation or essay writing skill is being asked about "when JFK was born". If so…

> "who was the president when JFK was born?", the answer is clearly not JFK Yes, you can deduce that, since the question is nonsensical (trivial) if "the president" refers to JFK in that context. > If you flip it to "was JFK strong at writing essays when the president was a high school freshman" This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common t…

> This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common to have ill-defined references that are later clarified, and there's no formal rule that enforces that all terms must be defined before use.

References that are later clarified is exactly how we arrive at "the president when JFK was born" as defining Woodrow Wilson as the object of the sentence (and it makes much more sense to ask about how tall President Wilson was in the context of 1917 than to wonder how "tall" baby "president" JFK was in 1917).

Whereas "when the president was a high school freshman" can't be used to identify anybody else [unless there is a time period in which a high school freshman was the president!], even if the sentence hadn't already made it clear that JFK was the object of the sentence. But it's still an example of bad writing.

Even when there's no ambiguity, we don't do sentences like "The SpaceX founder was about 18 inches tall when Elon Musk was born" which is basically how you're implying the president question - which has a much more reasonable alternative interpretation - could be answered...

It is possible to write a truly ambiguous sentence in English; nobody is arguing otherwise. But whilst the original sentence might make you think a bit, it's not naturally interpreted as being equally/more likely to be a question about the not-tall not-president in the context of the time period instead of the person with height and presidency in the context of the time period, unless you're being obtuse or lack fluency in the language.

Re: Why This AI Moment May Be the Real Deal

#206

Post by YeGoblynQueenne » Tue, Aug 01, 2023, 10:26 PM UTC

DebtDeflation wrote: post #57

danaris wrote: post #20

Earlier quoted context omitted.

Thank you. This is one of the questions far too few people seem to be paying attention to. "Thinking" in any way that we truly understand the term requires consciousness , and consciousness requires much more continuity than LLMs have. It would need continuity of input as well as continuity of learning in order to even be able to begin to approach something we might recognize as consciousness.

Imagine a hypothetical black box that could correctly answer any question you ask it and perform any task that you instruct it to perform. In terms of the impact such a thing would have on the world, would it matter if it were conscious? Would it even be desirable for it to be conscious? IMO, discussions of consciousness and self-awareness are a complete red herring when it comes to the topic of AGI.

Well, imagine a model of the apparent motion of the planets where each planet moved on a perfectly circular orbit with any number of smaller circles on top of it, with the main circle of the orbit centered on a point between the Earth, or even the Sun, and another point a bit further from it. So, you know, an epicyclical model of the motion of the planets [1].

"In terms of the impact such a thing would have on the world", as you put it, would it matter if that model was completely wrong, despite its great predictive power?

Did we gain something when we figured out how the planets really move?

_______________

[1] https://en.wikipedia.org/wiki/Deferent_and_epicycle

Re: Why This AI Moment May Be the Real Deal

#207

Post by [deleted] » Tue, Aug 01, 2023, 10:30 PM UTC

iinnPP wrote: post #11

optimalsolver wrote: post #10

What's a large language model doing when it's not being queried? Am I correct that they only compute information when dealing with a prompt? If so, that seems like a fundamental flaw. An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.

Hooking an LLM up to a loop would solve that. Then you can find a way to include a described video feed and method of movement into the mix.

[deleted]

Re: Why This AI Moment May Be the Real Deal

#208

Post by gmerc » Tue, Aug 01, 2023, 11:04 PM UTC

__loam wrote: post #151

gmerc wrote: post #47

Earlier quoted context omitted.

Our brain is made of interconnected systems but somehow expect LLM architecture to encompass the whole spectrum. Nothing stops you from running a loop that involves other systems such as long term memory (vector /dev storage), visual pre-processor (CNN), auto lora, and more. That’s the fundamental flaw with most of the criticism - the tech is out only a few short months in the hands of everyone. The disruption will c…

Another flaw is trying to anthropomorphize the tech

What does that even mean in this context?

Re: Why This AI Moment May Be the Real Deal

#209

Post by famouswaffles » Tue, Aug 01, 2023, 11:29 PM UTC

YeGoblynQueenne wrote: post #206

DebtDeflation wrote: post #57

Earlier quoted context omitted.

Imagine a hypothetical black box that could correctly answer any question you ask it and perform any task that you instruct it to perform. In terms of the impact such a thing would have on the world, would it matter if it were conscious? Would it even be desirable for it to be conscious? IMO, discussions of consciousness and self-awareness are a complete red herring when it comes to the topic of AGI.

Well, imagine a model of the apparent motion of the planets where each planet moved on a perfectly circular orbit with any number of smaller circles on top of it, with the main circle of the orbit centered on a point between the Earth, or even the Sun, and another point a bit further from it. So, you know, an epicyclical model of the motion of the planets [1]. "In terms of the impact such a thing would have on the wo…

"Wrong" doesn't make much sense here. The more inductive bias we think is appropriate which we try to shove into models, the worse they perform. There's also an awful lot of fabrication the brain does with sense data, rationales etc. all of this is to say we have no clue what makes us tick. This means there's absolutely no guarantee we would recognize a replication of "human"

Something different is not necessarily "wrong". a plane's flight is no less "true" than a bird. It's not flying the "wrong" way.

Trying to elevate our very poor and wrong understanding of "human" to be the same as "right" or "true" is very silly. Even Biology with its set of constraints does not always solve the same problem the same way. Who are you to dub one way "right" ? Makes no sense.

Re: Why This AI Moment May Be the Real Deal

#210

Post by jiggawatts » Tue, Aug 01, 2023, 11:31 PM UTC

ChatGTP wrote: post #38

jiggawatts wrote: post #28

Earlier quoted context omitted.

> Where on the S-shaped curve are we? No-one knows. We can however make educated guesses based on: - Extrapolating the last couple of years of progress. - Scientific studies of LLM quality scaling in proportion to input data size, parameter count, and total training compute ops. - How much more training data is available. - Reasonable budgets, especially the cost-efficiency of inference , which appears to be more lim…

It really stands to reason that at some point these systems will just be "too smart" and we won't find any value in something that much more intelligent. It's like a 4 year old trying to understand a seasoned professor explaining some concept, we just won't get it. Imagine a hypothetical scenario where some ultra-intelligent beings delivered a book with all the answers humans ever wanted, but we needed an IQ of 500 t…

From what I've seen, the increases in LLM intelligence is not that type of smart. On the contrary, the better LLMs write in a more neutral and clear tone, a bit like a Wikipedia article.

Actually, Wikipedia articles are already sort-of a "distillation of the Internet" into a type of average. By having many people edit them, the individual personal quirks are smoothed out, leaving a type of average-of-humans result.

This reminds me of the "average faces" project about a decade ago where they blended photos of random people together. The results were often very attractive, because it's the small flaws that make normal people unattractive. The blending of many samples smooths out the flaws, resulting in a "perfect" but ordinary face. Not some post-human face, but a perfectly human face.

That's what I see LLMs doing: as smart as us, but no smarter. Great breadth certainly, but not greater depth.