Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

People are just as bad as my LLMs

wilsoniumite.com

Post reply on HN
Print view View original

111–120 of 173 posts

Re: People are just as bad as my LLMs

#111

Post by Nevermark »

lo_zamoyski wrote: post #101
itchyjunk wrote: post #86

Earlier quoted context omitted.

What is your measure of intelligence?

I think using the word “intelligence” when speaking of computers, beyond a kind of figure of speech, is anthropomorphizing, and it is a common pseudoscientific habit that must go. What is most characteristic about human intelligence is the ability to abstract from particular, concrete instances of things we experience. This allows us to form general concepts which are the foundation of reason. Analysis requires conce…

> I think using the word “intelligence” when speaking of computers, beyond a kind of figure of

Intelligence is what we call problem solving when the class of "problem" that a being or artifact is solving is extremely complex, involves many or near uncountable combinations of constraints, and is impossible to really characterize well. Other than examples, of data points, and some way for the person or artifact to extract something general and useful from them.

Like human languages and sensibly weaving together knowledge on virtually every topic known to humans, whether any humans have put those topics together before or not.

Human beings have widely ranging abilities in different kinds of thinking, despite our common design. Machines, deep learning architectures, underpinnings are software. There are endless things to try, and they are going to have a very wide set of intelligence profiles.

I am staggered how quickly people downplay the abilities of these models. We literally don't know the principles they have learned (post training) for doing the kinds of processing they do. The magic of gradient algorithms.

They are far from "perfect", but at what they do there is no human that can hold a candle to them. They might not be creative, but I am, and their versatility in discussing combinations of topics I am fluent in, and am not, is incredibly helpful. And unattainable from human intelligence. Unless I had a few thousand researchers, craftsman, etc. all on a Zoom call 24/7. Which might not work out so well anyway.

I get that they have their glaring weaknesses. So do I! So does everyone I have ever had the pleasure to meet.

If anyone can write a symbolic or numerical program to do what LLM's are doing now - without training, just code - even on some very small scale, I have yet to hear of it. I.e. someone who can demonstrate they understand the style of versatile pattern logic they learn to do.

(I am very familiar with deep learning models and training algorithms and strategies. But they learn patterns suited to the data they are trained on, implicit in the data that we don't see. Knowing the very general algorithms that train them doesn't shed light on the particular pattern logic they learn for any particular problem.)

Re: People are just as bad as my LLMs

#112

Post by re-thc »

collingreen wrote: post #108
re-thc wrote: post #107

Earlier quoted context omitted.

> The ability to create novel solutions without a priori knowledge. If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people. So I disagree. There's a lot of ways to be intelligent. Not just the research and scientific type.

It takes novel solutions to walk down the street, interact with folks, dodge random incoming obstacles, respond to comments, and a bazillion other things almost everyone does all the time. I probably agree that most people aren't engaged very often and even when they are they suck at being awesome but that really isn't the bar being mentioned here.

> It takes novel solutions to

Right

> but that really isn't the bar being mentioned here.

Yes, the bar was "novel solutions WITHOUT priori knowledge"

So you've changed the definition. Please re-read what I disagree with and it's not just the novel part i.e. if I read it all from the Internet and copied it to be successful then that fails this definition.

Re: People are just as bad as my LLMs

#113

Post by bawolff »

So if you give a bunch of people a boring task, pay them the same regardless of if they treat it seriously or not - the end result is they do a bad job!

Hardly a shocker. I think this say more about the experimental design then it does about AI & humans.

Re: People are just as bad as my LLMs

#114

Post by bawolff »

re-thc wrote: post #107
cudgy wrote: post #103

Earlier quoted context omitted.

The ability to create novel solutions without a priori knowledge.

> The ability to create novel solutions without a priori knowledge. If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people. So I disagree. There's a lot of ways to be intelligent. Not just the research and scientific type.

> If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people.

I think most people agree with this statement.

Re: People are just as bad as my LLMs

#115

Post by bawolff »

dontseethefnord wrote: post #49
II2II wrote: post #38

Earlier quoted context omitted.

While I have never heard of someone referring to 7 as a lucky number, 7 is the most common sum of two rolled dice. So I can see how people would regard it as a lucky number. Along the same lines, I assume that someone who mentions 42 as a random number has at least some interest in science fiction.

You must be living under a rock if you’ve never heard of 7 as a lucky number.

I have never heard of 7 being lucky until this thread.

I think you overestimate how cultural diverse western countries are when it comes to small things like this.

Re: People are just as bad as my LLMs

#116

Post by isaacremuant »

So many articles like this HN have a catchy title and then a short article that doesn't really conclude the title.

The experiment itself is so fundamentally flawed it's hard to begin criticizing it. HN comments as a predictor of good hiring material is just as valid as social media profile artifacts or sleep patterns.

Just because you produce something with statistics (with or without LLMs) and have nice visuals and narratives doesn't mean is valid or rigorous or "better than nothing" for decision making.

Articles like this keep making it to the top of HN because HN is behaving like reddit where the article is read by few and the gist of the title debated by many.

Re: People are just as bad as my LLMs

#118

Post by leptons »

rainsford wrote: post #67
leptons wrote: post #34

Earlier quoted context omitted.

Employees get fired all the time, and the more wrong answers I get from an LLM, the less I use it until it's never used again.

But an LLM doesn't understand "never used again" as a consequence and the threat of it is useless as a motivation to improve (also because LLMs have no concept of "motivation" or "threats" or anything else).

You're talking about LLMs as if they are some kind of singular entity, but LLMs as used for coding only exist as a product of a company that employs humans. If nobody uses the LLM because it sucks, those people will be out of a job.

Re: People are just as bad as my LLMs

#119

Post by skydhash »

re-thc wrote: post #107
cudgy wrote: post #103

Earlier quoted context omitted.

The ability to create novel solutions without a priori knowledge.

> The ability to create novel solutions without a priori knowledge. If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people. So I disagree. There's a lot of ways to be intelligent. Not just the research and scientific type.

Creating novel solutions have nothing to do with academic. Almost everyone encounters unexpected situations, that they have to think about to solve. It may not been novel as a whole, but for the person, it is.

Re: People are just as bad as my LLMs

#120

Post by tharkun__ »

sigpwned wrote: post #96
rainsford wrote: post #74

> ...a lot of the safeguards and policy we have to manage humans own unreliability may serve us well in managing the unreliability of AI systems too. It seems like an incredibly bad outcome if we accept "AI" that's fundamentally flawed in a way similar to if not worse than humans and try to work around it rather than relegating it to unimportant tasks while we work towards a standard of intelligence we'd otherwise ex…

I don’t disagree. But I also wonder if there even is an objective “right” answer in a lot of cases. If the goal is for computers to replace humans in a task, then the computer can only get the right answer for that task if humans agree what the right answer is. Outside of STEM, where AI is already having a meaningful impact (at least in my opinion), I’m not sure humans actually agree that there is a right answer in m…

Does "knowing what today is" count as "Outside STEM"? Coz my interactions with LLMs are certainly way worse than most people.

Just tried it:

   tell me the current date please

   Today's date is October 3, 2023.
Sorry ChatGPT, that's just wrong and your confidence in the answer is not helpful at all. It's also funny how different versions of GPT I've been interacting with always seem to return some date in October 2023, but they don't all agree on the exact day. If someone knows why, please do tell!

Most real actual human people would either know the date, check their phone or their watch or be like "Oh, that's a good question lol!". But somehow GPTs always be the 1% of people that will lie to know the answer to whatever question you ask them. You know, the kind that evening talk shows will ask ask. Questions like "how do do chickens lay eggs" and you get all sorts of totally completely b0nkers but entirely "confidently told" answers. And of course they only show the ones that give the b0nkers con-man answers. Or the obviously funnily stupid people.

Of course absent access to a "get the current date" function it makes sense why an LLM would behave like it does. But it also means: not AGI, sorry.

Post reply on HN