AI isn’t good enough • Hacker News Board

Re: AI isn’t good enough

#301

Post by marcosdumay » Fri, Aug 25, 2023, 4:26 PM UTC

dontupvoteme wrote: post #49

intended wrote: post #35

Earlier quoted context omitted.

I can take a bet that it haha already failed - the hype cycle has already made a promise that LLMs can’t keep. Hallucinations to the normal person are a bug. The issue is that only humans can hallucinate. We know there is a “reality”. For an LLM, everything it does is a hallucination. That’s why you have more POCs than production goods. Your “hallucination rate” is unknown. Yesterday Ars has an article that described…

Why not "simply" multigen every (important) query and take the statistical average? Hallucinations are random, the truth isn't. This is absurdly expensive with GPT4, cheaper with 3, and dirt cheap locally with LLaMA

What makes you say that neural networks hallucinations are random? There's absolutely no reason for them to be.

Re: AI isn’t good enough

#302

Post by TeMPOraL » Fri, Aug 25, 2023, 4:27 PM UTC

worrycue wrote: post #156

layer8 wrote: post #150

Earlier quoted context omitted.

Maybe it’s more accurate to say that LLMs lack (self-)awareness. Because when you point out things that make no sense, they do have some limited ability to produce reasoning about that. But I agree that this lack of awareness is a serious and maybe fundamental deficit.

And how often does it get that wrong too? It’s more likely it’s just, once again, generating the most probable answer - and if you shake the magic 8 ball enough you will get the answer you were expecting.

Yeah, but the thing is, this seems exactly what people are doing too, at the boundary of conscious and unconscious, with the "inner voice" being most directly comparable to LLMs. It too generates language that feels like best completion, regardless of whether the output is logically correct or not.

Re: AI isn’t good enough

#303

Post by ericksoa » Fri, Aug 25, 2023, 4:31 PM UTC

When I see VCs make this argument, all I hear is "oh shit, we're late to the party" and doing something like this as a desperate attempt to bring valuations down.

Re: AI isn’t good enough

#304

Post by unusualmonkey » Fri, Aug 25, 2023, 4:31 PM UTC

jedbrown wrote: post #294

earthboundkid wrote: post #250

Earlier quoted context omitted.

I am a mild LLM skeptic. But I find the response of "oh, it's all just post-crypto scamming" really weird. Crypto was a total scam. There was never a concrete, non-criminal (important caveat) application where crypto was easier than just using PayPal or whatever. LLMs are very imperfect and still have a lot of work to do, but they do actually do some job related tasks today. If I have JS snippet and I wish it were in…

The LM industry valuation would be way smaller if they were not laundering behavior that would be illegal if a human did it. If "AI" were required to practice clean-room design ( https://en.wikipedia.org/wiki/Clean_room_design ) to avoid infringing copyright, we would laugh at the ineptitude. If people believed the FTC-CFPB-DOJ-EEOC joint statement was going to lead to successful prosecutions, the industry valuation…

Humans aren't required to use clean room design. Using copyrighted materials as inspiration/reference is not uncommon or illegal.

Re: AI isn’t good enough

#305

Post by NumberWangMan » Fri, Aug 25, 2023, 4:34 PM UTC

ChatGTP wrote: post #113

savolai wrote: post #101

Earlier quoted context omitted.

It seems to me that the real question here is what is true human intelligence. Ai has made it plain to see, by being able to replicate it so convincingly, that much of what we have considered intelligence has been pattern matching or acting as complex parrots. There is much more to the abilities of human body-mind-emotional-experiential being, but it is only slowly becoming mainstream. (Edit: Of course there are also…

I'm getting suspicious that there is a bit of a blind spot in understanding the world and the usefulness of what we call intelligence, as Noval Yuah Harari says, intelligence is overrated. Look at what we've done to the planet and our environment we have fucked it properly, yet we consider ourselves to be intelligent? Could it be that intelligence is overrated and discovery of new ideas / thing is underrated? Our ego…

What we've done to the planet is perhaps less a consequence intelligence, more a consequence of multi-polar traps. Though it's in our collective self-interest to protect the planet, it's in our individual self interest to ignore the problem / prepare so that our own children can "weather the storm".

It kinda depends on how much you care about people across the world and future generations.

But yeah, if humans were more intelligent, we probably would have sorted all this out by coming up with better coordination mechanisms, and by overcoming our tribal tendencies more effectively.

Re: AI isn’t good enough

#306

Post by ambrozk » Fri, Aug 25, 2023, 4:37 PM UTC

RandomLensman wrote: post #252

ambrozk wrote: post #230

Earlier quoted context omitted.

This is exactly the point. The pessimists get the pedantic thrill of pointing out that, 60 years ago, some proponents were overconfident. But they neglect to notice the larger picture, which is one of extraordinary progress. They'll be sitting in the passenger seat of a self-driving car, arranging their travel itinerary with a chatbot fluent in English and 60 other languages, and smugly commenting on HackerNews about…

Lots of progress in some areas, very little in others. Neither the pessimists nor the optimists got it right. (Btw., a really hardened pessimist might even say your example is mostly things that were doable when there were not many computers around at all... taxi driver to drive you, a secretary to book a flight, meet in a club to discuss things, ...)

To me, if artificial intelligence means anything, it means automating mundane tasks -- most of which humans can accomplish already. If someone tells me they're an AI pessimist because they think automating tasks like "converse intelligently about nearly any subject at length" and "drive my car anywhere while I sleep in the back" isn't an impressive AI achievement, then I think our disagreement pertains to our ambitions for the field, and lies outside the realm of the technical.

Re: AI isn’t good enough

#307

Post by chongli » Fri, Aug 25, 2023, 4:38 PM UTC

garblegarble wrote: post #282

chongli wrote: post #256

Earlier quoted context omitted.

86 billion (or 600+ trillion) times 20 years is a lot of computation What is that in kilowatt hours? Human brains are remarkably energy-efficient with their compute. We should get credit for that. Comparing one of us to an AI being trained at a data centre with the energy budget of a small city isn’t really fair, is it?

Seems that ~100W is regularly quoted as the average human energy output, so that'd be something like 18 MWh for 20 years (assuming 100% efficiency of food input). I suppose there's also the energy cost of all the training infrastructure (daycare, school teachers, homes, transportation, etc.), and also all the energy consumption of all humans that have come before and built those cities and knowledge infrastructure (a…

I don't think it's fair to include the billion years of human evolution as a cost on the human side of the chart but not include it in the AI side. AIs didn't evolve themselves out of the primordial ooze, they were built by humans and required herculean human effort to develop and improve. They stand on our shoulders, yet they're still in infancy when it comes to capability. I have yet to see any evidence of an LLM being able to replace an accountant, or an HR person, never mind a trash collector.

The best use of LLMs that I've seen so far is as a boilerplate-producing autocomplete system. Considering that we have better ways to automate this (better programming languages that can abstract away the boilerplate), this is not very high praise.

Edit: If you look at the total footprint (considering not just the efficiency of the neurons, but the whole animal) the figures are very close - the llama model card indicates that v2 70B caused the emission of ~300T of CO2, and an average human in the US emits 16T of CO2 a year, so a human would emit ~320T of CO2 in 20 years. I assume children don't have as high a CO2 output, but even so it seems like it's the same order of magnitude.

If you're going to include the whole animal on the human side, you need to include the whole supply chain on the LLM side. The cost of building all the fabs and doing all the R&D to develop and manufacture model training-specific computers (matrix multiplier hardware). Just like with crypto, these resources had to be diverted away from other things (e.g. causing the price of gamers' graphics cards to skyrocket). It's only fair to interrogate the ROI.

Re: AI isn’t good enough

#308

Post by naasking » Fri, Aug 25, 2023, 4:39 PM UTC

philipwhiuk wrote: post #241

We don't need better LLMs to eliminate all McDonalds workers. In fact there's no language model needed there at all. The job just needs better robotics.

I think you're underestimating how LLMs can help here:

https://www.deepmind.com/blog/rt-2-new-model-translates-visi...

Re: AI isn’t good enough

#309

Post by wredue » Fri, Aug 25, 2023, 4:42 PM UTC

earthboundkid wrote: post #250

wredue wrote: post #232

Earlier quoted context omitted.

Blockchain was a complete bust, and middle managers need buzzwords to sell senior leadership. That’s my personal take on the current wave.

I am a mild LLM skeptic. But I find the response of "oh, it's all just post-crypto scamming" really weird. Crypto was a total scam. There was never a concrete, non-criminal (important caveat) application where crypto was easier than just using PayPal or whatever. LLMs are very imperfect and still have a lot of work to do, but they do actually do some job related tasks today. If I have JS snippet and I wish it were in…

I apologize, because HN does tend to automatically go to crypto when discussing blockchain.

I was not talking about crypto. Blockchain “solutions” in enterprise were spinning up all over the place for non-crypto applications. In particular, in you work financial, supply chain, government or random startups, you probably heard blockchain a lot in non-crypto contexts.

Re: AI isn’t good enough

#310

Post by ambrozk » Fri, Aug 25, 2023, 4:53 PM UTC

ksaho wrote: post #266

ambrozk wrote: post #234

Earlier quoted context omitted.

Humans will never be intelligent. They're optimized for producing offspring, not reasoning. Humans may appear to be intelligent from a distance, but talk to one for any length of time and you'll find they make basic errors of reasoning that no truly thinking being would fall for. /s

Take out the /s tag and you are right on the money. Humans can not be trusted with anything because they are trivially fallible. Humans are terribly stupid, destroy their own societies and refuse to see reason. They also hallucinate when their destructive tendencies start catching up to them.

If the most intelligent machines ever observed in the universe do not count as "intelligent," then we have a semantic, and not a substantive difference of opinion.