Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI isn’t good enough

skventures.substack.com

Post reply on HN
Print view View original

91–100 of 374 posts

Re: AI isn’t good enough

#91

Post by trashtester »

andrewjl wrote: post #21
borissk wrote: post #11

Very interesting article. IMHO a key factor in how the world changes in the next 10-20 years will be how far the current wave of AI, based on neural networks can go. If scientists and engineers manage to implement autonomous cars and robots that can do basic human tasks like clean, wait and take care of the sick and elderly we will wake up in a brand new world. It will put a lot of stress on the basics of society, su…

> If scientists and engineers manage to implement autonomous cars and robots that can do basic human tasks like clean, wait and take care of the sick and elderly we will wake up in a brand new world. It will put a lot of stress on the basics of society, such as democracy and having to work for a salary. Based on the trends TFA discusses we'll be seeing severe social stress if basic tasks don't get automated. All due…

The only way we will NOT see severe social stress is if AI is able to barely keep up with the decline in labour supply.

My gut feeling is that it will either be too little or way too much.

Anyway, once they're able to fully robot-ify the police force, social upheaval may become easy to deal with for those in power.

Re: AI isn’t good enough

#92

Post by dontupvoteme »

albertzeyer wrote: post #81
usrbinbash wrote: post #67

Earlier quoted context omitted.

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

Look at the scaling laws. We found that extrapolating the performance given a few data points with smaller models is actually very accurate. That's how they determined hyper parameters, by tuning them on multiple smaller scale models and then extrapolating. So far, all those predictions were quite good. Together with a bigger model, we also need more data to get better performance. If we add video and audio to the te…

Scaling laws apply to a single model. The best single model right now is supposedly a 8x mixture of experts, so not even really a single model in the purist sense.

I still expect the final solution will be more along the lines of picking the best model(s) from a sea of possible models, switching them in and out as needed, and then automatically reiterating as needed.

Re: AI isn’t good enough

#93

Post by edf13 »

usrbinbash wrote: post #67
ambrozk wrote: post #18

This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Or, as the author puts it, "we are at the tail end of the first wave of large language model-based AI... [it] ends somewhere in the next year or two with the kinds of limits people are running up against." I want to know only one thing, which is what gives him the confidence necessary to say that. If that one stat…

> This entire piece is based on one massive, unsupported assertion, which is that LLM progress will cease. Which is countered by...the assertion that it won't? LLMs won't get intelligent. That's a fact based on their MO. They are sequence completion engines. They can be fine tuned to specific tasks, but at their core, they remain stochastic parrots. > I want to know only one thing, which is what gives him the confide…

> They are sequence completion engines

But at the basic level - isn't our own brain just a sequence completion engine too?

Re: AI isn’t good enough

#94

Post by csydas »

petermcneeley wrote: post #3

"too few people for all the jobs, for the most part" - Are wages going up? "But almost everywhere else needs people. Badly. Across retail, restaurants, manufacturing, trades, and on and on, companies are struggling to hire. " - Wow sounds like an exciting star trek future that I cant wait to be part of! "labor became more expensive than capital" - So wages must be going up right? "In essence, the authors show that fo…

I'm a bit disappointed this part of the conversation is buried under everyone arguing about whether the article is right about AI or not, since the real interesting part of the article is why the author(s) seem to think that the reason McDonalds is doing signing bonuses is because people are just just not working, and the article never seems to really understand why people aren't flocking to jobs colloquially known to be absolutely awful and shit jobs.

Similarly, I'm very disappointed that the discussions in the article focus so much on _replacement_ of these workers, but in terms of human replacement, it really seems like AI is better suited to displace management and executives, not the people who perform specialized and physical work.

AI should help people and extend their abilities, and right now it's very good at that. It can be used to make our lives easier _now_, not replace us, and I think this is the real reason the article is so heavily discussed. I think the article explains how the author released a lot of the conclusions they did, and I agree, the author is jumping around way too fast and in my interpretation, making some very wild assertion in the article without further fleshing out their points.

Re: AI isn’t good enough

#95

Post by dontupvoteme »

andrepd wrote: post #39
jasfi wrote: post #37

Earlier quoted context omitted.

Are you using GPT-4? If not, it's understandable. If you don't pay for ChatGPT, you get GPT-3.5. You can also get access to GPT-4 if you use the playground.

Why is it always the same reply? Yes, GPT4 is as useless as GPT3.5 on any non trivial task.

The WebUI was definitely nerfed but the API still seems ok.

Re: AI isn’t good enough

#96

Post by dontupvoteme »

thatjoeoverthr wrote: post #88
dontupvoteme wrote: post #49

Earlier quoted context omitted.

Why not "simply" multigen every (important) query and take the statistical average? Hallucinations are random, the truth isn't. This is absurdly expensive with GPT4, cheaper with 3, and dirt cheap locally with LLaMA

Just try it!

I'll give this a shot after work I think.

The question I have is what's a prompt which reliably hallucinates but still produces the correct answer some of the time?

I know it gets some python functions "wrong", but i think they were actually "right" in the version it was trained on, so software seems out.

Re: AI isn’t good enough

#97

Post by KirillPanov »

PaulDavisThe1st wrote: post #4
petermcneeley wrote: post #3

"too few people for all the jobs, for the most part" - Are wages going up? "But almost everywhere else needs people. Badly. Across retail, restaurants, manufacturing, trades, and on and on, companies are struggling to hire. " - Wow sounds like an exciting star trek future that I cant wait to be part of! "labor became more expensive than capital" - So wages must be going up right? "In essence, the authors show that fo…

Capital's willingness to ride out labor shortages without increasing wages seems to have become substantially greater in the last couple of decades. TIRED: labor shortage -> wages rise! WIRED: labor shortage -> y'all work harder for maybe a bit of overtime or maybe not, and we'll just absorb the complaints about customer service and product quality. Those damn employees aren't getting a penny more!

> we'll just absorb the complaints about customer service and product quality

AKA our economy consists entirely of monopolies.

Re: AI isn’t good enough

#98

Post by TeMPOraL »

Aperocky wrote: post #89
squeaky-clean wrote: post #87

Earlier quoted context omitted.

> LLMs won't get intelligent. I think this sentence doesn't mean much unless we have a strict definition of what intelligence means. Just today ChatGPT helped me solve a DNS issue that I would not have been able to solve on my own in one day, let alone an hour. I'd consider it already more intelligent than myself when it comes to DNS.

It's seen more DNS content than you and anybody else have seen in their entire lives, and are able to regurgitate what it read because it has far faster memory access than you did. A dictionary contain knowledge but no intelligence.

And LLMs are the opposite of dictionary, actually. They suck at storing facts. They excel at extracting patterns from noise and learning them. It's not obvious to me that this isn't intelligence; on the contrary, I feel it's very much a core component of it.

Re: AI isn’t good enough

#99

Post by jiggawatts »

dontupvoteme wrote: post #49
intended wrote: post #35

Earlier quoted context omitted.

I can take a bet that it haha already failed - the hype cycle has already made a promise that LLMs can’t keep. Hallucinations to the normal person are a bug. The issue is that only humans can hallucinate. We know there is a “reality”. For an LLM, everything it does is a hallucination. That’s why you have more POCs than production goods. Your “hallucination rate” is unknown. Yesterday Ars has an article that described…

Why not "simply" multigen every (important) query and take the statistical average? Hallucinations are random, the truth isn't. This is absurdly expensive with GPT4, cheaper with 3, and dirt cheap locally with LLaMA

That’s an established technique with papers written on the topic and everything.

Anecdotally I tested this by having GPT4 translate Acadian cuneiform — which it can just barely do. I had it do this four times and it returned four gibberish answers. I then prompted it with the source plus the four attempts and asked for a merged result.

It did it better than the human archeologists did! More readable and consistent. I compared it with the human version and it matched the meaning.

Expensive now… soon to be standard?

Re: AI isn’t good enough

#100

Post by trashtester »

tuatoru wrote: post #15
borissk wrote: post #11

Very interesting article. IMHO a key factor in how the world changes in the next 10-20 years will be how far the current wave of AI, based on neural networks can go. If scientists and engineers manage to implement autonomous cars and robots that can do basic human tasks like clean, wait and take care of the sick and elderly we will wake up in a brand new world. It will put a lot of stress on the basics of society, su…

The real world has many trillions of parameters. Also, doing stuff in the real world is all about sensors. Progress on integrating millions of sensors into a robot has been slow.

The problem is not integrating millions of sensors of the same type (like individual pixels in a camera), but rather to integrate 2-10 different types of sensors, like microphones, cameras, lidars and tactile/temperature sensors.

Progress has been slow mosly because it's been hand-written, such as the sensor fusion in the F-35.

While there are some attempts at using neural networks to do much of the integration (Like Tesla's self driving) they're mostly quite primitive, and lacks the kind of general understanding of the world that we're starting to se in LLM's.

We have started building multimodal neural networks, though, such as GPT-4's ability to handle images, though. he next step is to add video, which is the mode that requires the greatest amount of compute, we will be able to integrate it with lower bandwidth sensors using the same techniques.

That would be where general purpose models start to become good enough to be deployed in robots. Let's imagine your Tesla has the ability to reason about the world the way GPT-4 can (and with enough fine tuning that halicunations are gone), and that it can do that not only based on text, but also all the sensors of the car, then it might become able to handle situations outside its training set much better than today, possibly better than many humans.

Once we're able to build multimodal models able to handle realtime video and s

Post reply on HN