Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 is behind schedule

wsj.com

Post reply on HN
Print view View original

721–730 of 1001 posts

Re: GPT-5 is behind schedule

#721

Post by [deleted] »

swiftcoder wrote: post #627
reissbaker wrote: post #625

Earlier quoted context omitted.

Earlier in this thread, people mention the counterpoint to this: they Google the information from the LLM and do more reading. It's an excellent starting point for researching a topic: you can't trust everything it says, but if you don't know where to start, it will very likely get you to a good place to start researching. Similarly, while you can't fully trust everything a journalist says, it's obviously better to h…

> they Google the information from the LLM and do more reading The runway on this one seems to be running out fast - how long before all the google results are also non-expert opinions regurgitated by LLMs?

[deleted]

Re: GPT-5 is behind schedule

#722

Post by 1024core »

swiftcoder wrote: post #627
reissbaker wrote: post #625

Earlier quoted context omitted.

Earlier in this thread, people mention the counterpoint to this: they Google the information from the LLM and do more reading. It's an excellent starting point for researching a topic: you can't trust everything it says, but if you don't know where to start, it will very likely get you to a good place to start researching. Similarly, while you can't fully trust everything a journalist says, it's obviously better to h…

> they Google the information from the LLM and do more reading The runway on this one seems to be running out fast - how long before all the google results are also non-expert opinions regurgitated by LLMs?

People are forgetting about the content farms like Associate Content [1]. Since the early aughts, these content farms would happily produce expert-sounding content on anything that people were searching for. They would buy top search terms from search engines like Yahoo, hire English majors for dirt cheap, and have them produce "expert" content targeting those search terms. At least the LLMs have been trained on some relevant data!

[1] https://en.wikipedia.org/wiki/Yahoo_Voices

Re: GPT-5 is behind schedule

#723

Post by nycdatasci »

sampullman wrote: post #701
karel-3d wrote: post #693

I have to say I finally "caved in" to LLMs last month. While I still think Copilot is useless, I recently had a very complex code that did a lot of crazy bit-flipping and xoring, and I had no idea what is it doing, so I threw it to ChatGPT.. and it knew what it was doing. I also needed to rewrite this code to PHP (for... reasons) while I know very little PHP. And it did that! It was a bit wrong, I needed to correct a…

That's basically my experience. It's great for learning or getting things done when the subject is related to one you know well (i.e. you understand the fundamentals and can verify responses quickly). It's not so good for a completely new subject, or one you have a lot of experience in.

They're not useful when you're completely new to a subject? On the contrary, I've found that they are excellent when you have limited knowledge in a domain and less useful when you have expertise. They have allowed me to go from zero to functioning MVP on numerous computer vision projects, even though I have zero experience.

Re: GPT-5 is behind schedule

#724

Post by dcre »

BillyTheKing wrote: post #615
LASR wrote: post #610

So the team I lead does a lot of research around all the “plumbing” around LLMs. Both technical and from a product-market perspectives. What I’ve learned is that for the most part that AI revolution is not going to be because of PHD-level LLMs. It will be because people are better equipped to use the high-schooler level LLMs to do their work more efficiently. We have some knowledge graph experiments where LLMs contin…

just a question for understanding - if we say 'it learns', does it mean it actually learns this as part of its training data? or does this mean it's stored in a vector DB and it retrieves information based on vector search and then includes it in the context window for query responses?

The latter. “Learning” in the comment clearly refers to adding to the knowledge graph, not about training or fine-tuning a model. “and then you can RAG them.”

Re: GPT-5 is behind schedule

#725

Post by redmajor12 »

FrustratedMonky wrote: post #698
ithadtobe119 wrote: post #638

Earlier quoted context omitted.

(throwaway account because of what I'm about to say, but it needs to be said) While my main use case for LLMs is coding just like most people here, there are lots of areas that are being ignored. Did you know llama 3.X models have been trained as psychotherapists? It's been invaluable to dump and discuss feelings with it in ways I wouldn't trust any regular person. When real therapists also cost more than what people…

This seems to be part of a side plot in Blade Runner 2049. The movie was about replicants of course, but in the background, the technology shown with the AI being a companion, it was a huge corporate hit, a big seller. In the background you see ad's for it, and they reference it as their most popular product. And, as you allude to, in the movie it was both for loneliness AND sexual. They interacted like a relationshi…

What I liked about this in Blade Runner, was that if replicants are "people" (more the topic of the first movie), then it's not much of a stretch to consider software-AI as people, too. It would have been great if this question had been further explored in the 2md movie instead of just accepted.

Re: GPT-5 is behind schedule

#726

Post by SamPatt »

pera wrote: post #650
netdevphoenix wrote: post #640

Earlier quoted context omitted.

> The ability to “talk to an expert” about any topic I’m curious about and ask very specific questions has been invaluable to me. It is dangerous to assume that LLMs are experts on any topic. With or without quotes. You are getting a super fast journalist intern with a huge memory but inability to reason critically, lacking understanding about anything and huge unreliability when it comes to answering questions (you…

It's scary to think that we are moving into this direction: I can see how in the next few years politicians and judges will use LLMs as neutral experts. And all in the hand of a few big tech corporations...

They aren't just in the hands of big corporations though.

The open source, local LLM community is absolutely buzzing right now.

Yes, the big companies are making the models, but enough of them are open weights that they can be fine tuned and run however you like.

I think LLMs genuinely do present an opportunity to be neutral experts, or at the least neutral third parties. If they're run in completely transparent ways, they may be preferable to humans in some circumstances.

Re: GPT-5 is behind schedule

#727

Post by TeMPOraL »

discreteevent wrote: post #589
TeMPOraL wrote: post #543

Earlier quoted context omitted.

So LLMs are unreliable experts, okay. They're still useful if you understand their particular flavor of unreliability (basically, they're way too enthusiastic) - but more importantly, I bet you have exactly zero human experts on speed dial. Most people don't even know any experts personally, much less have one they could call for help on demand. Meanwhile, the unreliable, occasionally tripping pseudo-experts named GP…

> Most people don't even know any experts personally, much less have one they could call for help on demand. Most people can read original sources.

They can, but they usually don't, unless forced to.

(Incidentally, not that different from LLMs, once again.)

Re: GPT-5 is behind schedule

#728

Post by jordanb »

1024core wrote: post #722
swiftcoder wrote: post #627

Earlier quoted context omitted.

> they Google the information from the LLM and do more reading The runway on this one seems to be running out fast - how long before all the google results are also non-expert opinions regurgitated by LLMs?

People are forgetting about the content farms like Associate Content [1]. Since the early aughts, these content farms would happily produce expert-sounding content on anything that people were searching for. They would buy top search terms from search engines like Yahoo, hire English majors for dirt cheap, and have them produce "expert" content targeting those search terms. At least the LLMs have been trained on some…

So with AI Google has cut out the middleman and insourced the content farm.

Re: GPT-5 is behind schedule

#729

Post by soulofmischief »

ciconia wrote: post #551
t_serpico wrote: post #208

One fundamental challenge to me is that if each training run because more and more expensive, the time it takes it to learn what works/doesn't work widens. Half a billion dollars for training a model is already nuts, but if it takes 100 iterations to perfect it, you've cumulatively spent 50 billion dollars... Smaller models may actually be where rapid innovation continues simply because of tighter feedback loops. O3…

When you think about it it's astounding how much energy this technology consumes versus a human brain which runs at ~20W [1]. [1] https://hypertextbook.com/facts/2001/JacquelineLing.shtml

You've discovered the importance of well-formed priors. The human brain is the result of millions of years of very expensive evolution.

Re: GPT-5 is behind schedule

#730

Post by dmead »

karel-3d wrote: post #693

I have to say I finally "caved in" to LLMs last month. While I still think Copilot is useless, I recently had a very complex code that did a lot of crazy bit-flipping and xoring, and I had no idea what is it doing, so I threw it to ChatGPT.. and it knew what it was doing. I also needed to rewrite this code to PHP (for... reasons) while I know very little PHP. And it did that! It was a bit wrong, I needed to correct a…

Kill php with fire please.
Post reply on HN