Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

salon.com

Post reply on HN
Print view View original

141–150 of 164 posts

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#141

Post by hammyhavoc »

mellosouls wrote: post #107
keskival wrote: post #90

Large Language Models aren't a silver bullet – they don't solve all your problems. But they are a holy grail – as a universal common sense module they give IT systems a capability they never had before, a capability which has been sought after from since computers became a thing, a capacity for common sense. We now have that capacity and that alone will revolutionize the world. The chatbots aren't about chat, they ar…

We will certainly achieve AGI during this year... As exciting and transformative as GPT3+ is, let's not get too hypey. You need to back up outlandish claims with actual evidence and references. As discussed many times in this forum there's no evidence of sentience or any reason to consider the current systems to be even on the path to AGI.

Props up falling share prices though.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#142

Post by hammyhavoc »

idopmstuff wrote: post #35

I'm surprised this article is getting upvoted - it feels like very lazy journalism to me. > The discomforting reality is that, while Altman and his ilk have been predicting an exponential acceleration of productivity, we have been experiencing a deceleration. This is a very big claim, and there is absolutely nothing to back it up. The only specific reference to productivity is about an MIT paper that showed increases…

> This is very poor journalism

Then be the change you want to see. Write a counterargument and submit it. Better yet, use an LLM to write the article and state the prompts used.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#143

Post by [deleted] »

munificent wrote: post #83
cl42 wrote: post #14

The first steam engines were also written off as being less powerful than a horse. The first electric motors were written off as being less powerful than steam engines. So it goes. I think both of these views can be true at the same time: ChatGPT (or, LLMs really) are revolutionary and they won't revolutionize the world the way technologists/researchers say. Early adopters will use the technology and do amazing thing…

> Unions are already pushing back on AI (truckers, federal employees in Canada, writers in Hollywood) and maybe rightly so. Interesting bit of historical trivia: In the US, the main truck driver's union is the International Brotherhood of Teamsters. What is a teamster? Historically, it was a person who wrangled a team of horses or oxen to pull a wagon. That profession was effectively eliminated by the creation of the…

[deleted]

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#144

Post by simonw »

nottorp wrote: post #56
simonw wrote: post #29

This article is from March 2023, which in LLM terms is pretty old! The "How many bears have the Russians sent into space?" question still returns hallucinations with ChatGPT 3.5 but, unsurprisingly, gets a correct answer from GPT-4.

But is it because GPT-4 is better, or because the owners have hand tuned the public embarrassing examples out by hand?

It's definitely because GPT-4 is better - I've seen this kind of improvement across all kinds of prompts I've tried myself as well.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#145

Post by vidarh »

sharemywin wrote: post #63
vidarh wrote: post #38

Earlier quoted context omitted.

It seems to do ok at coming up with starting points or give you options if you're stuck. But the quality of the prose it comes up with is indeed awful. It gets a bit better if you ask it to write in the style of a specific author, but marginally so. I guess maybe it gets to mediocre fan fiction level. That's still pretty impressive, but not very usable for creative writing yet.

Don't most create writers have a process? I wonder if there's a series of steps(prompts) that could be used to get it to out put something much better. I know I've used it to write something I would have never even attempted let alone tried to write on my own. It came out ok, but better than I could have done on my own.

Some do. Others will just sit down and write. But the problem isn't so much that it can't handle plot. That is amenable to process - there are huge numbers of different processes, and one that might work well for GPT is something called the Snowflake method, which is basically iterative refinement. E.g. start with a one line description, expand it to a paragraph, expand each paragraph to a paragraph, then to a page or a few, and eventually to a list of scenes, and write out the scenes. Oversimplified (there's some steps with character sheets etc. too).

For that it might well be useful, because you could do one iteration at a time, edit the output to keep/reject ideas and do the next step.

But the challenge is that while it might not be "easy", it's the less time consuming part of a novel (certainly has been for me). The time consuming part is writing out the scenes, and the part GPT so far is awful at is the prose. So even if you manage to get it to produce a coherent script setting out what should happen, you still (so far) will have to expect to rewrite the entire thing anyway. That may or may not be useful to you. For my part I suspect I'd write faster from scratch than trying to edit and keep it consistent.

That said, given how far it's gotten I wouldn't at all be surprised if it can get to reasonable prose in another couple of versions.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#146

Post by hammyhavoc »

jstarfish wrote: post #130
throw_nbvc1234 wrote: post #103

Earlier quoted context omitted.

And we should blindly trust anything you read on the internet written by a real person? It's still valuable as a "search engine" with a different interface, especially if you can describe the problem but don't know the words to search for. For me that's a common issue when jumping into a new space.

At least there is usually a comments section on websites where people can call out bullshit. If you're not an expert in your field, GPT will successfully lie to you. There's no second opinion. Rather than a feelings filter, it needs an integrity filter.

There is a trend now of not having a comment section because it reduces server overhead by being able to serve a completely static page, and also directs the outrage and moderation required for it to social media, thus driving further traffic and reducing work required to deal with the fallout of it.

LLMs don't lie, that implies they have intent and is giving too much credibility to the idea that AGI may happen. They get things wrong because it's trying to use the wrong tool for the task at hand. It's a next word prediction engine, and that's not very useful for most things. By their very nature, they "hallucinate", but they don't really hallucinate, they just give the wrong output because it is just an LLM.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#147

Post by hammyhavoc »

ethanbond wrote: post #123
PartiallyTyped wrote: post #41

Earlier quoted context omitted.

Copilot and chatgpt are some of the services i am perfectly content paying for. ChatGPT is a great way to delve into new topics, and for that alone it’s worth it.

Curious if you've queried ChatGPT on topics that are not new to you and been impressed with its conveyance of relevant information? I think there's value here, but I also think people are way overestimating the quality of the "knowledge" they think they're receiving from it on novel topics.

No, it's objectively shit for a lot of things. The more technical and abstract something is in its concepts, the worse it becomes. It is just an LLM and that means it is inappropriate by its very nature for most things.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#148

Post by hammyhavoc »

mediaman wrote: post #111
jhp123 wrote: post #68

Earlier quoted context omitted.

LLMs also hallucinate during summarization tasks, adding topics that were not in the original

I've built internal systems that do summarization based on knowledge retrieval systems for specific nonpublic corporate information. With GPT-4, I find very little hallucinating. It very rarely deviates from the source material. Every time I've found something unexpected, there was a problem in the source material provided to the model.

"very little" is still an unacceptable amount for most fields.

Quantify "very little" over what time period, variations of use, fail states, sample size.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#149

Post by jstarfish »

hammyhavoc wrote: post #146
jstarfish wrote: post #130

Earlier quoted context omitted.

At least there is usually a comments section on websites where people can call out bullshit. If you're not an expert in your field, GPT will successfully lie to you. There's no second opinion. Rather than a feelings filter, it needs an integrity filter.

There is a trend now of not having a comment section because it reduces server overhead by being able to serve a completely static page, and also directs the outrage and moderation required for it to social media, thus driving further traffic and reducing work required to deal with the fallout of it. LLMs don't lie, that implies they have intent and is giving too much credibility to the idea that AGI may happen. They…

Fraud requires intent. Lying is behavioral.

Pathological liars don't intend to deceive. They can't seem to stop themselves. They literally hallucinate events even when presented with evidence to the contrary and we still call them liars.

A "hallucination engine" presented as a source of truth is fundamentally dishonest. We don't go to fortune-tellers when we need legal advice. We don't learn law from reading every Matlock script ever written.

A human hallucinating behind the wheel and plowing into traffic gets jail and a license suspension. An AI doing the same gets a press release about FSD being right around the corner.

This tech is literally replacing humans (by abstraction of the responsibilities they assume). It needs to be held accountable as one even if it's not technically AGI (or wherever the goalposts are being moved to next).

But our continued faith in tech as a growth industry depends on it not being perceived as a lie, so we must keep up the illusion that your car is just suffering from cyberpsychosis when it hallucinates and drives you off a cliff.

Re: Don't believe the hype: why ChatGPT is not the “holy grail” of AI research

#150

Post by TheCaptain4815 »

Vespasian wrote: post #78
TheCaptain4815 wrote: post #36

Earlier quoted context omitted.

GPT4 is significantly better at code fyi, not sure if you used default ChatGPT (which is 3.5turbo). Premium ChatGPT gives you access to GPT4, but the API gives you even more access with the ability to edit the "system" prompt which Sam Altman's said is very important and which I can attest to from my testing.

I did and do try it with GPT4 and the API and a custom prompt to help me in rust. Expecting only knowledge from 2021 it is hit and miss. Helped me to write some scripts but still fails to recognize tasks that are not possible in the language and halluciantes a plausible solution. It tends to suggest non non compiling and (after being asked to do so) "corrects" bugs (some real some imaginary) and sometimes gets stuck.…

There’s also the code interpreter functionality that’s in alpha access. Not everyone has access to it but it allows you to upload spreadsheets, code files, etc and then uses gpt4 to try and interpret and fix the code.
Post reply on HN