Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

'Megalopolis' trailer's fake critic quotes were AI-generated

variety.com

Post reply on HN
Print view View original

51–57 of 57 posts

Re: 'Megalopolis' trailer's fake critic quotes were AI-generated

#51

Post by viraptor »

ChrisMarshallNY wrote: post #33
nsagent wrote: post #6

Was just talking with a microbiology researcher yesterday about ChatGPT. They use it for helping to write code for analyzing their lab data and they were surprised I said it's likely to lead them astray at times, especially if they don't know the code well enough to spot the errors it might generate. Despite my PhD being in NLP, they still seemed incredulous at my assertion. I truly think the general public have not…

I figured that out, the first time I submitted a Swift question to ChatGPT, where I would normally have used StackOverflow. It hallucinated an Apple API call. Not just that, but that nonexistent call was the entire fulcrum of its "solution."

That's a problem with lots of less common languages. The accuracy seems to scale with the number of github repos and SO questions. Most Swift code being in proprietary iOS/MacOS apps doesn't help it.

Re: 'Megalopolis' trailer's fake critic quotes were AI-generated

#52

Post by ChrisMarshallNY »

viraptor wrote: post #51
ChrisMarshallNY wrote: post #33

Earlier quoted context omitted.

I figured that out, the first time I submitted a Swift question to ChatGPT, where I would normally have used StackOverflow. It hallucinated an Apple API call. Not just that, but that nonexistent call was the entire fulcrum of its "solution."

That's a problem with lots of less common languages. The accuracy seems to scale with the number of github repos and SO questions. Most Swift code being in proprietary iOS/MacOS apps doesn't help it.

Also, Swift allows extensions of fundamental types. I often declare extensions of Int and Array. It’s fairly common to add computed properties and functions to basic APIs.

I’m pretty sure that a number of folks extended the API with this property (which is what I did, once I figured out how to address the issue. It just made sense to do so), and ChatGPT read that as a standard system call.

That’s likely to be an issue with more mainstream languages, as extension is a fairly common pattern, these days.

I think I may start tagging my extensions (like a naming prefix), to signal they are added after the fact. I do a lot of open-source work, so it’s likely to be used as training data.

Re: 'Megalopolis' trailer's fake critic quotes were AI-generated

#53

Post by surgical_fire »

viraptor wrote: post #51
ChrisMarshallNY wrote: post #33

Earlier quoted context omitted.

I figured that out, the first time I submitted a Swift question to ChatGPT, where I would normally have used StackOverflow. It hallucinated an Apple API call. Not just that, but that nonexistent call was the entire fulcrum of its "solution."

That's a problem with lots of less common languages. The accuracy seems to scale with the number of github repos and SO questions. Most Swift code being in proprietary iOS/MacOS apps doesn't help it.

Eh, I had the same experience asking questions on Java and Python.

The more strict your requirements, the less reliable LLMs are.

Re: 'Megalopolis' trailer's fake critic quotes were AI-generated

#54

Post by gwervc »

nsagent wrote: post #6

Was just talking with a microbiology researcher yesterday about ChatGPT. They use it for helping to write code for analyzing their lab data and they were surprised I said it's likely to lead them astray at times, especially if they don't know the code well enough to spot the errors it might generate. Despite my PhD being in NLP, they still seemed incredulous at my assertion. I truly think the general public have not…

> Despite my PhD being in NLP, they still seemed incredulous at my assertion.

I've the feeling the term "AI" erased in a lot people mind the fact text generation is a classical NLP text; they actually had no clue what NLP is in the first place. So we became to the current absurd situation where opinions of people with actual expertise and knowledge highly related to the thing are dismissed.

Re: 'Megalopolis' trailer's fake critic quotes were AI-generated

#55

Post by jessekv »

krapp wrote: post #10
nsagent wrote: post #6

Was just talking with a microbiology researcher yesterday about ChatGPT. They use it for helping to write code for analyzing their lab data and they were surprised I said it's likely to lead them astray at times, especially if they don't know the code well enough to spot the errors it might generate. Despite my PhD being in NLP, they still seemed incredulous at my assertion. I truly think the general public have not…

>I wonder how long until that fact becomes commonplace. When it starts killing enough people to make the news.

Self-driving cars already make the news when they crush pedestrians or plow themselves into trucks on the highway. Companies are still pushing/pursuing the tech. Why would the public response to LLM's be different?

Re: 'Megalopolis' trailer's fake critic quotes were AI-generated

#56

Post by viraptor »

surgical_fire wrote: post #53
viraptor wrote: post #51

Earlier quoted context omitted.

That's a problem with lots of less common languages. The accuracy seems to scale with the number of github repos and SO questions. Most Swift code being in proprietary iOS/MacOS apps doesn't help it.

Eh, I had the same experience asking questions on Java and Python. The more strict your requirements, the less reliable LLMs are.

Using agents improves this dramatically if you want to try again. Putting a linter, test runner and self reflection in the loop fixes a ridiculous number of LLM issues.

Re: 'Megalopolis' trailer's fake critic quotes were AI-generated

#57

Post by zx8080 »

minkles wrote: post #50
zx8080 wrote: post #46

Earlier quoted context omitted.

> totally unaware of the possibility this could happen. A rare human would easily admit their mistakes. Especially when their career and money and power is at stake.

That is why I didn't go into academia. I saw someone burn an entire research outcome on purpose because it'd compromise their speciality entirely.

Mind sharing which speciality?
Post reply on HN