Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT is now finding bugs in databases

celerdata.com

Post reply on HN
Print view View original

41–50 of 204 posts

Re: ChatGPT is now finding bugs in databases

#41

Post by yencabulator »

derefr wrote: post #12
crazysim wrote: post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

It can't access the internet now . But it was fed a snapshot of the Internet during training. So if you tell it a URL, it can potentially remember what the page looked like, if it happened to see it during training.

In either scenario, it's hallucinating. Either JSON queries were supported in that snapshot of the documentation, or they weren't.

Re: ChatGPT is now finding bugs in databases

#42

Post by philliphaydon »

swatcoder wrote: post #9
crazysim wrote: post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

Hah. Of course the authors are not aware. OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data. The whole point of giving the thing away for free these last few months is to let people make fools of themselves like this while pumping up OpenAI’s value as a technology firm. It’s no accident that they don’t engage to cl…

[dead]

Re: ChatGPT is now finding bugs in databases

#43

Post by ehnto »

dcow wrote: post #27
goatlover wrote: post #17

Earlier quoted context omitted.

The concern is the increasing number of anecdotes where people are taking ChatGPT's output at face value, even though everyone knows by now that it does sometimes get stuff wrong and does hallucinate. I thought it would take longer.

The problem is that people also sometimes get stuff wrong and hallucinate. So ChatGPT is just mimicking how we behave and it seems completely normal.

While it's true humans are fallible, it is more clear what someome should and should not get wrong, and we can make safer assumptions about the knowledge we get from certain individuals. ChatGPT doesn't "know" anything, you can get it to be right and wrong about the same detail in the same conversation. It's not clear what it should be correct about, and so it's a pretty poor source of information.

Re: ChatGPT is now finding bugs in databases

#44

Post by shallichange »

jahsome wrote: post #25
ldjkfkdsjnv wrote: post #5

Today at work I hadnt used apache spark in a long time. I needed to do some quick and dirty data analytics on large datasets. Rather than read documentation, I just put the queries i needed in english into chatGPT, and it spit out perfectly working code. The queries werent that complex, but it saved me hours googling a bunch of random spark SQL syntax. The old school way is google sends you links, and through those c…

I've had numerous days just like this. The perfect use case for ChatGPT for my personal workflow has been to give me the 70-percent-of-the-way-there skeleton for a given query, Terraform module, or even just to prevent myself from lookup up the syntax of for loops in for the 30,000th time. I can then take that 70 percent chunk and bolt on my edge cases as I go. There hasn't really been a time where the answer didn't…

I had it write my green card justification. Awesome corp-speak BS.

Re: ChatGPT is now finding bugs in databases

#45

Post by ss892714028 »

crazysim wrote: post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

[dead]

Re: ChatGPT is now finding bugs in databases

#46

Post by zoklet-enjoyer »

nuclearnice3 wrote: post #28
paxys wrote: post #19

Earlier quoted context omitted.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

Before the had the internet, back when we still had newspapers, we had a similar phenomenon referred to as Gell-Mann Amnesia. > Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. >…

This is why I've said Trump is right about fake news. When I say that people think I'm a weirdo Trumper.

Re: ChatGPT is now finding bugs in databases

#47

Post by colechristensen »

jahsome wrote: post #38
ceejayoz wrote: post #4

Earlier quoted context omitted.

> The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it". Yep. I asked it to summarize a URL. It happily invented something completely fabricated and convincing sounding (until you go to the URL) based of the domain name and article slug.

I asked it to generate me documentation for a handful Bicep definitions and include links to the official Microsoft Docs. It generated about 50% erroneous links, but once I updated my prompt and fed it the "root" of the MS docs I wanted it to use, it was able to correct all the urls. At the time it _seemed_ like it wasn't just doing a mindless string replace, because different parts of the url changed on most of the…

These models have probably been trained on a large portion of the Internet.

The training data however can be a few years old.

I have noticed this when trying to get it to generate some code, it was clearly using considerably old versions.

Re: ChatGPT is now finding bugs in databases

#48

Post by oceanplexian »

taberiand wrote: post #26
micromacrofoot wrote: post #6

Earlier quoted context omitted.

we’ve made the startling discovery that we’re rather predictable

My hypothesis is that we will soon see the emergence of a super-human artificial intelligence, not because of some astounding breakthroughs in our understanding of intelligence, but because of irrefutable proof that humans just aren't all that complicated to surpass.

Agreed, artificial intelligence doesn’t need to turn into Skynet to be “super human”, just better than the average 85-90 IQ human.

Re: ChatGPT is now finding bugs in databases

#49

Post by dcow »

ehnto wrote: post #43
dcow wrote: post #27

Earlier quoted context omitted.

The problem is that people also sometimes get stuff wrong and hallucinate. So ChatGPT is just mimicking how we behave and it seems completely normal.

While it's true humans are fallible, it is more clear what someome should and should not get wrong, and we can make safer assumptions about the knowledge we get from certain individuals. ChatGPT doesn't "know" anything, you can get it to be right and wrong about the same detail in the same conversation. It's not clear what it should be correct about, and so it's a pretty poor source of information.

And then there are trolls.
Post reply on HN