Live data from Hacker News

ChatGPT is now finding bugs in databases

celerdata.com

61–70 of 204 posts

Re: ChatGPT is now finding bugs in databases

#61

Earlier quoted context omitted.

People are always made unnecessary. The smart ones then find other ways in which they are. Not sure how this is any different.

What we are discussing is something that removes the need for problem solving. What will the supposed smart do if there’s a magical machine that can solve all problems that can be represented as text?

I don't know about others but I'll be enjoying the beach, partying, watching movies, etc.

Re: ChatGPT is now finding bugs in databases

#62
post #19

Earlier quoted context omitted.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

I think a lot of overly-eager and under-informed people are jumping on the bandwagon and shouting half-baked proclamations from the rooftops to get clicks and views. But underneath all the bullshit is something truly useful, so I wouldn't necessarily say that Confidently Incorrect™ is ChatGPT's primary ability. Coincidentally, over the past few weeks of this ChatGPT craze, I've needed to create a lot of fake data to…

While I agree with your premise, the "toil" that you mention is a very small use case for a tiny minority of people. And automating it away has always been trivial. What would have taken 30 minutes to write a Python script now takes 5 minutes to come up with the perfect prompt. And then you have to test and be sure that it does what you want. Neat, but not world-changing.

Meanwhile ChatGPT is already being hailed as the Google killer. People are using it to write essays and form opinions. Plenty of online and offline debates contains phrases from ChatGPT repeated verbatim. It's being adopted in newsrooms around the world. So what it is good for in theory and what people are actually using it for are worlds apart. IMO the technology is going to have its "self driving car kills pedestrian" moment very soon because of over-eager and careless users, and that is going to set the entire field back.

Re: ChatGPT is now finding bugs in databases

#63
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

How is it getting information partially correct, but not able to verify the correctness to at least be internally consistent with itself?

Re: ChatGPT is now finding bugs in databases

#64
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

Hah. Of course the authors are not aware. OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data. The whole point of giving the thing away for free these last few months is to let people make fools of themselves like this while pumping up OpenAI’s value as a technology firm. It’s no accident that they don’t engage to cl…

> OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data.

When one logs on to Chat GPT, they get the following notices:

> This is a free research preview.

> Our goal is to get external feedback in order to improve our systems and make them safer.

> While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice.

> Conversations may be reviewed by our AI trainers to improve our systems.

> Please don't share any sensitive information in your conversations.

Effectively no guidance or instructions indeed. Perhaps ChatGPT is sometimes confidently incorrect because it learned from online commenters.

Re: ChatGPT is now finding bugs in databases

#65
post #62

Earlier quoted context omitted.

I think a lot of overly-eager and under-informed people are jumping on the bandwagon and shouting half-baked proclamations from the rooftops to get clicks and views. But underneath all the bullshit is something truly useful, so I wouldn't necessarily say that Confidently Incorrect™ is ChatGPT's primary ability. Coincidentally, over the past few weeks of this ChatGPT craze, I've needed to create a lot of fake data to…

While I agree with your premise, the "toil" that you mention is a very small use case for a tiny minority of people. And automating it away has always been trivial. What would have taken 30 minutes to write a Python script now takes 5 minutes to come up with the perfect prompt. And then you have to test and be sure that it does what you want. Neat, but not world-changing. Meanwhile ChatGPT is already being hailed as…

I disagree. Toil is part of everyone’s job and what creates procrastinators. I used it recently to help me write a browser extension, where I find writing vanilla js and xpath expressions very tedious (just a bunch of syntax lookup) and would not have even bothered in the first place.

If you approach it with the right mindset, it’s extremely valuable.

Re: ChatGPT is now finding bugs in databases

#66
post #19
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

That's almost every reddit thread I've been involved in.

Oratory prowess is way more important than accuracy. If you ever want to "lose a debate" on reddit, pick something that's counter-intuitive or widely misunderstood and use nuance and citations in your defense. You'll get cyberbullied every time - sometimes even banned from the sub.

Re: ChatGPT is now finding bugs in databases

#67

What is the training referred to in the article? Fine tuning or do they just mean hours of "in context learning" (talking to the bot)?

It's not clear from the context, but I would guess that for the one that worked, they gave it several examples in one conversation and eventually it picked up the syntax. The "dozens of hours of training" that didn't work, and can't work, are because ChatGPT doesn't immediately roll all conversations back into the training set. It also specifically told them that its knowledge cutoff of 2021 means OpenAI will not knowingly add this information to the model.

Re: ChatGPT is now finding bugs in databases

#68
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

It’s been worth it to me to feed it code. Here a session log from an encryption discussion https://docs.google.com/document/d/1e2RV-ytnwTq1LRY0BRPgNWCr...

Re: ChatGPT is now finding bugs in databases

#70

Earlier quoted context omitted.

Hah. Of course the authors are not aware. OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data. The whole point of giving the thing away for free these last few months is to let people make fools of themselves like this while pumping up OpenAI’s value as a technology firm. It’s no accident that they don’t engage to cl…

> OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data. When one logs on to Chat GPT, they get the following notices: > This is a free research preview. > Our goal is to get external feedback in order to improve our systems and make them safer. > While we have safeguards in place, the system may occasionally generate…

Did you read those statements?

None of them say what the product is, what it does, or how to use it — which are the kind of thing one might call guidance and instructions.

Those are just disclaimers to make sure you don’t hold OpenAI responsible for anything you do with this thing, or blame them for anything they do with your attempts.

Post reply on HN