Live data from Hacker News

ChatGPT is now finding bugs in databases

celerdata.com

151–160 of 204 posts

Re: ChatGPT is now finding bugs in databases

#151
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

> ChatGPT can't access the external internet.

This might not be entirely true. People were able to fetch urls and data from the internet that couldn't possibly be in any training dataset (in eg posted the same day for the first time).

https://medium.com/@neonforge/i-knew-it-chatgpt-has-access-t...

Re: ChatGPT is now finding bugs in databases

#152
post #53

Earlier quoted context omitted.

In the pre-LLM world we already had rumors/myths/lies/propaganda becoming "facts" after being reinforced through repetition without verification. (Relevant xkcd: Citogenesis https://xkcd.com/978/ .) Now, instead of authoritative sounding humans who are possibly wrong or with an agenda, we have an "infallible", "impartial" oracle capable of inventing whatever it wants to. Yes, I have seen ChatGPT treated as infallible…

Perhaps it's a good thing, everyone will have to become experts in discerning the truth. Much like Video Game graphics looked "real life" until the next gen came out, perhaps society will start to see through any rumours/myths/lies/propaganda. Either that or we descend into a state of anomie.

Doubt it. I can hardly verify things happening in my city even if I try, let alone e.g. things happening on the other side of the globe which the media keeps telling me about in order to drum up the next cold/hot war. I used to be able to tell Photoshop jobs by non-experts, but soon enough everyone and their mum will probably be able to produce completely fake videos indistinguishable from real ones, so even "video evidence" will mean little.

Re: ChatGPT is now finding bugs in databases

#153

Earlier quoted context omitted.

Can you give an example? I see this advice a lot but usually devoid of practical tips.

Here's an attempt. Looks pretty good to me. It's sort of like, how would you imagine an expert would respond if all you said was "simulate a discussion between a flow engineer and geologist". Without providing any further context or hints into what you want, I also wouldn't be surprised if I get a generic response. But instead if you sort of know what you want to ask, they'd have more to work with and be able to bett…

Oh yeah - and now I change this and suddenly the thing is fun and slightly unreal:

"Simulate a discussion between a project manager and a public health officer in the planning phases of a drill operation. Use field-specific terminology and aim for accuracy, the scenario will be used to teach a petroleum engineering class focused on public relations."

GPT: > Project Manager: Good morning, I'm glad to have you here to discuss the planning phases of our upcoming drill operation. As a project manager, my priority is to ensure that all aspects of the drill operation are conducted in a safe and efficient manner.

> Public Health Officer: Good morning. As a public health officer, my role is to ensure that the drill operation does not pose a risk to public health and the environment. I'll be happy to discuss our strategies for achieving that goal.

> Project Manager: Great. One of the first things we need to do is to identify potential health hazards associated with the drill operation. As you know, drilling involves the use of various chemicals, which can pose a health risk if not handled properly.

> Public Health Officer: Absolutely. We'll need to carefully review the chemicals being used and ensure that they are being handled in accordance with proper safety protocols. We'll also need to develop emergency response plans in case of accidental spills or leaks.

> Project Manager: Agreed. Another concern is air quality. The drilling process can generate dust and emissions, which can have adverse effects on air quality.

> Public Health Officer: That's right. We'll need to monitor air quality throughout the operation to ensure that it meets the standards set by the Environmental Protection Agency. We'll also need to take measures to minimize dust and emissions, such as using dust control equipment and properly maintaining all equipment.

> Project Manager: And what about water quality? We'll be drilling in close proximity to bodies of water, and we need to ensure that we don't contaminate them.

> Public Health Officer: Yes, that's a critical concern. We'll need to make sure that all wastewater and other fluids are properly contained and disposed of. We'll also need to monitor water quality regularly to ensure that there are no leaks or spills.

> Project Manager: It sounds like we have a lot of work to do to ensure that this drill operation is safe and environmentally responsible.

> Public Health Officer: Indeed. But by working together and following proper protocols and procedures, we can ensure that the operation is successful without posing a risk to public health or the environment.

I guess there was some confusion about roles/at the existing temperature roles swap (if you rerun it, you get different results, essentially swapping the lines...)

EDIT: and third try actually arrives at really nice hallucinations:

> Project Manager: Good morning, Dr. Smith. Thank you for taking the time to meet with me today. We are in the planning phases of a drill operation and we want to ensure that we are following all necessary public health protocols. Can you provide some guidance on what we should consider?

> Public Health Officer: Absolutely, it's my pleasure to help. The first thing you should consider is ensuring that all workers on-site are up-to-date with their immunizations, particularly for diseases such as tetanus and hepatitis.

> Project Manager: Understood. We require all our workers to have up-to-date immunizations before starting work, but we will double-check to ensure that all workers have the necessary immunizations.

> Public Health Officer: That's great to hear. In addition to immunizations, you should also consider providing training on proper hygiene practices, such as hand-washing and avoiding contact with contaminated materials.

Re: ChatGPT is now finding bugs in databases

#154
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

> ChatGPT can't access the external internet. This might not be entirely true. People were able to fetch urls and data from the internet that couldn't possibly be in any training dataset (in eg posted the same day for the first time). https://medium.com/@neonforge/i-knew-it-chatgpt-has-access-t...

This actually shows perfectly how convincing ChatGTP can be. The author is too eager to believe the output from ChatGTP, even though this should be a clear indicator something else is happening:

> But ChatGPT has rewritten my article in a different style, what a interesting finding…

The URL conveniently contains the topic of the blog-article, meaning ChatGTP can 'hallucinate' the contents, even without internet connection.

To show this, I tried to follow the same steps as @neonforge, but with a non-existing URL. Unfortunately `lynx` 'didn't work', but curl showed some interesting information.

`curl https://medium.com/@peter/learn-how-to-use-the-magic-of-terr...` returned some html. Instead of a 404 and a element containing just 'Medium' it hallucinated the following title tag: `Learn How to Use the Magic of Terraform Locals (Step-by-Step Guide) | by Peter | Medium`

I'm not saying that ChatGTP definitely has no access to internet, but so far I have not seen any proof or indication that it does.

Re: ChatGPT is now finding bugs in databases

#155

Earlier quoted context omitted.

> OpenAI handed this tool to people with effectively no guidance or instructions so that they can trade on the glow of misconceptions and harvest user data. When one logs on to Chat GPT, they get the following notices: > This is a free research preview. > Our goal is to get external feedback in order to improve our systems and make them safer. > While we have safeguards in place, the system may occasionally generate…

Did you read those statements? None of them say what the product is, what it does, or how to use it — which are the kind of thing one might call guidance and instructions. Those are just disclaimers to make sure you don’t hold OpenAI responsible for anything you do with this thing, or blame them for anything they do with your attempts.

They say what it is: "This is a free research preview. Our goal is to get external feedback in order to improve our systems and make them safer."

It is not a product. They open it so that public can contribute to help them.

They do not say the aim of this research.

Re: ChatGPT is now finding bugs in databases

#156
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

> ChatGPT can't access the external internet. This might not be entirely true. People were able to fetch urls and data from the internet that couldn't possibly be in any training dataset (in eg posted the same day for the first time). https://medium.com/@neonforge/i-knew-it-chatgpt-has-access-t...

I don't think this shows at all that ChatGPT can access the internet? It looks like ChatGPT was given a URL with a title and hallucinated a blog post based on that title. The author then brushes it off as 'written in a different style', when it just looks like a totally different article overall (or maybe it's testament to the fact that too many Medium articles are low quality and indistinguishable from the output of a LLM!)

If anything, this blog post is a perfect example about how people can put in whatever they want as an input and take the output as truth, without any rigorous approach about what would count as a true fact about the model.

Re: ChatGPT is now finding bugs in databases

#157
post #146

Earlier quoted context omitted.

It also can be prompted to read the contents of a webpage, I have used it to help me craft a regex expression for removing junk utilizing invoke-webrequest to parse out CPU models compatible with 22H2 upgrades from Microsoft(that page did not exist in 2021).

You mean it prompted you to go and get the contents of the page? Chatgpt does not access the internet when running. Bing chat does though.

No, I do not mean that. I provided it the URL and then asked it to use the contents of the CPU table from the invoke-request to strip away everything after the X and it not only did it, it even provided the example output of the array it modified.

Here is an excerpt from my chat:

" can you please modify the current array using the Microsoft listed CPUs in totality from the URL https://learn.microsoft.com/en-us/windows-hardware/design/mi...

Certainly! Here's the updated $compatibleCPUs array based on the list of supported Intel processors for Windows 11 version 21H2 from Microsoft's documentation:

"

Re: ChatGPT is now finding bugs in databases

#158

Earlier quoted context omitted.

That's almost every reddit thread I've been involved in. Oratory prowess is way more important than accuracy. If you ever want to "lose a debate" on reddit, pick something that's counter-intuitive or widely misunderstood and use nuance and citations in your defense. You'll get cyberbullied every time - sometimes even banned from the sub.

It’s amazing to me in all my years on the internet how many communities I insert myself into, then in private feel like a pretender, and yet how relatively uncommon it is that people call me an idiot.

Good afternoon, fellow Imposter Syndrome Haver :) Just jumping in to say that you are most likely not a pretender or an idiot, you're probably just very self-conscious, hold yourself to high standards and give yourself lofty goals.

Re: ChatGPT is now finding bugs in databases

#159

Here is an example of ChatGPT being confidently wrong in something domain specific that I know about. Here's what I asked: "I'm playing Old School Runescape. I want to efficiently level up my Mining. I'm currently level 90 and I want to do a tick-manipulating method. Something like 3t4g. I don't understand when to click and what I need. Can you explain the method?" Here's the method it replied minus kruft at the star…

> Obviously it's not understanding anything and it's forming sentences that sound natural in the context of the question > To me this is an example of being very cautious of trusting what it says when you ask it about something you don't know about. More and more like humans all the time

Humans who don’t understand what they are talking about we should not listen to nor trust. By the same logic it would be stupid to listen to / trust ChatGPT.

Re: ChatGPT is now finding bugs in databases

#160

Earlier quoted context omitted.

Look at the table of contents of this book titled "Eugenics" https://archive.org/details/naturessecretsre00shan/page/6/mo... There were multiple points of interest for the Eugenicists. One of them was for the general public and how to get them to have a proscribed conduct and behavior. Maybe you've seen those diagrams where they have 2 life paths depicted from the Victorian era. That's from Eugenics. Example: https:/…

Manners and conduct existed before eugenics and continues to exist after eugenics. You can't pull one blighted moment in the 1920s as proof the whole idea is faulty. There is little accuracy in your description of events. Peterson said we share serotonin with lobsters and it fuels a victory/defeat set of behaviours. If you drop the serotonin from his idea, it's no longer what he was talking about. Your description fa…

[deleted]
Post reply on HN