Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT is now finding bugs in databases

celerdata.com

Post reply on HN
Print view View original

161–170 of 204 posts

Re: ChatGPT is now finding bugs in databases

#161

Post by nqrac »

kristopolous wrote: post #82
smogcutter wrote: post #73

Earlier quoted context omitted.

From those examples, I think you’re maybe also running up against people generally having a low threshold for giving a stranger on the internet the benefit of the doubt. It’s hard to be surprised that “eugenics, but the good kind, just hear me out!” and “state’s rights, but not the disingenuous lost cause kind, just hear me out!” are a tough sell with complete strangers who have no prior reason to trust you or be int…

correct! Btw, I removed them from my examples, didn't want to "discuss" that here. I'm kind of a history nerd and it's like a fishing hook with delicious bait when I see stuff like that. "Oh wait! You think it was just the south that argued states rights?! Actually (brings up fugitive slave clause jurisprudence)" Or how manner and conduct books of the 1800s, eugenics texts, and jordan peterson's 12 rules for life are…

Peterson an academic who is clueless about the real world. His best work was discussing politics in connection with liberty, especially the Trucker protests.

His biggest flaw is that (like the other Daily Wire pundits) he never goes into economics and has absurd ideas that we live in a competence hierarchy.

The whole "intellectual dark web" imploded after the start of the Ukraine war, when they first went full neocon, then realized that their audience does not like that and now start backpedaling massively.

Re: ChatGPT is now finding bugs in databases

#162

Post by joshschreuder »

Angostura wrote: post #145
raspyberr wrote: post #141

Here is an example of ChatGPT being confidently wrong in something domain specific that I know about. Here's what I asked: "I'm playing Old School Runescape. I want to efficiently level up my Mining. I'm currently level 90 and I want to do a tick-manipulating method. Something like 3t4g. I don't understand when to click and what I need. Can you explain the method?" Here's the method it replied minus kruft at the star…

If you want a simpler example - try asking it for a list of english words where the letter V is silent. On the other hand, I do find it quite useful for suggesting improvements to things. You can't rely on the suggestions, but they can be thought provoking.

One of the words it gave me was “Yclept (archaic form of "called" or "named")” which is both silent and invisible!

Re: ChatGPT is now finding bugs in databases

#163

Post by simonw »

password1 wrote: post #151
crazysim wrote: post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

> ChatGPT can't access the external internet. This might not be entirely true. People were able to fetch urls and data from the internet that couldn't possibly be in any training dataset (in eg posted the same day for the first time). https://medium.com/@neonforge/i-knew-it-chatgpt-has-access-t...

That was all hallucination.

If you want to prove this to yourself, tail a server log and paste a URL into ChatGPT and see if you get a hit.

Re: ChatGPT is now finding bugs in databases

#164

Post by simonw »

derefr wrote: post #12
crazysim wrote: post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

It can't access the internet now . But it was fed a snapshot of the Internet during training. So if you tell it a URL, it can potentially remember what the page looked like, if it happened to see it during training.

That's very unlikely. ChatGPT doesn't contain a full copy of everything it saw when it was trained - it has model weights that were updated by that training process.

It's very unlikely the model weights would contain enough detail to let it match content to the URL that the content originally came from.

Re: ChatGPT is now finding bugs in databases

#165

Post by chrismorgan »

password1 wrote: post #151
crazysim wrote: post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

> ChatGPT can't access the external internet. This might not be entirely true. People were able to fetch urls and data from the internet that couldn't possibly be in any training dataset (in eg posted the same day for the first time). https://medium.com/@neonforge/i-knew-it-chatgpt-has-access-t...

That article is bunkum. There’s not the slightest reason to believe that it “rewrote his article in a different style”: it hallucinated a completely different article based on the URL. The other “proof” is all clearly wildly inconsistent and hallucinated: it’s a desktop machine, a server, inside Google in I think France, in the USA, in AWS, in Germany, and I think the last couple of “tests” don’t make sense anyway.

It convinced him of what he wanted to believe. He didn’t do a very good job of testing.

Re: ChatGPT is now finding bugs in databases

#166

Post by simonw »

honeybadger1 wrote: post #157
IanCal wrote: post #146

Earlier quoted context omitted.

You mean it prompted you to go and get the contents of the page? Chatgpt does not access the internet when running. Bing chat does though.

No, I do not mean that. I provided it the URL and then asked it to use the contents of the CPU table from the invoke-request to strip away everything after the X and it not only did it, it even provided the example output of the array it modified. Here is an excerpt from my chat: " can you please modify the current array using the Microsoft listed CPUs in totality from the URL https://learn.microsoft.com/en-us/window…

It hallucinated the list of processors based on the URL.

Here's an experiment I just did to prove that it hallucinates content based on URLs: https://fedi.simonwillison.net/@simon/109998914078891574

Or try it yourself: Try these two prompts in separate ChatGPT sessions:

> List the processors on this page: https://bit.ly/422t51T0

Then:

> List the processors on this page: https://learn.microsoft.com/en-us/windows-hardware/design/mi...

That Bitly link is one I just created that redirects to the Microsoft page.

Re: ChatGPT is now finding bugs in databases

#167

Post by simonw »

IanCal wrote: post #146
honeybadger1 wrote: post #40

Earlier quoted context omitted.

It also can be prompted to read the contents of a webpage, I have used it to help me craft a regex expression for removing junk utilizing invoke-webrequest to parse out CPU models compatible with 22H2 upgrades from Microsoft(that page did not exist in 2021).

You mean it prompted you to go and get the contents of the page? Chatgpt does not access the internet when running. Bing chat does though.

Even Bing can't make outbound HTTP requests - it can only access cached snapshots of pages held in the Bing search index. I confirmed this on Twitter with one of the Bing execs: https://twitter.com/mparakhin/status/1628646262890237952

Re: ChatGPT is now finding bugs in databases

#168

Post by zoklet-enjoyer »

kragen wrote: post #114
zoklet-enjoyer wrote: post #94

Earlier quoted context omitted.

Some of us from the forum totse emailed local news stations, schools, and police departments pretending to be concerned parents. I wrote my high school and local news and said that my child was using jenkem and linked to an article from a local news station from somewhere else in the country to kind of back up my claims. https://www.snopes.com/fact-check/jenkem/

Wow, so jenkem was largely a TOTSE creation? Do you have any idea how the original stories from Africa got started — were there TOTSE members in Zambia?

There's a BBC article from 1999 about jenkem in Africa. It's messed up and sad. Someone saw that and turned it into a meme. Jenkem in America was a totse creation

http://news.bbc.co.uk/2/hi/africa/406067.stm

Re: ChatGPT is now finding bugs in databases

#169

Post by alophawen »

kristopolous wrote: post #66
paxys wrote: post #19

Earlier quoted context omitted.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

That's almost every reddit thread I've been involved in. Oratory prowess is way more important than accuracy. If you ever want to "lose a debate" on reddit, pick something that's counter-intuitive or widely misunderstood and use nuance and citations in your defense. You'll get cyberbullied every time - sometimes even banned from the sub.

Can confirm. Was banned from /r/golang for speaking well of Rust. I use both.
Post reply on HN