Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT is now finding bugs in databases

celerdata.com

Post reply on HN
Print view View original

141–150 of 204 posts

Re: ChatGPT is now finding bugs in databases

#141

Post by raspyberr »

Here is an example of ChatGPT being confidently wrong in something domain specific that I know about. Here's what I asked:

"I'm playing Old School Runescape. I want to efficiently level up my Mining. I'm currently level 90 and I want to do a tick-manipulating method. Something like 3t4g. I don't understand when to click and what I need. Can you explain the method?"

Here's the method it replied minus kruft at the start and end: " 1. Equip your pickaxe and head to the location where you want to mine.

2. Find a 3x3 square of rocks that you want to mine. The square should be in a straight line in front of you, with the rock you want to mine in the middle.

3. Stand in the square so that your character is in the middle of the square, facing the rock you want to mine.

4. Start mining the rock as normal.

5. As soon as you see the animation of your pickaxe hitting the rock, immediately right-click on the rock and select "Prospect". This will cancel the animation of your pickaxe hitting the rock and allow you to perform another action.

6. As soon as you see the prospect animation start, immediately right-click on the rock and select "Mine". This will cancel the prospect animation and allow you to perform another action.

7. Repeat steps 5 and 6 as quickly as possible.

8. Every fourth action, switch to a different rock in the 3x3 square. "

This is total nonsense. Not only is this not 3t4g but the actions it says don't work like that in the game. Obviously it's not understanding anything and it's forming sentences that sound natural in the context of the question. OSRS has one of the best most complete wikis of any game. To me this is an example of being very cautious of trusting what it says when you ask it about something you don't know about.

Re: ChatGPT is now finding bugs in databases

#142

Post by fallingknife »

raspyberr wrote: post #141

Here is an example of ChatGPT being confidently wrong in something domain specific that I know about. Here's what I asked: "I'm playing Old School Runescape. I want to efficiently level up my Mining. I'm currently level 90 and I want to do a tick-manipulating method. Something like 3t4g. I don't understand when to click and what I need. Can you explain the method?" Here's the method it replied minus kruft at the star…

> Obviously it's not understanding anything and it's forming sentences that sound natural in the context of the question

> To me this is an example of being very cautious of trusting what it says when you ask it about something you don't know about.

More and more like humans all the time

Re: ChatGPT is now finding bugs in databases

#143

Post by B1FF_PSUVM »

raspyberr wrote: post #141

Here is an example of ChatGPT being confidently wrong in something domain specific that I know about. Here's what I asked: "I'm playing Old School Runescape. I want to efficiently level up my Mining. I'm currently level 90 and I want to do a tick-manipulating method. Something like 3t4g. I don't understand when to click and what I need. Can you explain the method?" Here's the method it replied minus kruft at the star…

ChatGPT is a bullshit generator.

Earlier this week I was discussing with family why "you should not freeze things twice". Someone made the mistake of asking ChatGPT, and it confidently pooped out that bacteria keep reproducing in frozen food!

(The actual reason is that freezing does not actually kill bacteria already present - cooling/freezing stops the growth, but it is cumulative. So if you have stuff out, then re-freeze, then out again, the second time it may reach toxic levels.)

P.S. https://www.theregister.com/2023/03/02/chatgpt_considered_ha... - fake obituary link and all ...

Re: ChatGPT is now finding bugs in databases

#144

Post by stevenhuang »

IIAOPSW wrote: post #139
stevenhuang wrote: post #92

Earlier quoted context omitted.

you need to prompt it with more specific knowledge; if you're generic in your prompts it will be generic in its responses. inject some field-specific terminology and be esoteric in your queries, etc its latent representation is vast, I find you can get much higher quality+interesting responses if the prompts are tuned appropriately

This is at best going to slightly improve the output. I asked about something hyper specific and also extremely trivial: transit wayfinding type queries specific to the NYC subway. All the semantics are services and stations. A service is nothing more than a list of stations it stops at and a station is nothing more than a list of services which stop at it. Eg if I tell it "I'm on the F, how do I get to Time Sq", it…

You're right, it seems particularly bad in navigation. I lead with some general queries on the NYC system, asked it for the common stops for service lines F and A, and it also hallucinated. The wiki for the service lines have pretty complete representations here, so this data should be suitably represented in its training corpus.

Would be interested in how a multimodal LLM such as PaLM-E (trained on maps, etc) fair in these sorts of queries.

https://www.reddit.com/r/MachineLearning/comments/11krgp4/r_...

Re: ChatGPT is now finding bugs in databases

#145

Post by Angostura »

raspyberr wrote: post #141

Here is an example of ChatGPT being confidently wrong in something domain specific that I know about. Here's what I asked: "I'm playing Old School Runescape. I want to efficiently level up my Mining. I'm currently level 90 and I want to do a tick-manipulating method. Something like 3t4g. I don't understand when to click and what I need. Can you explain the method?" Here's the method it replied minus kruft at the star…

If you want a simpler example - try asking it for a list of english words where the letter V is silent.

On the other hand, I do find it quite useful for suggesting improvements to things. You can't rely on the suggestions, but they can be thought provoking.

Re: ChatGPT is now finding bugs in databases

#146

Post by IanCal »

honeybadger1 wrote: post #40
nextaccountic wrote: post #13

Earlier quoted context omitted.

> ChatGPT can't access the external internet. ChatGPT by itself can't but if you build a product on top of its API, you can fetch the url itself and send the data in the ChatGPT's prompt

It also can be prompted to read the contents of a webpage, I have used it to help me craft a regex expression for removing junk utilizing invoke-webrequest to parse out CPU models compatible with 22H2 upgrades from Microsoft(that page did not exist in 2021).

You mean it prompted you to go and get the contents of the page? Chatgpt does not access the internet when running. Bing chat does though.

Re: ChatGPT is now finding bugs in databases

#147

Post by gonzo41 »

paxys wrote: post #19
crazysim wrote: post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

Same effect happens with newspapers. Check your sources.

Re: ChatGPT is now finding bugs in databases

#148

Post by theonlybutlet »

soco wrote: post #137
danielvaughn wrote: post #56

Earlier quoted context omitted.

I think a lot of overly-eager and under-informed people are jumping on the bandwagon and shouting half-baked proclamations from the rooftops to get clicks and views. But underneath all the bullshit is something truly useful, so I wouldn't necessarily say that Confidently Incorrect™ is ChatGPT's primary ability. Coincidentally, over the past few weeks of this ChatGPT craze, I've needed to create a lot of fake data to…

Like said somewhere on the internet, we don't need AI to create art, we need AI to do the dishes and free our time to create art.

But art's the easy part, I wonder if that's why it's fun...

Re: ChatGPT is now finding bugs in databases

#149

Post by theonlybutlet »

oefrha wrote: post #53
paxys wrote: post #19

Earlier quoted context omitted.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

In the pre-LLM world we already had rumors/myths/lies/propaganda becoming "facts" after being reinforced through repetition without verification. (Relevant xkcd: Citogenesis https://xkcd.com/978/ .) Now, instead of authoritative sounding humans who are possibly wrong or with an agenda, we have an "infallible", "impartial" oracle capable of inventing whatever it wants to. Yes, I have seen ChatGPT treated as infallible…

Perhaps it's a good thing, everyone will have to become experts in discerning the truth. Much like Video Game graphics looked "real life" until the next gen came out, perhaps society will start to see through any rumours/myths/lies/propaganda. Either that or we descend into a state of anomie.

Re: ChatGPT is now finding bugs in databases

#150

Post by barrysteve »

kristopolous wrote: post #98
pfannkuchen wrote: post #88

Earlier quoted context omitted.

I’m no fan of Peterson, but I’m curious— how are intentionally altering the gene pool of a group and cleaning your room colinear?

Look at the table of contents of this book titled "Eugenics" https://archive.org/details/naturessecretsre00shan/page/6/mo... There were multiple points of interest for the Eugenicists. One of them was for the general public and how to get them to have a proscribed conduct and behavior. Maybe you've seen those diagrams where they have 2 life paths depicted from the Victorian era. That's from Eugenics. Example: https:/…

Manners and conduct existed before eugenics and continues to exist after eugenics. You can't pull one blighted moment in the 1920s as proof the whole idea is faulty.

There is little accuracy in your description of events. Peterson said we share serotonin with lobsters and it fuels a victory/defeat set of behaviours. If you drop the serotonin from his idea, it's no longer what he was talking about.

Your description falls to my mind as a velvet blanket to hide underneath from scary racist and nazi ideas that have nothing to do with Victorian era manners and metaphors about behaviour, expanded out of natural science.

You can't draw a 'perimeter' around the wind.

Post reply on HN