Live data from Hacker News

ChatGPT is now finding bugs in databases

celerdata.com

121–130 of 204 posts

Re: ChatGPT is now finding bugs in databases

#121
post #19
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

> There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect.

Sounds like the old days to me, where we would meet up , discuss the world and what have you, without any means to fact check anything online.

Changes there was a real expert on the subject in your circle was at best 20% So we just enjoyed / continue our life. No harm done as long as nobody took the discussion as gospel.

Re: ChatGPT is now finding bugs in databases

#122
post #19
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

When I've asked ChatGPT about things I know a little about I find its response is often wrong. So much so that I always respond with "Are you sure that's correct?" it then often responds "I'm sorry you are right, I should have said... " and then says something a little closer to the truth.

I wouldn't rely on ChatGPT or any system that relies on it, except perhaps for summarising/rephrasing a single, known document. It worries me that people are using it for software development - creating stubs, bootstrap code. It's perhaps OK if you know the language/platform well enough to fix issues, but if you then use it on something that your not an expert in, what is the outcome.

For example:

Finding the source of literary quotes - miss attributing them - took 5 or 6 QA responses for it to get close.

Explaining how a p-channel enhancement mode MOSFET works - explained n-channel

Explain regex - often gets the technical parts right but can make very bad suggestions on what the regex aim is.

Name some classic cocktails created in Europe - suggested Manhattan, which it said was acceptable because it was popular in Europe

Suggest some gin based cocktails that don't include citrus, suggested those with lemon juice.

Re: ChatGPT is now finding bugs in databases

#123

Earlier quoted context omitted.

The concern is the increasing number of anecdotes where people are taking ChatGPT's output at face value, even though everyone knows by now that it does sometimes get stuff wrong and does hallucinate. I thought it would take longer.

A very annoying trend I've already started noticing on some forums (including here unfortunately) is someone asking a question and one of the replies is someone saying "I put your question into ChatGPT and it said..."

I downvote and flag every chatgpt generated comment, even if they’re up front about it

If I want to know what chatgpt thinks about something, I can ask it myself

Re: ChatGPT is now finding bugs in databases

#124
post #19

Earlier quoted context omitted.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

I think a lot of overly-eager and under-informed people are jumping on the bandwagon and shouting half-baked proclamations from the rooftops to get clicks and views. But underneath all the bullshit is something truly useful, so I wouldn't necessarily say that Confidently Incorrect™ is ChatGPT's primary ability. Coincidentally, over the past few weeks of this ChatGPT craze, I've needed to create a lot of fake data to…

When I want a quick insight to launch me into further inquiry, chatgpt has been so much more useful than regular search with the first 10 hits of AI generated advertisement spam. I use one that provide its source as well, so I can check.

Reddit tends to be a second best bet, and the most upvoted is not always the best, on a busy thread some ‘real’ experts are usually a bit down with lengthy descriptions of their analysis.

Search is third, it’s quick but I do often need to append stackoverflow and reddit to get meaningful answers. I forgo Google completely more and more since the quality is so low, perhaps 2% of my queries go there tops.

Re: ChatGPT is now finding bugs in databases

#125
post #19
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

I compared ChatGPT to a middle tech manager that just heard he has to create a last minute presentation. He knows all the current buzz words to sounds smart.

Re: ChatGPT is now finding bugs in databases

#126
post #3

ChatGPT can't access the external internet. The example ChatGPT instruction where they "teach it" just has the user giving it a URL to their docs and ChatGPT is like "oh ok got it" (Figure 3). I'm not sure if the blog post writers are aware it's hallucinating what the JSON implementation is and hasn't actually read the documentation at all.

[dead]

Re: ChatGPT is now finding bugs in databases

#127

Earlier quoted context omitted.

you need to prompt it with more specific knowledge; if you're generic in your prompts it will be generic in its responses. inject some field-specific terminology and be esoteric in your queries, etc its latent representation is vast, I find you can get much higher quality+interesting responses if the prompts are tuned appropriately

Can you give an example? I see this advice a lot but usually devoid of practical tips.

Here's an attempt. Looks pretty good to me.

It's sort of like, how would you imagine an expert would respond if all you said was "simulate a discussion between a flow engineer and geologist". Without providing any further context or hints into what you want, I also wouldn't be surprised if I get a generic response. But instead if you sort of know what you want to ask, they'd have more to work with and be able to better answer your query.

Note I know nothing about this field, I just looked at https://en.wikipedia.org/wiki/Engineering_geology#Soil_and_r... and understood this was for oil and gas, so:

> Simulate a discussion between a flow engineer and a geologist in the planning phases of a drill operation. Use field-specific terminology and aim for accuracy, the scenario will be used to teach a petroleum engineering class focused on deformation monitoring.

Flow Engineer: Good morning, I'm the flow engineer for this drill operation. Nice to meet you.

Geologist: Hi, I'm the geologist in charge of the geological analysis for this project.

Flow Engineer: Great, I understand we need to work together to ensure the success of this drill operation. Can you tell me more about the geological features of the area we're drilling in?

Geologist: Sure. We're drilling in an area with significant deformation, caused by tectonic activity. There are a variety of rock formations, including sandstone, shale, and limestone. There's also a possibility of encountering fault zones, which could pose a risk to the drilling operation.

Flow Engineer: Thanks for the information. We'll need to take into account the deformation and rock properties when designing the wellbore trajectory. Have you conducted any geomechanical analysis?

Geologist: Yes, we've conducted a rock mechanics study to determine the stress and strain characteristics of the formations in the area. Based on the results, we recommend using a multilateral well design with a horizontal section to avoid encountering the fault zones and minimize the impact of deformation.

Flow Engineer: That's helpful. We'll need to ensure that our drilling fluid properties are appropriate for the formations we'll be encountering. Can you tell me more about the porosity and permeability of the rock formations?

Geologist: The porosity and permeability of the sandstone formations vary, but in general, they're relatively high, which means we'll need to be careful when selecting our drilling fluid to avoid fluid loss into the formation. The shale formations are less porous and permeable, which means we'll need to use a different type of fluid to avoid damaging the formation.

Flow Engineer: Thanks for the information. We'll take that into account when selecting our drilling fluid. We'll also need to monitor deformation during the drilling process. Do you have any recommendations for deformation monitoring tools we should use?

Geologist: Yes, we recommend using acoustic emission sensors to monitor rock fracturing and deformation, as well as tilt sensors to monitor any changes in the wellbore inclination. We also recommend conducting periodic borehole imaging to ensure that the wellbore is remaining stable.

Flow Engineer: Excellent. We'll ensure that we incorporate those tools into our drilling plan. Thank you for your time and expertise. I'm looking forward to working with you on this project.

Re: ChatGPT is now finding bugs in databases

#128
post #103
post #62

Earlier quoted context omitted.

While I agree with your premise, the "toil" that you mention is a very small use case for a tiny minority of people. And automating it away has always been trivial. What would have taken 30 minutes to write a Python script now takes 5 minutes to come up with the perfect prompt. And then you have to test and be sure that it does what you want. Neat, but not world-changing. Meanwhile ChatGPT is already being hailed as…

> Meanwhile ChatGPT is already being hailed as the Google killer. The only Google-killing aspect of it is a better understanding of user queries. Although given how often I have to remind it of the details of my questions, maybe it's not even good at that.

Given it has probably "hardcoded" a lot of questions by the usage of "finetuning" via RL, probably the latter statement is true. Also it has one way only to understand your query. If that way is wrong: welcome synonyms replaced by the tokenizer, welcome hallucinations by raising the temperature. Or you can introduce "context" (8000 tokens) or retrain.

Re: ChatGPT is now finding bugs in databases

#129

Earlier quoted context omitted.

This one is different because it will pass the threshold of "normality". Just as we woke up with DALL. E 2 one morning in April 2022 and with ChatGPT one morning in November 2022, one day in the future, perhaps in the next 3-5-10 years, we will wake up with some 5, 10, 100 terabytes of neural weights fully able to drive a car from a single camera under any circumstance, millions of times safer than a perfect human dr…

This is like saying in 1700 that the invention of farming automation will displace 90%+ of farmers, what ever will they do? Automation is helpful, but it’s also scary. I guarantee that nobody 300 years ago thought that many of the 90%+ of displaced farmers would end up sitting in front of screens, moving data around. And I also guarantee that farmers in the 1700s were just as tired and sick as the driving industry to…

The point is that there will be no more "new jobs": once we have successfully managed to externalize in silico basic human decision-making (accelerate, brake, make a left, make a right in 2/3D space, but also decisions in other spaces: decisions in the space of language, of images, and so forth) it's game over for 99+% of all the jobs across all the industries. All that remains are the entertainers (very few), the brains (even fewer), and the owners (even fewest) †.

The transition will not be scary: it will be criminal. The governments and general policy decisional factors are so out of touch it's laughable, just look at the French government as they try to raise the retirement age to 64 [1].

† Because this is the crux of the matter: are we able to build a civilization where everyone is an owner, or will only the few of the fewest be the owners. And today, for instance, only those with 50 million USD and above are owners [2], because first of all you must own your life, and if you can't afford a lifesaving $3.5 million gene therapy [3] if you need it (not to mention a 100 USD eye surgery), you don't own your life.

[1] https://www.france24.com/en/france/20230309-french-young-peo...

[2] https://en.wikipedia.org/wiki/Ultra_high-net-worth_individua...

[3] https://www.webmd.com/a-to-z-guides/news/20230215/life-savin...

Re: ChatGPT is now finding bugs in databases

#130
post #19

Earlier quoted context omitted.

There's this internet phenomenon (common on Reddit and other such forums) where you find high-quality discussions on a variety of topics and everyone seems so clever and informative, until you come across a thread on a topic that you are actually an expert on and find that every top-voted comment is objectively incorrect. ChatGPT is essentially that but on steroids. Its primary ability is being able to sound extremel…

I think a lot of overly-eager and under-informed people are jumping on the bandwagon and shouting half-baked proclamations from the rooftops to get clicks and views. But underneath all the bullshit is something truly useful, so I wouldn't necessarily say that Confidently Incorrect™ is ChatGPT's primary ability. Coincidentally, over the past few weeks of this ChatGPT craze, I've needed to create a lot of fake data to…

[deleted]
Post reply on HN