Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

131–140 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#131

Post by kieselguhr_kid »

ethbr0 wrote: post #86
ryanjshaw wrote: post #44

Earlier quoted context omitted.

Maybe 5% of the business/operations people I've worked with would even want to do this. The rest prefer a smart human to walk them through building a report - most of the time this is because they don't actually know what they need, and they actually need an expert to sit down and figure it out with them.

The longer I'm in the business, the more I feel like my value as a software developer is identifying and asking clarifying questions.

When I was new, I saw one of my more experienced colleagues ask a few questions that together saved the company more than $1 mil each year. ChatGPT might be a threat to automate some low-level tasks or help eliminate bugs, but it is nowhere near ready to evaluate the context of a system, understand its history, or think* through the consequences of a major business decision.

* or think at all, in any meaningful way.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#132

Post by Shorel »

ethbr0 wrote: post #86
ryanjshaw wrote: post #44

Earlier quoted context omitted.

Maybe 5% of the business/operations people I've worked with would even want to do this. The rest prefer a smart human to walk them through building a report - most of the time this is because they don't actually know what they need, and they actually need an expert to sit down and figure it out with them.

The longer I'm in the business, the more I feel like my value as a software developer is identifying and asking clarifying questions.

True, so true.

We work as translators. We translate intentions into actual descriptions.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#133

Post by rafaelero »

Very clever application of GPT, thanks for sharing. For the more complex queries, I suspect Chain of Thought can help. Just ask the model to describe each step before writing the final query. Also, you can add self-consistency to step 5, which you are kind of already doing. Let it generate something like 20 corrected queries and then select the one that generates the most common result between all of them.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#134

Post by boredemployee »

TeMPOraL wrote: post #87
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

I've done too little SQL to be traumatized by it, but I experienced it plenty of times doing more complex data processing on the backend. I would e.g. spend a week on first discussing a solution, then researching both theoretical background and relevant libraries, and finally writing code to do the math and render the answers (probability distributions). I eyeball them, they seem to all match expectations, so I decla…

Lol, countless times that I ended up with the right solution in my head while taking a shower or the day after, because after chillin my mind found that the previous code was wrong.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#135

Post by kieselguhr_kid »

hospadar wrote: post #129

I wonder if/when we'll get comfortable with the errors that an AI like this makes. Certainly human analysts still make errors, and may be able to explain them (which I think LLMs would have a hard time doing), but what if the overall error rate is less than a human analyst? I imagine this is sort of similar to our comfort with self-driving cars - what if they make fewer dangerous mistakes than humans? Would we actual…

Your last question hints at another question we also need to explore: do or will humans apply the same level of scrutiny to automatic outputs than they do to human ones? I find we tend to treat them as more authoritative. Are we more likely to unthinkingly accept mistakes by a machine?

As a brief aside, I wonder if we'll see increasing rates of accidents as the rate of AI drivers rises. I think it's hard to think about those effects in a vacuum - they have fewer injury accidents _now_, but will they if they go from making up a trivial percent of traffic to 10%? 25%?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#136

Post by hn_user2 »

supernova87a wrote: post #125

Side/meta-question: Do you all think that GPT and such will see a pattern of usefulness starting with: 1) blatantly wrong but helping to train/give examples to the most rudimentary and beginning stages of people learning a task? (since that's what it's doing at the same time?) I.e. replacing low-skilled intro training, or more charitably, helping to make it possible for far more people to learn something with assista…

Meta-Meta question: Why is "actually worrisome" part of "more capable"?

If worrisome is because of job prospects for an existing career, my advice is to embrace new tech, not fear it. Just keeping up with things even if you don't use it right away is usually enough to stay relevant. Things don't really evolve that quickly, and so as long as you don't resist change completely things usually work out.

Heck even if you hate change, and don't want to change at all, there will be plenty of old legacy companies with the old fashioned manual reports needing a person in a chair to type out the SQL to make them. Just like there are still (good paying) jobs for those COBOL experts today.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#137

Post by ResearchCode »

burnished wrote: post #114
ResearchCode wrote: post #100

Earlier quoted context omitted.

Still waiting for those autonomous cars.

They are already here. We can haggle over error rate and incidents per million miles or what have you but that is more an argument about safety and suitability.

Those must be some big gotchas, because self-driving cars have not replaced a job here, yet.

Large language models show more promise for replacing non-programming office workers.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#138

Post by Shorel »

TillE wrote: post #72
mjburgess wrote: post #37

Earlier quoted context omitted.

In my experience, in many cases, the so-called "first draft" (of code, content, etc.) is more just helpful for a mental warm-up. It isn't actually suitable. The time it takes to write/code/etc. isnt in the typing, that's trivial. It's in modelling the problem (, audience) and solving it. ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the ques…

The similar thought I had yesterday was that ChatGPT is pretty good at generating sample code that doesn't otherwise exist. It can be really useful if you treat it like that. Of course I've also had it call functions and use constants which don't exist, and fail to correct itself even after repeated prompting. So even that only goes so far.

Imagine if ChatGPT (or any other competitor) had access to a compiler and a set of tests for the program, a la HackerRank.

You then give it the prompt and a lot of test cases to pass in order to consider the task complete.

I can see this concept being implemented and working before the end of the year.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#139

Post by jtdev »

jeremyjh wrote: post #73
rezonant wrote: post #65

While this is very cool, SQL was designed to be used by business people. We need to go back to that model, where we train the business people who need these analytics how to use SQL to uncover the result. That along with a rigorous policy for including the queries that produced the result so the query logic can be checked would go a long way to actually taking advantage of the data we're collecting as businesses.

Sorry friend, but this is a bit out of touch. Maybe that was the original design intent of SQL, but understanding the application's data model is beyond a lot of sql analysts, much less their business partners.

[dead]

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#140

Post by RhodesianHunter »

kieselguhr_kid wrote: post #131
ethbr0 wrote: post #86

Earlier quoted context omitted.

The longer I'm in the business, the more I feel like my value as a software developer is identifying and asking clarifying questions.

When I was new, I saw one of my more experienced colleagues ask a few questions that together saved the company more than $1 mil each year. ChatGPT might be a threat to automate some low-level tasks or help eliminate bugs, but it is nowhere near ready to evaluate the context of a system, understand its history, or think* through the consequences of a major business decision. * or think at all, in any meaningful way.

I agree with you to a point, but I think the only reason that it can't understand the context of a system is because it hasn't been trained on that system's code and documentation, which is obviously a future coming soon.
Post reply on HN