Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

111–120 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#111

Post by Eji1700 »

LeftHandPath wrote: post #97
Eji1700 wrote: post #71

Earlier quoted context omitted.

Glad i'm not the only one who saw this. Feels extremely overblown. Very impressive, don't get me wrong, and is yet another bridge closer to letting business side do plain text queries. But that said, stuff like this has existed for awhile, and it's certainly not replacing analysts. For a start, this assumes a VERY clean model, which is of course what you strive for, but the moment you step out of FAANG you'll find th…

Still, while it might not be useful yet , can’t it stand its own as an intriguing proof of concept? The first car was hardly ready to replace the horse. But look where we are now.

Sure, if that was the title of the article. Instead its saying its ready to replace a full time position, which its not even close to.

And this is something people have been saying about things like AI and crypto for years now. They've both made some interesting developments, and they both face serious hurdles (some fundamental).

AI obviously has a ton of real world use cases, but overhyping things like this doesnt actually help.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#112

Post by Micoloth »

TeMPOraL wrote: post #87
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

I've done too little SQL to be traumatized by it, but I experienced it plenty of times doing more complex data processing on the backend. I would e.g. spend a week on first discussing a solution, then researching both theoretical background and relevant libraries, and finally writing code to do the math and render the answers (probability distributions). I eyeball them, they seem to all match expectations, so I decla…

That’s a great writeup of what it’s like to learn to do this job ahaha.

Everbody goes through this I think (Arguably I’m still going through this…)

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#114

Post by burnished »

ResearchCode wrote: post #100
nerpderp82 wrote: post #83

Earlier quoted context omitted.

> ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed You can give it the datamodel and a semantic description and it will do a pretty good job of reprojecting your data. People talk about chatgpt as being a fixed thing, it isn't. ChatGPT is the Apple II of LLMs. 3-4 moore units from now every issue that people think is a "gotc…

Still waiting for those autonomous cars.

They are already here. We can haggle over error rate and incidents per million miles or what have you but that is more an argument about safety and suitability.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#115

Post by speedgoose »

pmontra wrote: post #113

I understand that this is a demo and it's goal is estimating how good the AI could become in future. However given that we still need a SQL analyst to engineer the prompt, did ChatGPT save time to the analyst or increased the amount of work?

It’s likely going to save time on the long term.

https://xkcd.com/1205/

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#116

Post by fifilura »

TeMPOraL wrote: post #87
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

I've done too little SQL to be traumatized by it, but I experienced it plenty of times doing more complex data processing on the backend. I would e.g. spend a week on first discussing a solution, then researching both theoretical background and relevant libraries, and finally writing code to do the math and render the answers (probability distributions). I eyeball them, they seem to all match expectations, so I decla…

I feel the pain.

Programming back-end code in an IDE takes you very far from working with the numbers on front of you.

Is it possible to prototype in notebooks and add the results and input to unit tests?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#117

Post by burnished »

knodi123 wrote: post #46
LeanderK wrote: post #26

I love those ChatGPT projects! Of course it's silly and nobody is really replacing somebody with a program that confidently get half it's answers wrong. But it's fun to just naively let ChatGPT solve the problem. But I wonder what it's going to look like in a few years. Currently, it's really just a demo that got surprisingly huge traction. I think the most pressing problem is not to get ChatGPT smarter but to get it…

> nobody is really replacing somebody with a program that confidently get half it's answers wrong I gotta tell you, I use copilot to help with my coding, and it still sends a shiver down my spine when it writes the entire database migration based off of the filename, or writes 4 pages of automated tests that work right on the first try.

Do you check that those tests can fail at all? That seems like a common human error, might be present here?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#118

Post by throwaway123--4 »

boredemployee wrote: post #79
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

>> The best fix imo is to slather a battery of views over your data to minimise the risk of getting the joins wrong, and it'd be interesting to see how that approach could improve the bot's quality. Yes, the success of the bot's query (at least on GPT-3) is a combination of (a literal and good) prompt and how well did you explain each of the columns to the model.

[flagged]

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#119

Post by hinkley »

treeman79 wrote: post #67
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

20 years ago I Knew a sql analyst that would write his tests ahead of time. Give X data, report should be Y. He would then figure out the SQL. He Didn’t know any other programming. This was before TDD took off. Blew my mind.

People dismissed the XP book by saying it was just stating practices that already existed. Articulating and selling an idea is a huge part of the challenge. Fifteen years later if you're having any success at all with Scrum it was because your team adopted the half of XP that didn't outright disagree with Scrum.

I liked the Refactoring book because it gave me names for things I figured out in 1996 (unfortunately while using VI as an editor, gave myself RSI in the process). It was part of an education in the value of books that help you only by existing so you can hand them to people who ask a lot of questions.

I had a boss who was big in the Kanban 'movement'. We got along famously, in part because I re-invented a Kanban board in 1994 when faced with a convenient whiteboard for the first time.

You can do a lot of amazing things with decent first principles. You can make some amazingly large messes without them.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#120

Post by mjburgess »

kietay wrote: post #106
nerpderp82 wrote: post #83

Earlier quoted context omitted.

> ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed You can give it the datamodel and a semantic description and it will do a pretty good job of reprojecting your data. People talk about chatgpt as being a fixed thing, it isn't. ChatGPT is the Apple II of LLMs. 3-4 moore units from now every issue that people think is a "gotc…

In your analogy to the Apple II, is there a fundamental problem people claimed for early computers, which didn’t just boil down to “they need to be faster”? There are fundamental limitations to what an LLM can achieve.

You missed the ever-powerful ellipsis in what he said:

Today "..." and then it will be solved.

It's the most powerful operator in the whole of applied computer science.

Post reply on HN