Replacing a SQL analyst with 26 recursive GPT prompts • Hacker News Board

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#11

Post by dmitriid » Wed, Jan 25, 2023, 5:58 PM UTC

logicallee wrote: post #4

kilotaras wrote: post #2

Interesting. Probably won't work for harder queries, but would be a good tool to make simpler queries, or parts of harder ones.

my opinion: probably will work for harder queries, because GPT is a form of general intelligence. (At least ChatGPT is. It is a much more advanced form of intelligence than davinci-003, which the poster used.)

It's not even close to being a form of general intelligence. It's an advanced language model trained on a large set of data. It gets confused for even the simplest of things, especially code.

You can trick it into working more-or-less correctly on simple sets and simple tasks, but in the wrong run it will be more wrong than correct.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#12

Post by xeyownt » Wed, Jan 25, 2023, 5:59 PM UTC

Wow, this looks so fun to play with.

As pointed out in the blog post, the iterative process is very close to the mental process an analyst would follow. This is the beginning of an AI actually thinking ^^

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#13

Post by jamiequint » Wed, Jan 25, 2023, 5:59 PM UTC

I wonder how much more accurate this would get if fine tuned on a set of SQL problems? Could even fine tune it on a per-company basis using queries that had been written by analysts in the past.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#14

Post by LastTrain » Wed, Jan 25, 2023, 5:59 PM UTC

jeltz wrote: post #6

kilotaras wrote: post #2

Interesting. Probably won't work for harder queries, but would be a good tool to make simpler queries, or parts of harder ones.

Yeah, it doesn't. A friend of mine played around with this and on trickier questions you got confidently incorrect answers.

That is almost worse than it completely not working.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#15

Post by berkle4455 » Wed, Jan 25, 2023, 6:00 PM UTC

arrosenberg wrote: post #9

You didn't replace a SQL Analyst, you just gave them a query generator. End data consumers don't understand the data model, assumptions, quirks, etc. If they fire the analyst, they are going to wind up drawing a lot of bad conclusions on anything more complicated than simple aggregations.

You just described most SQL analysts I've worked with

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#16

Post by yrgulation » Wed, Jan 25, 2023, 6:01 PM UTC

xeyownt wrote: post #12

Wow, this looks so fun to play with. As pointed out in the blog post, the iterative process is very close to the mental process an analyst would follow. This is the beginning of an AI actually thinking ^^

[dead]

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#17

Post by LeanderK » Wed, Jan 25, 2023, 6:01 PM UTC

dmitriid wrote: post #11

logicallee wrote: post #4

Earlier quoted context omitted.

my opinion: probably will work for harder queries, because GPT is a form of general intelligence. (At least ChatGPT is. It is a much more advanced form of intelligence than davinci-003, which the poster used.)

It's not even close to being a form of general intelligence. It's an advanced language model trained on a large set of data. It gets confused for even the simplest of things, especially code. You can trick it into working more-or-less correctly on simple sets and simple tasks, but in the wrong run it will be more wrong than correct.

to be fair, we do not know how close it's to "general intelligence". Could be lightyears away, or could be surprisingly close. We just know that ChatGPT is not general intelligence, but there's also no agreed upon definition what general intelligence is.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#18

Post by logicallee » Wed, Jan 25, 2023, 6:02 PM UTC

LeanderK wrote: post #10

triyambakam wrote: post #5

Earlier quoted context omitted.

> GPT is a form of general intelligence. Is it really? I hadn't heard that

no it's not. It's just a neural network. Although it's impressive nobody is seriously calling it "general intelligence".

Tu quoque :) (You're "just a neural network" too.)

I am seriously calling it general intelligence. Maybe I'm nobody, but my assistant does my bidding.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#19

Post by ellisv » Wed, Jan 25, 2023, 6:02 PM UTC

berkle4455 wrote: post #15

arrosenberg wrote: post #9

You didn't replace a SQL Analyst, you just gave them a query generator. End data consumers don't understand the data model, assumptions, quirks, etc. If they fire the analyst, they are going to wind up drawing a lot of bad conclusions on anything more complicated than simple aggregations.

You just described most SQL analysts I've worked with

I think that's because most SQL analyst who can understand the data model, assumptions, quirks, etc. usually get promoted into other positions.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#20

Post by jawns » Wed, Jan 25, 2023, 6:03 PM UTC

The problem is that you never really know whether the chat bot gets it right or terrifically wrong unless you already know how to do the task yourself.

And in some cases, paying an analyst $50/hr. for a higher degree of confidence than you can get from a $1 chat bot is still very much worth it.

The stakes are higher, too. If the chat bot gets it wrong, what are you going to do, fire it? There goes a small trickle of revenue to OpenAI. Whereas if the analyst gets it wrong, there goes their livelihood.

That said ... this will help the $50/hr. analyst improve their productivity!