Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

291–300 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#291

Post by ivegotnoaccount »

contravariant wrote: post #198
ethbr0 wrote: post #86

Earlier quoted context omitted.

The longer I'm in the business, the more I feel like my value as a software developer is identifying and asking clarifying questions.

I think I saw a comic years ago detailing a discussion that went something like this: A: Eventually we won't need programmers people will just tell the computer what they need and it will generate the code for them. B: True, there's actually already an industry term for a specification that's detailed enough to generate a working program from. A: Oh, what is it called? B: Code.

Yup. I think it was from Commitstrip.

edit: found it. https://www.commitstrip.com/en/2016/08/25/a-very-comprehensi...?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#292

Post by kqr »

freejazz wrote: post #289
kqr wrote: post #281

Earlier quoted context omitted.

I'm saying there are applications where you don't have to know! As long as the fraction incorrect is less than 50 % and you have 2:1 odds on the consequences you don't have to know which 50 % are incorrect.

Really? Then why pay for the thing in the first place. Why keep the data and make the queries if the results don't actually matter? I'm impressed that you have the ability to envision such a possibility, perhaps you can use that ability to come up with something reasonably likely as opposed to "conceptually possible".

I'm trying to think of easy examples. You're right that none obvious come to mind. I'm sure there is a sweetspot where we can make more money from cheap-but-sometimes-wrong GPT queries than paying for an analyst to be more definitively correct, but I'm tired and a bit fuzzy on the exact parameters.

I'll continue to think about it and write something up!

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#293

Post by andyreagan »

simonw wrote: post #207
rexreed wrote: post #147

Earlier quoted context omitted.

How is it being fine-tuned to be able to generate relevant SQL queries on the crunchbase data set? How does this jibe with the article saying they imported 2.5m records from crunchbase? Where and how are you able to import or fine tune the API-accessible GPT-3 this way? Any pointers for where I can look to do the same? Edit: I noticed you edited your comment above with some details. It seems that he used GPT-3 just t…

This is genuinely the cutting edge of how you do interesting things with language models like GPT-3 at the moment. Training these models with extra data turns out to be incredibly expensive and relatively ineffective. Instead, the most interesting research is all around tricks like this - figuring out ways to round-trip to the language model, then query other sources of data for the information that it needs, then se…

> Training these models with extra data turns out to be incredibly expensive and relatively ineffective.

I can see that it's expensive, but have you tried it for effectiveness?

BTW, your approach is very cool here.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#294

Post by pphysch »

rezonant wrote: post #230
pphysch wrote: post #107

Earlier quoted context omitted.

IME, the average SQL/RDBMS technical user doesn't even understand relational data modeling. They treat it as a bad spreadsheet. JOINs are seen as an annoyance that should be avoided rather than something extremely powerful. We are far away from average salesperson grasping it.

Thankfully you don't need to design a schema to use it. And while some schemas may indeed be too difficult for non-technical stakeholders dabbling in SQL, I think a huge percentage of them are not

You absolutely need to understand a data schema to write effective (SQL) queries against it.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#295

Post by freejazz »

kqr wrote: post #292
freejazz wrote: post #289

Earlier quoted context omitted.

Really? Then why pay for the thing in the first place. Why keep the data and make the queries if the results don't actually matter? I'm impressed that you have the ability to envision such a possibility, perhaps you can use that ability to come up with something reasonably likely as opposed to "conceptually possible".

I'm trying to think of easy examples. You're right that none obvious come to mind. I'm sure there is a sweetspot where we can make more money from cheap-but-sometimes-wrong GPT queries than paying for an analyst to be more definitively correct, but I'm tired and a bit fuzzy on the exact parameters. I'll continue to think about it and write something up!

Hard to imagine a business model where you sell that data that is only right 60% of the time. Maybe in a world where the other best data is even less reliable.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#296

Post by TeMPOraL »

2fast4you wrote: post #196
TeMPOraL wrote: post #179

Earlier quoted context omitted.

Hm. I like your idea! It's one of the few that I think have a chance of working in practice. In my experience, there is great value in describing your ideas to people who don't have the background to fully understand them, don't really care about them, and are in a bit of a trollish mood - the ways in which they misunderstand what you're saying, or pick on (what you think are) random, irrelevant things, is highly inf…

If you’ve had some success with an existing model, I think I’ll explore the idea with GPT3! Getting the prompt right is gonna be tricky, do you remember how you got AI Dungeons to play along?

I'd set up a basic story prompt, describing the roles and personalities of my character and a few NPCs, as well as relationships between all of them. I would then introduce my idea in form of a role-play, usually with my character blurting it out - then let the language model fill in reactions of other characters. From then, I'd just play along with the story.

It wasn't the most efficient way of extracting commentary out of a language model - particularly out of one that was optimized to generate plot twists and setting changes instead - but it was a very fun way.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#297

Post by minsc_and_boo »

fifilura wrote: post #253
minsc_and_boo wrote: post #153

Earlier quoted context omitted.

Yeah, IME the good business savvy SQL analysts get sucked into non-SQL assigned roles and nailing it because they write their own scripts.

Why so. Why do you mean scripts are better than SQL?

*SQL scripts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#298

Post by shinycode »

kqr wrote: post #283
shinycode wrote: post #249

Earlier quoted context omitted.

Yeah and whose responsibility is it when not catched in time and there this consequences / damage ?

The consequences would be accounted for up front and paid out of the savings from using GPT.

Which price tag are you willing to put on a loved one’s life ? Some consequences of fully automated systems can go deep into human life cost.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#299

Post by skissane »

antihipocrat wrote: post #213
devmor wrote: post #209

Earlier quoted context omitted.

It doesn't seem like you've really replaced anyone with this. You spent 8 hours doing the work that you could have paid an SQL analyst to do in much less. Unless you're saying that your time is worth less than you'd pay the analyst?

I think the idea is that once built it would be a service that could parse a question, then automatically develop and run any query in response. Sounds cool until it produces the wrong results.. then you'll need to hire an analyst to check every query just in case.

Put the requests in a queue. Have the bot generate the response. Then forward the response to a human analyst to double-check. A human can surely double-check a response much faster than they can produce one from scratch.

In many professions, it is common to have junior staff members do the grunt work, and then the more senior staff just review their work and either sign off on it, correct it, or send it back to be redone. You could use the same pattern here, replacing the junior staff with an AI, but keeping the senior one.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#300

Post by tremon »

blowski wrote: post #258
mdcox wrote: post #224

Earlier quoted context omitted.

no one wants to automate themselves out of a job, only other people.

We all believe our job is so challenging and has such special requirements that it _can't_ be automated. It requires someone with the kind of experience learned with wisdom over a long time. Blah blah blah.

Except for the ones of us who keep automating our jobs so that we can spend our effort on more challenging tasks.
Post reply on HN