Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

31–40 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#31

Post by mgdev »

jeltz wrote: post #24
berkle4455 wrote: post #15

Earlier quoted context omitted.

You just described most SQL analysts I've worked with

Yeah, that is an issue a friend of mine has right now with his team. He randomly ended up as the manger of the data analytics team despite no analytics background (or really much programming background either). And one of his main frustrations is that the analysts do not understand the data model or the business. Before he became their manager his team just took report requests and then wrote some SQL and delivered s…

> He randomly ended up as the manger of the data analytics team despite no analytics background

I assure you, it wasn't random. It was punitive. ;)

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#32

Post by beckon69 »

logicallee wrote: post #23
dmitriid wrote: post #11

Earlier quoted context omitted.

It's not even close to being a form of general intelligence. It's an advanced language model trained on a large set of data. It gets confused for even the simplest of things, especially code. You can trick it into working more-or-less correctly on simple sets and simple tasks, but in the wrong run it will be more wrong than correct.

So it fails your "not even close to being a form of general intelligence" bar. For me it clearly unambiguously no-contest 100% definitely exhibits behaviors that qualify it as a limited form of general intelligence.

> limited form of general intelligence. this seems like a bit of a misnomer, no?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#33

Post by modriano »

LastTrain wrote: post #14
jeltz wrote: post #6

Earlier quoted context omitted.

Yeah, it doesn't. A friend of mine played around with this and on trickier questions you got confidently incorrect answers.

That is almost worse than it completely not working.

In my experience, it's unambiguously worse to produce wrong answers confidently than to return no answers, although I expect confidence men (also known as "con men") have a different opinion on this.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#35

Post by satisfice »

berkle4455 wrote: post #15
arrosenberg wrote: post #9

You didn't replace a SQL Analyst, you just gave them a query generator. End data consumers don't understand the data model, assumptions, quirks, etc. If they fire the analyst, they are going to wind up drawing a lot of bad conclusions on anything more complicated than simple aggregations.

You just described most SQL analysts I've worked with

Except now with no accountability whatsoever. Just a magic box that you get to worship.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#36

Post by newswasboring »

counttheforks wrote: post #28
logicallee wrote: post #18

Earlier quoted context omitted.

Tu quoque :) (You're "just a neural network" too.) I am seriously calling it general intelligence. Maybe I'm nobody, but my assistant does my bidding.

Then you're very much misinformed. > my assistant does my bidding It doesn't. Try asking it how to make a bomb, then search for the same on youtube. One of the two yields a useful result, the other gives a moralistic lecture.

Fwiw if I ask a human assistant how to make a bomb I'll probably get a moral lecture too.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#37

Post by mjburgess »

jawns wrote: post #20

The problem is that you never really know whether the chat bot gets it right or terrifically wrong unless you already know how to do the task yourself. And in some cases, paying an analyst $50/hr. for a higher degree of confidence than you can get from a $1 chat bot is still very much worth it. The stakes are higher, too. If the chat bot gets it wrong, what are you going to do, fire it? There goes a small trickle of…

In my experience, in many cases, the so-called "first draft" (of code, content, etc.) is more just helpful for a mental warm-up. It isn't actually suitable.

The time it takes to write/code/etc. isnt in the typing, that's trivial. It's in modelling the problem (, audience) and solving it.

ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed

That its answers satisfy the requirements would require ChatGPT to ask you questions (and much else besides)

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#38

Post by satisfice »

This is yet another formula for a buggy app, courtesy of a man who doesn’t think critically.

Somehow the image of 50,000 e-bikes in a landfill comes to mind, with a bankrupt founder pleading “but it was a cool idea!”

This is a cool idea, but nothing in this article explains how it is a responsible idea.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#39

Post by satisfice »

jamiequint wrote: post #22
dmitriid wrote: post #11

Earlier quoted context omitted.

It's not even close to being a form of general intelligence. It's an advanced language model trained on a large set of data. It gets confused for even the simplest of things, especially code. You can trick it into working more-or-less correctly on simple sets and simple tasks, but in the wrong run it will be more wrong than correct.

"It's not even close to being a form of general intelligence. ... It gets confused for even the simplest of things, especially code. Congratulations, you just described most humans :)

Except humans can be held responsible.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#40

Post by bluecoconut »

This is great~ There's been some really rapid progress on Text2SQL in the last 6 months, and I really thinking this will have a real impact on the modern data stack ecosystem!

I had similar success with lambdaprompt for solving Text2SQL (https://github.com/approximatelabs/lambdaprompt/) where one of the first projects we built and tested was a Text-to-SQL very similar to this

Similar learnings as well:

- Data content matters and helps these models do Text2SQL a lot

- Asking for multiple queries, and selecting from the best is really important

- Asking for re-writes of failed queries (happens occasionally) also helps

The main challenge I think with a lot of these "look it works" tools for data applications, is how do you get an interface that actually will be easy to adopt. The chat-bot style shown here (discord and slack integration) I can see being really valuable, as I believe there has been some traction with these style integrations with data catalog systems recently. People like to ask data questions to other people in slack, adding a bot that tries to answer might short-circuit a lot of this!

We built a prototype where we applied similar techniques to the pandas-code-writing part of the stack, trying to help keep data scientists / data analysts "in flow", integrating the code answers in notebooks (similar to how co-pilot puts suggestions in-line) -- and released https://github.com/approximatelabs/sketch a little while ago.

Post reply on HN