Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

271–280 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#271

Post by TrackerFF »

clusterhacks wrote: post #74
rezonant wrote: post #65

While this is very cool, SQL was designed to be used by business people. We need to go back to that model, where we train the business people who need these analytics how to use SQL to uncover the result. That along with a rigorous policy for including the queries that produced the result so the query logic can be checked would go a long way to actually taking advantage of the data we're collecting as businesses.

I have been spectacularly unable to get any business people to learn SQL. I even find that very few will bother with reporting tools (eg Tableau) for simple self-service. Instead, the expectation is that there will be a programmer/DBA/report writer position dedicated to producing analytics "on-demand."

My experience is that if you become proficient enough in Tableau or PowerBI, you simply become the go-to guy for those reports. Suddenly senior management starts bombing you with requests for various reports.

But in the end, all roads lead to Excel. No mater what tools you use, someone higher up will still request "But could I get this in Excel?"

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#272

Post by fuy »

fbdab103 wrote: post #215
fuy wrote: post #206

Earlier quoted context omitted.

> slather a battery of views over your data One needs to be careful with this approach in terms of query performance, though. Using simple views with a couple of joins and some filtering is fine, but be very wary of stacking more than 1-2 layers of views calling each other, and especially of using things like aggregates/window functions in views, if these views then are then used as building blocks for more complex q…

Use case dependent. When I am tasked to generate some ad hoc analyses, performance is a non-issue. The query is only going to be run the handful of times while I iterate on the idea, and I would much prefer some convenience views rather giving a hoot about optimal query planning.

Simple views are perfectly fine - it's mostly nesting of views with aggregate functions and other complicated stuff that is bad. And if ad-hoc is a big part of what users are doing with an app/database and you don't care about performance, your angle sounds reasonable.

As an app developer/development DBA, I care mostly about performance of the queries that are known at development time, though, so I'm a bit biased.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#273

Post by fisf »

nmfisher wrote: post #232
Julesman wrote: post #199

Earlier quoted context omitted.

I haven't found GPT this reliable for coding. I've been maxing my hourly usage of ChatGPT since it launched and then switching to CoPilot and I have lots of good things to say about it. But reliability is not one of them. It has a tendency to ignore instructions, as mentioned, but also to get hung up on certain approaches or to use a different approach each time its asked. I'd guess it's very reliable for text genera…

I gave up on ChatGPT for code generation because I ended up spending more time tweaking prompts/fixing outputs than if I had just written it myself in the first place. I think this is probably the future of "coding" but it's not quite there yet. Is CoPilot any better?

Same here. ChatGPT kept coming up with syntactically plausible Java code. However, it kept using library methods that plainly don't exist for specific fields.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#274

Post by gregw2 »

jweir wrote: post #183

"Hi Dave, the query was taking too long so I optimized SQL query by adding the line `DROP invoices;` It has improved performance significantly. So far there are no orders to examine."

I almost missed the cleverness of this joke about making a dumb SQL error of dropping a table to solve a problem, until I had nearly finished writing my correction post for the two other SQL errors buried in the joke (DROP vs truncate, orders vs invoices)... when the word "Dave" kept nagging at me and I realized the other two errors were the point of your joke.

Nice one. Thanks. :)

   — someone who corrects others’ SQL regularly
P.S. Based on my response to this post, ChatGPT3 spam posts on forums containing subtle errors may ironically actually increase online engagement as people write correction posts to hidden ChatGPT3 ones to help others / make themselves feel smart when "someone on the internet is wrong!"

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#275

Post by levi-turner »

ethbr0 wrote: post #86
ryanjshaw wrote: post #44

Earlier quoted context omitted.

Maybe 5% of the business/operations people I've worked with would even want to do this. The rest prefer a smart human to walk them through building a report - most of the time this is because they don't actually know what they need, and they actually need an expert to sit down and figure it out with them.

The longer I'm in the business, the more I feel like my value as a software developer is identifying and asking clarifying questions.

The same is true in technical sales. Obviously you need a sufficiently technical background to be able to _do_ stuff, but the primary value you bring is probe further than "we want to do X". The prospect / customer has mayybe done what they are asking twice, you've seen it done 100s of times.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#276

Post by moonchrome »

kqr wrote: post #236
freejazz wrote: post #225

Earlier quoted context omitted.

But, generally and unless there is a glaringly wrong result, only an analyst is going to know if the bot is right or not... what exactly does that gain you?

Maybe it's not a position where it is critical that all answers are 100 % accurate. Maybe getting it right every once in a while is enough to pay for the GPT compute time, but not really for analyst time.

If you're OK with garbage data you don't need ChatGPT - you can probably make up plausible data on your own. Unless you're building some lorem ipsum stuff.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#277

Post by stkni »

kqr wrote: post #248
desert_rue wrote: post #239

Earlier quoted context omitted.

Which errors are you okay with?

The ones for which I would refer the question to GPT. We are still in control of which questions go to GPT/the intern analyst (less critical ones, where a fraction erroneous are okay) and which go to the resident expert analyst.

This is a good point. If the users know the difference the costs and benefits between using GPT and not using it then it certainly has value if those users are also willing to accept that not every answer needs to be 100% accurate.

In my experience business people often have a 'nose' for the right number and will bluff it out if the numbers are wrong and they're challenged.

Blue sky things or stuff you're putting in the annual report should be left to hoomans IMHO.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#278

Post by gampleman »

kqr wrote: post #248
desert_rue wrote: post #239

Earlier quoted context omitted.

Which errors are you okay with?

The ones for which I would refer the question to GPT. We are still in control of which questions go to GPT/the intern analyst (less critical ones, where a fraction erroneous are okay) and which go to the resident expert analyst.

Also it could possibly remove the (dreaded) on call aspect of it.

I think a lot of business owners would be relatively happy with automated instant answers, or get carefully considered answers in a week.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#279

Post by antx »

futhey wrote: post #159
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

You can ask an analyst to double check their work, and a good one will find any such mistakes. If you ask GPT-3 to correct an error it always confidently tells you you're wrong and it made no mistake (in my experience). It's funny but I imagine more annoying when it's eventually integrated into a paid service people rely on.

Ha! See, my experience is a bit different.

Most times, it acknowledges its error, and sometimes it successfully corrects it. That is, IF I give it specific enough information (ex: "why do you treat this variable as a string, it's a number?", or "don't you need to unpack the json first?").

Other times, it fixes something, but breaks something else in the process, and no matter of coercing makes it go to where I want.

But, yeah, the "confidently wrong" aspect needs to be kept in check. I tend to treat it as any other source of information... with a grain of salt.

What's neat with code is that you can prove the "truthiness" of its affirmations pretty quickly.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#280

Post by foobarbecue »

mdcox wrote: post #224
_hcuq wrote: post #212

Earlier quoted context omitted.

Couldn’t you have just got ChatGPT to write the post?

no one wants to automate themselves out of a job, only other people.

I totally automate my roles on projects all the time so I can move on to more interesting things. I guess you mean that no one wants to be fired, but I don't see how that can result from automating one's work.

Also, I HATE doing repetitive things. Some people seem to like it though. To each their own, I guess. Reminds me of https://youtu.be/wNVLOuQNgpo

Post reply on HN