Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

211–220 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#211

Post by bcbrown »

layer8 wrote: post #55
knodi123 wrote: post #46

Earlier quoted context omitted.

> nobody is really replacing somebody with a program that confidently get half it's answers wrong I gotta tell you, I use copilot to help with my coding, and it still sends a shiver down my spine when it writes the entire database migration based off of the filename, or writes 4 pages of automated tests that work right on the first try.

> writes 4 pages of automated tests that work right on the first try Not sure that’s the right criterion to assess the quality and adequacy of tests.

I'd be impressed if AI wrote a test suite that immediately found a bunch of valid bugs. I'd be nonplussed if it wrote a test suite that "passed" on the first try. The presence of passing tests does not imply an absence of bugs, and the hard part of developing a test suite is the test design, not the implementation.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#212

Post by _hcuq »

kvh wrote: post #192
mtc010170 wrote: post #174

Hmmm... I'm surprised I'm not seeing anyone else question the validity of this taking "2 hours" Given that it's written on the blog for the product it's using, this reads to me a lot like a pure sales pitch. They want us to believe if you use Patterns (which is neat), your company will be much more cost-effective. I'm not saying that's bad - that's probably the right thing to do with their company blog, and it's cool…

Author here, I’ve updated the post. The first draft of this app and blog post took me two hours, but I kept coming back with new ideas and tweaks throughout the week. By the end, I’d certainly spent more than two hours (more like 8?), so you’re right, I just failed to update the post. The main point stands — it’s surprisingly good for the amount of effort put in (although unclear how much more juice you could get out…

Couldn’t you have just got ChatGPT to write the post?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#213

Post by antihipocrat »

devmor wrote: post #209
kvh wrote: post #192

Earlier quoted context omitted.

Author here, I’ve updated the post. The first draft of this app and blog post took me two hours, but I kept coming back with new ideas and tweaks throughout the week. By the end, I’d certainly spent more than two hours (more like 8?), so you’re right, I just failed to update the post. The main point stands — it’s surprisingly good for the amount of effort put in (although unclear how much more juice you could get out…

It doesn't seem like you've really replaced anyone with this. You spent 8 hours doing the work that you could have paid an SQL analyst to do in much less. Unless you're saying that your time is worth less than you'd pay the analyst?

I think the idea is that once built it would be a service that could parse a question, then automatically develop and run any query in response.

Sounds cool until it produces the wrong results.. then you'll need to hire an analyst to check every query just in case.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#215

Post by fbdab103 »

fuy wrote: post #206
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

> slather a battery of views over your data One needs to be careful with this approach in terms of query performance, though. Using simple views with a couple of joins and some filtering is fine, but be very wary of stacking more than 1-2 layers of views calling each other, and especially of using things like aggregates/window functions in views, if these views then are then used as building blocks for more complex q…

Use case dependent. When I am tasked to generate some ad hoc analyses, performance is a non-issue. The query is only going to be run the handful of times while I iterate on the idea, and I would much prefer some convenience views rather giving a hoot about optimal query planning.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#216

Post by thamer »

extr wrote: post #8

Wonder what the costs are for this per question? I imagine supplying so many tokens for context makes the querying a lot more expensive. Though still no doubt cheaper than hiring another analyst.

This is answered in the blog post:

> This led to completion chains of over 20 calls to GPT for a single user question. […] it takes 15 seconds and costs $1 in credits vs $50 and 1 hour for the analyst.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#217

Post by [deleted] »

_hcuq wrote: post #212
kvh wrote: post #192

Earlier quoted context omitted.

Author here, I’ve updated the post. The first draft of this app and blog post took me two hours, but I kept coming back with new ideas and tweaks throughout the week. By the end, I’d certainly spent more than two hours (more like 8?), so you’re right, I just failed to update the post. The main point stands — it’s surprisingly good for the amount of effort put in (although unclear how much more juice you could get out…

Couldn’t you have just got ChatGPT to write the post?

[deleted]

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#218

Post by im3w1l »

clbrmbr wrote: post #197
ajuc wrote: post #151

Earlier quoted context omitted.

Well it's trained to simulate humans and that includes simulating the errors.

This is an exceptionally deep, interesting, and important statement. Is it true? Does GPT-3 occasionally make spelling errors for instance? Is the rate of spelling errors higher or lower than the average human? How about versus the average in the training data? How does the prompt affect the frequency of such errors. Ditto for other kinds of objectively measurable error.

In my experience it makes less errors if you prompt it to reason carefully step by step.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#219

Post by onos »

mtc010170 wrote: post #174

Hmmm... I'm surprised I'm not seeing anyone else question the validity of this taking "2 hours" Given that it's written on the blog for the product it's using, this reads to me a lot like a pure sales pitch. They want us to believe if you use Patterns (which is neat), your company will be much more cost-effective. I'm not saying that's bad - that's probably the right thing to do with their company blog, and it's cool…

>> They want us to believe if you use Patterns (which is neat)…

What do they do? I can’t tell.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#220

Post by jeffbee »

im3w1l wrote: post #218
clbrmbr wrote: post #197

Earlier quoted context omitted.

This is an exceptionally deep, interesting, and important statement. Is it true? Does GPT-3 occasionally make spelling errors for instance? Is the rate of spelling errors higher or lower than the average human? How about versus the average in the training data? How does the prompt affect the frequency of such errors. Ditto for other kinds of objectively measurable error.

In my experience it makes less errors if you prompt it to reason carefully step by step.

Opposite of my experience. If you talk it through each step you can back it into persistently claiming that 0^2=1.
Post reply on HN