Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

231–240 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#231

Earlier quoted context omitted.

But, generally and unless there is a glaringly wrong result, only an analyst is going to know if the bot is right or not... what exactly does that gain you?

It gets you a really sophisticated 'auto-complete' feature

Not really. I'd guess that most people can tell if auto-complete is providing the answer they "wanted".

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#232

Hmmm... I'm surprised I'm not seeing anyone else question the validity of this taking "2 hours" Given that it's written on the blog for the product it's using, this reads to me a lot like a pure sales pitch. They want us to believe if you use Patterns (which is neat), your company will be much more cost-effective. I'm not saying that's bad - that's probably the right thing to do with their company blog, and it's cool…

I haven't found GPT this reliable for coding. I've been maxing my hourly usage of ChatGPT since it launched and then switching to CoPilot and I have lots of good things to say about it. But reliability is not one of them. It has a tendency to ignore instructions, as mentioned, but also to get hung up on certain approaches or to use a different approach each time its asked. I'd guess it's very reliable for text genera…

I gave up on ChatGPT for code generation because I ended up spending more time tweaking prompts/fixing outputs than if I had just written it myself in the first place. I think this is probably the future of "coding" but it's not quite there yet.

Is CoPilot any better?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#233

Earlier quoted context omitted.

Asking the right question is surprisingly hard sometimes. I'd love a tool I could converse with to come up with a question at the core of my thoughts.

Rubber Duck AI chat bot, you explain your problem and it critiques your wording leading you sharpen your argument.

> Rubber Duck AI chat bot

This sounds more like friendly AI than [different] "Devil's Advocate chat bot."

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#235
post #209

Earlier quoted context omitted.

It doesn't seem like you've really replaced anyone with this. You spent 8 hours doing the work that you could have paid an SQL analyst to do in much less. Unless you're saying that your time is worth less than you'd pay the analyst?

I think the idea is that once built it would be a service that could parse a question, then automatically develop and run any query in response. Sounds cool until it produces the wrong results.. then you'll need to hire an analyst to check every query just in case.

As if the analyst doesn't get the results wrong! For 1/50 of the price, maybe a few more errors are acceptable, even.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#236
post #210

Earlier quoted context omitted.

They built a bot which can answer any number of questions, each of which would have needed some analyst time. Given that the analyst rotation was an entire day once every N weeks, and the bot took 1 day to make, this is going to pay for itself after 1 week. This all assumes that the bot doesn't need tweaking for every answer — i.e. it gets at least some answers right without needing modifications to the bot — which a…

But, generally and unless there is a glaringly wrong result, only an analyst is going to know if the bot is right or not... what exactly does that gain you?

Maybe it's not a position where it is critical that all answers are 100 % accurate. Maybe getting it right every once in a while is enough to pay for the GPT compute time, but not really for analyst time.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#237
post #193

Earlier quoted context omitted.

I've done too little SQL to be traumatized by it, but I experienced it plenty of times doing more complex data processing on the backend. I would e.g. spend a week on first discussing a solution, then researching both theoretical background and relevant libraries, and finally writing code to do the math and render the answers (probability distributions). I eyeball them, they seem to all match expectations, so I decla…

The traumatising experience of SQL is that the scenario you describe being scale-invariant. The smallest question and the largest question both have the potential to be completely wrong for any inane reason. Often turns out that metrics the company has been using for months are totally bogus too in my experience, which is not a fun conversation to have. So the life of an SQL analyst in their early days (it gets bette…

This is very on point.

As an Analyst your asset is to own the data and know all the odds and quirks.

Very similar to how developers need to own their codebase. Yes you can ask chatGPT to write a complex function. But your job as a developer is to see how it fits in with the rest of the framework. The logic is the simple part.

As many pointed out you also need to be brutally honest with the numbers and also the problems you may have caused or fixed.

Trying to hide any problems will very quickly throw you downhill, into an extremely uncomfortable position.

Although before it acquires feelings and a sense of job safety, I guess chatGPT can be better at that part.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#238
In orgs where this need is usually present, the data can be massive and it takes some time to understand how it all fits together. There is also the issue of optimizing around indexes or writing queries that are cost efficient (especially if you using Athena/Presto/Big Query). Mistakes here can cost a lot of money or lock up the system so others can't use it.

I love this demo, but I feel like it would be better with a human in the loop because these edge cases can be so severe.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#239
post #235

Earlier quoted context omitted.

I think the idea is that once built it would be a service that could parse a question, then automatically develop and run any query in response. Sounds cool until it produces the wrong results.. then you'll need to hire an analyst to check every query just in case.

As if the analyst doesn't get the results wrong! For 1/50 of the price, maybe a few more errors are acceptable, even.

Which errors are you okay with?
Post reply on HN