Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

221–230 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#221

Post by csomar »

kieselguhr_kid wrote: post #131
ethbr0 wrote: post #86

Earlier quoted context omitted.

The longer I'm in the business, the more I feel like my value as a software developer is identifying and asking clarifying questions.

When I was new, I saw one of my more experienced colleagues ask a few questions that together saved the company more than $1 mil each year. ChatGPT might be a threat to automate some low-level tasks or help eliminate bugs, but it is nowhere near ready to evaluate the context of a system, understand its history, or think* through the consequences of a major business decision. * or think at all, in any meaningful way.

Though if it comes an AI with the capacity to include more context (ie: all company financials, communications, market analysis, etc...) it might be even more effective than a human with precise context.

Communication might be strictly email in the future. Or something that could be pipelined into the "AI" for context. Video/Calls might make it too at some point. Face to Face meetings strictly prohibited.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#222

Post by [deleted] »

antihipocrat wrote: post #213
devmor wrote: post #209

Earlier quoted context omitted.

It doesn't seem like you've really replaced anyone with this. You spent 8 hours doing the work that you could have paid an SQL analyst to do in much less. Unless you're saying that your time is worth less than you'd pay the analyst?

I think the idea is that once built it would be a service that could parse a question, then automatically develop and run any query in response. Sounds cool until it produces the wrong results.. then you'll need to hire an analyst to check every query just in case.

[deleted]

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#223

Post by csomar »

kieselguhr_kid wrote: post #201
RhodesianHunter wrote: post #140

Earlier quoted context omitted.

I agree with you to a point, but I think the only reason that it can't understand the context of a system is because it hasn't been trained on that system's code and documentation, which is obviously a future coming soon.

I'm not sure training these models on code and documentation will make that much of a difference. These models struggle significantly with subtlety, relevance, and correctness. It also doesn't have a theory of its own knowledge or confidence, and so tends to "hallucinate" and put out confidently-worded nonsense. Especially for complex and nuanced topics. A big part of my job in software is having a very sharpened gra…

The hallucination part is due to a lack of constraints. The AI can recognize constraints, but it can't recognize what it doesn't know for lack of context.

Prohibit all physical meetings. Force all communication through mediums that can be pipelined. Feed everything (accounting, contracts, law, etc...). Work will be then to architecture the AI to produce the optimal response.

The first company to figure this out will be too ahead. I don't think anything will be anywhere close to compete including nation states.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#224

Post by mdcox »

_hcuq wrote: post #212
kvh wrote: post #192

Earlier quoted context omitted.

Author here, I’ve updated the post. The first draft of this app and blog post took me two hours, but I kept coming back with new ideas and tweaks throughout the week. By the end, I’d certainly spent more than two hours (more like 8?), so you’re right, I just failed to update the post. The main point stands — it’s surprisingly good for the amount of effort put in (although unclear how much more juice you could get out…

Couldn’t you have just got ChatGPT to write the post?

no one wants to automate themselves out of a job, only other people.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#225

Post by freejazz »

jhugo wrote: post #210
devmor wrote: post #209

Earlier quoted context omitted.

It doesn't seem like you've really replaced anyone with this. You spent 8 hours doing the work that you could have paid an SQL analyst to do in much less. Unless you're saying that your time is worth less than you'd pay the analyst?

They built a bot which can answer any number of questions, each of which would have needed some analyst time. Given that the analyst rotation was an entire day once every N weeks, and the bot took 1 day to make, this is going to pay for itself after 1 week. This all assumes that the bot doesn't need tweaking for every answer — i.e. it gets at least some answers right without needing modifications to the bot — which a…

But, generally and unless there is a glaringly wrong result, only an analyst is going to know if the bot is right or not... what exactly does that gain you?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#226

Post by Dylan16807 »

jeffbee wrote: post #220
im3w1l wrote: post #218

Earlier quoted context omitted.

In my experience it makes less errors if you prompt it to reason carefully step by step.

Opposite of my experience. If you talk it through each step you can back it into persistently claiming that 0^2=1.

I'm pretty sure they're talking about errors it makes as part of neutral generation, not how much of a pushover it is.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#227

Post by catothedev »

freejazz wrote: post #225
jhugo wrote: post #210

Earlier quoted context omitted.

They built a bot which can answer any number of questions, each of which would have needed some analyst time. Given that the analyst rotation was an entire day once every N weeks, and the bot took 1 day to make, this is going to pay for itself after 1 week. This all assumes that the bot doesn't need tweaking for every answer — i.e. it gets at least some answers right without needing modifications to the bot — which a…

But, generally and unless there is a glaringly wrong result, only an analyst is going to know if the bot is right or not... what exactly does that gain you?

It gets you a really sophisticated 'auto-complete' feature

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#228

Post by rezonant »

clusterhacks wrote: post #74
rezonant wrote: post #65

While this is very cool, SQL was designed to be used by business people. We need to go back to that model, where we train the business people who need these analytics how to use SQL to uncover the result. That along with a rigorous policy for including the queries that produced the result so the query logic can be checked would go a long way to actually taking advantage of the data we're collecting as businesses.

I have been spectacularly unable to get any business people to learn SQL. I even find that very few will bother with reporting tools (eg Tableau) for simple self-service. Instead, the expectation is that there will be a programmer/DBA/report writer position dedicated to producing analytics "on-demand."

Yeah, not saying it's easy, or even that it's possible in every case. Most of the time I have failed to get it to happen- but recently I've seen more non-technical stakeholders take it up and use it successfully, so it is possible.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#229

Post by rezonant »

jeremyjh wrote: post #73
rezonant wrote: post #65

While this is very cool, SQL was designed to be used by business people. We need to go back to that model, where we train the business people who need these analytics how to use SQL to uncover the result. That along with a rigorous policy for including the queries that produced the result so the query logic can be checked would go a long way to actually taking advantage of the data we're collecting as businesses.

Sorry friend, but this is a bit out of touch. Maybe that was the original design intent of SQL, but understanding the application's data model is beyond a lot of sql analysts, much less their business partners.

Hey, no one said it was easy.

> but understanding the application's data model is beyond a lot of sql analysts

You need a better data model or better data analysts (probably the latter).

Putting analysts aside, I do agree though that the data model can be too complex for non-technical stakeholders, but in the vast majority of cases, the data model is simple and stakeholders are looking for basic statistical analysis and trendfinding.

Also, if a moderately skilled human SQL analyst doesn't understand a data model, well GPT has no chance.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#230

Post by rezonant »

pphysch wrote: post #107
rezonant wrote: post #65

While this is very cool, SQL was designed to be used by business people. We need to go back to that model, where we train the business people who need these analytics how to use SQL to uncover the result. That along with a rigorous policy for including the queries that produced the result so the query logic can be checked would go a long way to actually taking advantage of the data we're collecting as businesses.

IME, the average SQL/RDBMS technical user doesn't even understand relational data modeling. They treat it as a bad spreadsheet. JOINs are seen as an annoyance that should be avoided rather than something extremely powerful. We are far away from average salesperson grasping it.

Thankfully you don't need to design a schema to use it. And while some schemas may indeed be too difficult for non-technical stakeholders dabbling in SQL, I think a huge percentage of them are not
Post reply on HN