Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

101–110 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#101

Post by majormajor »

ethbr0 wrote: post #86
ryanjshaw wrote: post #44

Earlier quoted context omitted.

Maybe 5% of the business/operations people I've worked with would even want to do this. The rest prefer a smart human to walk them through building a report - most of the time this is because they don't actually know what they need, and they actually need an expert to sit down and figure it out with them.

The longer I'm in the business, the more I feel like my value as a software developer is identifying and asking clarifying questions.

IME, this is the number one reason outsourcing (whether on-shore or off-shore) fails for many projects.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#102

Post by [deleted] »

rowls66 wrote: post #56

Based on the natural language query provided, "Who were the largest biotech investors in 2022?" I can think of at least six possible answers based on these questions: 1. Does largest mean dollar amount, or number of investments? 2. Would number of investments count companies invested in or funding rounds invested in? 3. Does largest mean the largest total dollar amount invested in 2022, or does it mean larges dollar…

[deleted]

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#104

Post by pphysch »

layer8 wrote: post #55
knodi123 wrote: post #46

Earlier quoted context omitted.

> nobody is really replacing somebody with a program that confidently get half it's answers wrong I gotta tell you, I use copilot to help with my coding, and it still sends a shiver down my spine when it writes the entire database migration based off of the filename, or writes 4 pages of automated tests that work right on the first try.

> writes 4 pages of automated tests that work right on the first try Not sure that’s the right criterion to assess the quality and adequacy of tests.

But my GitHub badge says I have 105% test coverage! That means there are negative bugs, right...?

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#105

Post by jsdwarf »

tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

To me this boils down to Churchhills adage "Don't trust a statistical report unless you fabricated it by yourself". There are so many subtleties when translating business questions into SQL e.g. was the average deal size in the post's "trend" example computed using average/median/geometric mean formula? Should the trend only contain historic or also projected values. No serious report comes without data definitions, then its an exercise to the reader to spot flaws there.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#106

Post by kietay »

nerpderp82 wrote: post #83
mjburgess wrote: post #37

Earlier quoted context omitted.

In my experience, in many cases, the so-called "first draft" (of code, content, etc.) is more just helpful for a mental warm-up. It isn't actually suitable. The time it takes to write/code/etc. isnt in the typing, that's trivial. It's in modelling the problem (, audience) and solving it. ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the ques…

> ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed You can give it the datamodel and a semantic description and it will do a pretty good job of reprojecting your data. People talk about chatgpt as being a fixed thing, it isn't. ChatGPT is the Apple II of LLMs. 3-4 moore units from now every issue that people think is a "gotc…

In your analogy to the Apple II, is there a fundamental problem people claimed for early computers, which didn’t just boil down to “they need to be faster”?

There are fundamental limitations to what an LLM can achieve.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#107

Post by pphysch »

rezonant wrote: post #65

While this is very cool, SQL was designed to be used by business people. We need to go back to that model, where we train the business people who need these analytics how to use SQL to uncover the result. That along with a rigorous policy for including the queries that produced the result so the query logic can be checked would go a long way to actually taking advantage of the data we're collecting as businesses.

IME, the average SQL/RDBMS technical user doesn't even understand relational data modeling.

They treat it as a bad spreadsheet. JOINs are seen as an annoyance that should be avoided rather than something extremely powerful.

We are far away from average salesperson grasping it.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#108

Post by taberiand »

TOMDM wrote: post #68
rowls66 wrote: post #56

Based on the natural language query provided, "Who were the largest biotech investors in 2022?" I can think of at least six possible answers based on these questions: 1. Does largest mean dollar amount, or number of investments? 2. Would number of investments count companies invested in or funding rounds invested in? 3. Does largest mean the largest total dollar amount invested in 2022, or does it mean larges dollar…

Fully agree, the next iteration of this should get the model to prompt you when it discovers ambiguity in your request. User: "Who were the largest biotech investors in 2022?" Result: For clarification, do you mean: 1. Largest number of investments? 2. Largest total dollar amount invested in 2022 3. Largest dollar amount of new investment in 2022? 4. Something else User: option 3 Result: Okay, here are the results an…

Right now, you need to first ask ChatGPT to suggest disambiguations to the question that you're presenting it, then ask the question in that context:

User: What questions can I ask to disambiguate the question: "Who were the largest biotech investors in 2022?"

ChatGPT:

Here are a few questions you could ask to disambiguate the question "Who were the largest biotech investors in 2022?":

    Are you looking for individuals or companies that invested in biotech in 2022?
    Are you looking for the largest investors in terms of total investment amount or the number of biotech companies invested in?
    Are you looking for investors in a specific region or globally?
    Are you looking for investors in a specific biotech subfield (e.g. biopharmaceuticals, medical devices, etc.)?
    Are you looking for the investors at a specific point in 2022 or throughout the entire year?
Asking these questions will help to clarify the scope and specific details of the information you are looking for.

User: How would I disambiguate the term "largest"?

ChatGPT:

There are a few ways you could disambiguate the term "largest" when referring to biotech investors in 2022:

    By investment amount: Ask how much money was invested, this way you can determine who made the largest investment in terms of dollar value.
    By number of investments: Ask how many different biotech companies the investor invested in, this way you can determine who made the most investments.
    By percentage of ownership: Ask what percentage of ownership the investor has in the biotech companies they invested in, this way you can determine who has the largest stake in these companies.
    By market capitalization: Ask the market capitalization of the biotech companies the investor invested in, this way you can determine who has the largest investment in terms of market value.
By asking more specific questions, you can get a better idea of what is meant by "largest" and ensure that you are providing the most accurate and relevant information

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#109

Post by williamcotton »

modriano wrote: post #33
LastTrain wrote: post #14

Earlier quoted context omitted.

That is almost worse than it completely not working.

In my experience, it's unambiguously worse to produce wrong answers confidently than to return no answers, although I expect confidence men (also known as "con men") have a different opinion on this.

Hard code a preface of “My best guess is, and I could be wrong:”?

Show both the SQL query and the results and politely request that the user double check the work? Show multiple generated responses from different seeds?

Post reply on HN