Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Replacing a SQL analyst with 26 recursive GPT prompts

patterns.app

Post reply on HN
Print view View original

171–180 of 311 posts

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#171

Post by kirubakaran »

Normal_gaussian wrote: post #164
hinkley wrote: post #119

Earlier quoted context omitted.

People dismissed the XP book by saying it was just stating practices that already existed. Articulating and selling an idea is a huge part of the challenge. Fifteen years later if you're having any success at all with Scrum it was because your team adopted the half of XP that didn't outright disagree with Scrum. I liked the Refactoring book because it gave me names for things I figured out in 1996 (unfortunately whil…

> I had a boss who was big in the Kanban 'movement'. We got along famously, in part because I re-invented a Kanban board in 1994 when faced with a convenient whiteboard for the first time. I was showing slides of a pre-1900 engineering office as part of a presentation a while back when someone shouted out "look - they're doing kanban". And sure enough there was a pinboard with process headings on it and pinned items…

> It would not surprise me if they found one in Pompei

"Evacuate" was stuck in the backlog column

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#173

Post by MonkeyClub »

TeMPOraL wrote: post #87
tragomaskhalos wrote: post #45

Anyone who's been asked more than a couple of times for data that requires a non-trivial bit of ad-hoc SQL will know the sinking "oh shit" feeling that comes when you subsequently realise you borked the query logic in some subtle way and have accordingly emailed out a completely bogus answer/report. From the article it doesn't seem that GPT is significantly better or worse than a human in this regard, although an exp…

I've done too little SQL to be traumatized by it, but I experienced it plenty of times doing more complex data processing on the backend. I would e.g. spend a week on first discussing a solution, then researching both theoretical background and relevant libraries, and finally writing code to do the math and render the answers (probability distributions). I eyeball them, they seem to all match expectations, so I decla…

> b) the nicely looking graphs they saw are all pure bullshit, and the things that looked correct only seemed so, and only by accident.

They’re not “bullshit” per se, they’re just “prototypes” for further refinement :)

The road to success is trodden in stumbling half-drunken steps on the pebbles of failures.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#174

Post by mtc010170 »

Hmmm... I'm surprised I'm not seeing anyone else question the validity of this taking "2 hours" Given that it's written on the blog for the product it's using, this reads to me a lot like a pure sales pitch. They want us to believe if you use Patterns (which is neat), your company will be much more cost-effective.

I'm not saying that's bad - that's probably the right thing to do with their company blog, and it's cool nonetheless. But I do get a little tired of people putting stuff out there like this that warps (some peoples) perception around how long things actually take. We wonder why, as an industry, we misjudge timelines on things left and right.

Even if we take it at face value, this is a person who's intimately familiar with this product. So sure, it's easy to set things up when we've done it a bunch of times. If you were doing this, solving the novel problem that you're faced with, is that how long it would take? Plus that's not really what most of us get paid to do. We have to learn on the fly and figure stuff out as it comes.

So rather than have the provocative headline and conclusion, like a lot of other people have commented... this is really something that could amplify that $50/hour employee, not take their job away. And maybe we shouldn't read into the alleged speed so much. YMMV.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#175

Post by totalhack »

This seems fun, but certainly unnecessary. All of those questions could be answered in seconds using a warehouse tool like Looker or Metabase or https://github.com/totalhack/zillion (disclaimer: I'm the author and this is alpha-level stuff, though I use it regularly).

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#177

Post by jletienne »

cm2187 wrote: post #150

At least it learned from the training dataset to never fucking format numbers in a way that would be remotely readable to the human eye (like every other fucking sql and developer tool on earth). Because 133854113715608.0 is telling me exactly what I need to know.

wait what? that number is so big you know it's wrong. commas in numbers are nightmare to deal with and the .0 lets you know it's a decimal not an integer.

If it was formatted like 133,854,113,715,608, you'd be stuck trying to figure out if it possibly meant "133,854,113,715,608" or [133,854,113,715,608] which would be awful.

Format numbers in excel or any other reporting tool. lmfao i like your comment. what's miserable to you is a lifesaver for me

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#178

Post by mrbungie »

arrosenberg wrote: post #9

You didn't replace a SQL Analyst, you just gave them a query generator. End data consumers don't understand the data model, assumptions, quirks, etc. If they fire the analyst, they are going to wind up drawing a lot of bad conclusions on anything more complicated than simple aggregations.

You can easily add more components to start thinking about models/assumptions/etc, like adding interfaces to Data Catalogs (i.e. Glue Data Catalog).

As part of a POC I made, I built a similar bot without recursion for debugging and iterative query building though. It does the following:

- It predicts most probable entities from the question. - Searches AWS Glue Data Catalog for the most probable/useful tables. - It builds an Athena SQL Query from N most useful tables.

It obviously get it catastrophically wrong sometimes, but hell, it was a 3 hour POC. If you can make better indices that map entity->table relationships it should get better at searching tables. Add this kind of recursive/iterative debugging of queries, and you get at least something near a junior-level SQL Analyst.

These kind of bots are analogous to Stable Diffusion, they DO need a good prompter/puppeteer/solution-verifier. Most non-senior Data Analysts also need one anyways.

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#179

Post by TeMPOraL »

2fast4you wrote: post #124
TeMPOraL wrote: post #87

Earlier quoted context omitted.

I've done too little SQL to be traumatized by it, but I experienced it plenty of times doing more complex data processing on the backend. I would e.g. spend a week on first discussing a solution, then researching both theoretical background and relevant libraries, and finally writing code to do the math and render the answers (probability distributions). I eyeball them, they seem to all match expectations, so I decla…

Your anecdote gave me an idea. What if we use AI to enhance the communication between people instead of replacing the people? Imagine you give a problem statement to the LLM, then it tried to find all the subtleties, confusing parts, room for interpretation, etc. All of the people involved revise the problem statement to make it more clear and give it back to the LLM. Iterate like that until satisfied

Hm. I like your idea! It's one of the few that I think have a chance of working in practice.

In my experience, there is great value in describing your ideas to people who don't have the background to fully understand them, don't really care about them, and are in a bit of a trollish mood - the ways in which they misunderstand what you're saying, or pick on (what you think are) random, irrelevant things, is highly informative. The feedback you get from such people makes you think about things and in ways you wouldn't have thought otherwise.

The problem is, of course, you generally don't have a pool of such people available 24/7. However, LLMs today seem like they could fit this role just fine. They can't and won't understand the point you're trying to get across - but they can and will (if asked) pattern-match on vague language, logical jumps, sentences with multiple meanings, etc. They'll get it subtly wrong, too - much like a bored friend who's mostly focused on playing Angry Birds and hears only every third sentence of your monologue, and then blurts something to keep you talking for the next 30 seconds so they can focus on aiming the birds at the towers of pigs.

I would totally use a LLM-backed tool optimized to facilitate such conversation sessions. I actually tried this in the past, in AI Dungeons, and results were encouraging (i.e. responses got me to think in ways I normally don't).

Re: Replacing a SQL analyst with 26 recursive GPT prompts

#180

Post by Johnny555 »

If you’re willing to accept unverified results from an AI chat bot, you may as well just let the end user make their best guess using a query builder himself. My company requires that any queries used for official reporting or provided to the exec team get blessed by the data sciences team to avoid errant data from bad queries, I’m not sure an AI chat bot would remove this need.
Post reply on HN