Vanna.ai: Chat with your SQL database • Hacker News Board

Re: Vanna.ai: Chat with your SQL database

#241

Post by XCSme » Sat, Jan 20, 2024, 11:49 AM UTC

anon291 wrote: post #221

sagaro wrote: post #186

All these products that pitch about using AI to find insights from your data always end up looking pretty in demos and fall short in reality. This is not because the product is bad, but because there is enormous amount of nuance in DB/Tables that becomes difficult to manage. Most startups evolve too quickly and product teams generally tries to deliver by hacking some existing feature. Columns are added, some columns…

The AI needs to truly be 'listening' in in a passive way to all Slack messages, virtual meetings, code commits, etc and really be present whenever the 'team' is in order to get anything done.

Or maybe the database documentation has to be very comprehensive and the AI should have access to it.

Re: Vanna.ai: Chat with your SQL database

#242

Post by pamelafox » Sat, Jan 20, 2024, 7:17 PM UTC

zainhoda wrote: post #109

pamelafox wrote: post #25

I love that this exists but I worry how it uses the term “train”, even in quotes, as I spend a lot of time explaining how RAG works and I try to emphasize that there is no training/fine-tuning involved. Just data preparation, chunking and vectorization as needed.

Author of the package here. I would be open to alternative suggestions on terminology! The problem is that our typical user has never encountered RAG before.

Hm we typically talk about that phase as “data ingestion”. We also use verbs extract, chunk, index, but those are parts of ingestion. Another phrase we use is “data preparation”. The tricky thing is that your train() method is on the vanna object, so if you called it “ingest”, it wouldn’t be clear where the data was headed. But I think it’d be clearer than “train” given that has such a specific meaning in this space. You could also look to LlamaIndex API for naming inspiration.

Re: Vanna.ai: Chat with your SQL database

#243

Post by brnbrdg » Mon, Jan 22, 2024, 7:02 PM UTC

I think some of the doubters here are conflating a few things.

Any analyst will struggle if data quality is a mess, and there is a lack of clear semantic layer/useful documentation around how to query metrics/join tables from your warehouse. These are problems that affect human analysts as well as LLM based 'virtual' ones... However, these are separate problems being addressed by other players.

An llm powered chatbot that can consume adequate context from company systems should be able to perform on par with a junior analyst with a similar level of context. All else equal, the LLM analyst will be orders of magnitude cheaper and open up data analysis to non-sql-literate people (huge unlock).