Live data from Hacker News

Vanna.ai: Chat with your SQL database

github.com

21–30 of 243 posts

Re: Vanna.ai: Chat with your SQL database

#21
post #2

I'm curious about how this performs with more complex queries, like joins across five tables. Also, does the training phase actually involve writing SELECT queries by hand? In the age of ORMs and so on, many people have probably forgotten how to write raw SQL queries.

From my experience, GPT-4 will do just as well with joins as without. And that needs no specific, separate SQL training (which I assume tens of thousands of examples are already in).

Re: Vanna.ai: Chat with your SQL database

#22
post #4

Earlier quoted context omitted.

This works pretty well without a dedicated application today, e.g. "Knowing everything you do about music and music distribution, please define a database schema that supports albums, tracks, and artists" . If you have additional requirements or knowledge that the response doesn't address, just add it and re-prompt. When you're done, ask for the SQL to set up the schema in your database of choice.

Maybe my prompting needs to improve, I tried recently to get chatgpt to provide a schema for an sqlite database that implements vcard data in a normalised way. I gave up...

ChatGPT-3.5 or ChatGPT-4? There is a big difference.

For fun, I just asked ChatGPT-4 to generate a normalized database representation of vcard information: https://chat.openai.com/share/1c88813c-0a50-4ec6-ba92-4d6ff8...

It seems like a reasonable start to me.

Re: Vanna.ai: Chat with your SQL database

#24

I wish we had landed on a better acronym than RAG.

It doesn't matter, RAG is very temporary and will not be around long imho.

how else would you get private or recent data into an LLM without some form of RAG? The only aspect that might not be needed is the vector database

Re: Vanna.ai: Chat with your SQL database

#25
I love that this exists but I worry how it uses the term “train”, even in quotes, as I spend a lot of time explaining how RAG works and I try to emphasize that there is no training/fine-tuning involved. Just data preparation, chunking and vectorization as needed.

Re: Vanna.ai: Chat with your SQL database

#27
post #14

The most success I had with AI+SQL was when I started feeding errors from the sql provider back to the LLM after each iteration. I also had a formatted error message wrapper that would strongly suggest querying system tables to discover schema information. These little tweaks made it scary good at finding queries, even ones requiring 4+ table joins. Even without any examples or fine tuning data.

Please turn this into a product. There's enormous demand for that.

Re: Vanna.ai: Chat with your SQL database

#28
Is the architecture they use in this diagram currently the best way to train LLMs in general on custom data sets?

https://raw.githubusercontent.com/vanna-ai/vanna/main/img/va...

That is, store your trained custom data in vector db and then use RAG to retrieve relevant content and inject that into the prompt of the LLM the user is querying with?

As opposed to fine tuning or other methods?

Re: Vanna.ai: Chat with your SQL database

#30

Does it work with Google/Facebook ads data? Can I ask it to show best performing ads from BigQuery Facebook/Google ads data by supermetrics or improvado.

Aren't there already tons of apps answering that specific question? I think the strength of this approach is answering the non-obvious questions.
Post reply on HN