Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vanna.ai: Chat with your SQL database

github.com

Post reply on HN
Print view View original

201–210 of 243 posts

Re: Vanna.ai: Chat with your SQL database

#201

Post by [deleted] »

panarky wrote: post #184
benreesman wrote: post #148

Earlier quoted context omitted.

There’s daylight between personal toolsmithing and a VC-backed startup (both are fun sometimes and a grind sometimes). I’m getting together a bunch of related-sounding stuff in terms of integrating modern models into my workflow to polish up a bit and release MIT. If you’d like to have a hand tidying it up a little and integrating it with e.g. editors and stuff, I think the bundle would be a lot cooler for it!

Microsoft may well catch the rabbit that queries schemas and generates valid SQL. But that rabbit can't understand the meaning of the data just by looking at column names and table relationships. Let's say you want to know how sales and inventory are doing compared to last year at your chain of retail stores. Will Microsoft's rabbit be smart enough to know that the retail business is seasonal, so it must compare the…

[deleted]

Re: Vanna.ai: Chat with your SQL database

#202

Post by scoot »

lmeyerov wrote: post #194
sagaro wrote: post #186

All these products that pitch about using AI to find insights from your data always end up looking pretty in demos and fall short in reality. This is not because the product is bad, but because there is enormous amount of nuance in DB/Tables that becomes difficult to manage. Most startups evolve too quickly and product teams generally tries to deliver by hacking some existing feature. Columns are added, some columns…

Our theory is we are having simultaneously a bit of a Google moment and a Tableau moment. There is à lot more discovery & work to pull it off, but the dam has been broken. It's been am exciting time to work through with our customers: * Google moment: AI can now watch and learn how you and your team do data. Around the time Google pagerank came around, the Yahoo-style search engines were highly curated, and the seman…

> Around the time Google pagerank came around, the Yahoo-style search engines were highly curated

Hmmm, no. Altavista was the go-to search engine at the time (launched 1995), and was a crawler (i.e. not a curated catalog/directory) based search. Lycos predates that but had keyword rather than natural language search.

Google didn't launch until 1998.

Re: Vanna.ai: Chat with your SQL database

#203

Post by BenderV »

echelon wrote: post #27
bob1029 wrote: post #14

The most success I had with AI+SQL was when I started feeding errors from the sql provider back to the LLM after each iteration. I also had a formatted error message wrapper that would strongly suggest querying system tables to discover schema information. These little tweaks made it scary good at finding queries, even ones requiring 4+ table joins. Even without any examples or fine tuning data.

Please turn this into a product. There's enormous demand for that.

Shameless plug - https://github.com/BenderV/ada

It's an open source BI tool that does just that.

Re: Vanna.ai: Chat with your SQL database

#204

Post by joshstrange »

sagaro wrote: post #186

All these products that pitch about using AI to find insights from your data always end up looking pretty in demos and fall short in reality. This is not because the product is bad, but because there is enormous amount of nuance in DB/Tables that becomes difficult to manage. Most startups evolve too quickly and product teams generally tries to deliver by hacking some existing feature. Columns are added, some columns…

I couldn’t agree more. I’ve hooked up things to my DB with AI in an attempt to “talk” to it but the results have been lackluster. Sure it’s impressive when it does get things right but I found myself spending a bunch of time adding to the prompt to explain how the data is organized.

I’m not expecting any LLM to just understand it, heck another human would need the same rundown from me. Maybe it’s worth keeping this “documentation” up to date but my take away was that I couldn’t release access to the AI because it got things wrong too often and I could anticipate every question a user might ask. I didn’t want it to give out wrong answers (this DB is used for sales) since spitting out wrong numbers would be just as bad as my dashboards “lying”.

Demo DBs aren’t representative of shipping applications and so the demos using AI are able to have an extremely high success rate. My DB, with deprecated columns, possibly confusing (to other people) naming, etc had a much higher error rate.

Re: Vanna.ai: Chat with your SQL database

#205

Post by bob1029 »

BenderV wrote: post #203
echelon wrote: post #27

Earlier quoted context omitted.

Please turn this into a product. There's enormous demand for that.

Shameless plug - https://github.com/BenderV/ada It's an open source BI tool that does just that.

Yep this is pretty much what I was going for.

This is why I don't chase rabbits. Y'all already got a whole box of em sitting here.

Re: Vanna.ai: Chat with your SQL database

#206

Post by totalhack »

pietz wrote: post #192
totalhack wrote: post #151

Earlier quoted context omitted.

This somewhat already exists in the form of semantic layers, which if done well can handle many of the queries necessary in the course of business without business users needing to know any SQL. There will still be cases where you need direct SQL, and AI tools can help the development process there at a minimum.

Yes, there will be cases where you need SQL knowledge. There will also always be cases where knowing exactly how a database works under the hood is necessary. I think this is somewhat of a weak argument because you can always construct an example of how something may be helpful for a small group of people. The relevant question is: How many people who work with databases need to have a lot of experience with SQL? My…

Sure, AI will assist more and more in the cases where people must write SQL or manage a database, perhaps to the point you suggest.

But my point was actually that more people think they need to know SQL today than is actually the case. Excluding people that manage databases or cases that go direct to SQL for things like complex ETL, your average business user / marketer / etc should not be asked to write SQL or have to ask someone else to write SQL for them. Use a semantic layer instead with a UI on top and it's almost as easy as natural language.

Here is a example of one I made below, but there are others out there with more support. At my company, and the last few I've worked for, we use this approach for ~all day to day querying and a chunk of backend SQL replacement.

https://github.com/totalhack/zillion

Re: Vanna.ai: Chat with your SQL database

#207

Post by [deleted] »

sagaro wrote: post #186

All these products that pitch about using AI to find insights from your data always end up looking pretty in demos and fall short in reality. This is not because the product is bad, but because there is enormous amount of nuance in DB/Tables that becomes difficult to manage. Most startups evolve too quickly and product teams generally tries to deliver by hacking some existing feature. Columns are added, some columns…

[deleted]

Re: Vanna.ai: Chat with your SQL database

#208

Post by lmeyerov »

zurfer wrote: post #189
sagaro wrote: post #186

All these products that pitch about using AI to find insights from your data always end up looking pretty in demos and fall short in reality. This is not because the product is bad, but because there is enormous amount of nuance in DB/Tables that becomes difficult to manage. Most startups evolve too quickly and product teams generally tries to deliver by hacking some existing feature. Columns are added, some columns…

that is correct. GPT-4 is good on well-modelled data out of the box, but struggles with a messy and incomplete data model. Documenting data definitely helps to close that gap. However the last part you describe is nothing new (BI teams taking credit, and pushing on problems to data engineers). In fact there is a chance that tools like vanna.ai or getdot.ai bring engineers closer to business folks. So more honest conv…

Agreed, maybe I wasn't clear enough. I don't view it as BI team vs platform team vs whoever. Maybe a decrease in the need for PhD AI consultants for small projects, or to wait for some privileged IT team for basic tasks, so they can focus on bigger things.

Instead of Herculean data infra projects, this is a good time for figuring out new policy abstractions, and finding more productive divisions of labor between different days stakeholders and systems. Machine-friendly abstractions and structure are tools for predictable collaboration and automation. More doing, less waiting.

More practically, an increasing part of the Louie.ai stack is helping get the time-consuming quality, guardrails, security, etc parts under easier control of small teams building things. As-is, it takes a lot to give a great experience.

Re: Vanna.ai: Chat with your SQL database

#209

Post by l5870uoo9y »

thdespou wrote: post #200
vopi wrote: post #135

Earlier quoted context omitted.

Are all those Twitter testimonials fake? None seem to be actual accounts.

I bet even their Site design is AI generated...

It is based on Flowbite[1], ShadUI[2] and Landwind[3].

[1]: https://flowbite.com/

[2]: https://ui.shadcn.com/

[3]: https://demo.themesberg.com/landwind/

Re: Vanna.ai: Chat with your SQL database

#210

Post by eurekin »

joshstrange wrote: post #204
sagaro wrote: post #186

All these products that pitch about using AI to find insights from your data always end up looking pretty in demos and fall short in reality. This is not because the product is bad, but because there is enormous amount of nuance in DB/Tables that becomes difficult to manage. Most startups evolve too quickly and product teams generally tries to deliver by hacking some existing feature. Columns are added, some columns…

I couldn’t agree more. I’ve hooked up things to my DB with AI in an attempt to “talk” to it but the results have been lackluster. Sure it’s impressive when it does get things right but I found myself spending a bunch of time adding to the prompt to explain how the data is organized. I’m not expecting any LLM to just understand it, heck another human would need the same rundown from me. Maybe it’s worth keeping this “…

Speculating

How about a chat interface, where you correct the result and provide more contextual information about those columns?

Those chats could be later fed back to the model and ran a DPO optimisation on top

Post reply on HN