Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Dataherald AI – Natural Language to SQL Engine

github.com

Post reply on HN
Print view View original

81–90 of 108 posts

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#81

Post by screye »

Hey ! This is awesome. I have been playing around with the dirty DIN-SQL repo for a couple of weeks, so this is a delight.

A couple of questions:

* How easy is it to swap out the LLM ? (to Azure GPT4)

* Do you have 1 end2end custom user scenario ? It would really help to have 1 golden workflow (the fastest way to get things up and running) and 1 custom workflow (how to maximally configure the package to make it work the way you want) outlined in a demo/blog/tutorial.

Great work! I especially like how neat and transferrable the ideas in DIN-SQL are. Should naturally improve itself as new LLMs come out.

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#82

Post by MrezaPourreza »

iandanforth wrote: post #46

How do you do the initial query -> possible schema matching? Also, how do you control token counts?

For schema matching, we leverage embeddings. We create embeddings for the tables within the database and generate one for the natural language question provided. We then calculate the cosine similarity between these embeddings to determine a relevance score. This score is then forwarded to the agent, which uses it to identify the pertinent tables.

To minimize the token count used by Large Language Models (LLMs), we've designed our tools to handle most of the computational work internally. Consequently, only a small number of tokens are transmitted to the agent. The agent focuses solely on decision-making, while the tools handle the heavy lifting and similarity assessments.

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#83

Post by aazo11 »

screye wrote: post #81

Hey ! This is awesome. I have been playing around with the dirty DIN-SQL repo for a couple of weeks, so this is a delight. A couple of questions: * How easy is it to swap out the LLM ? (to Azure GPT4) * Do you have 1 end2end custom user scenario ? It would really help to have 1 golden workflow (the fastest way to get things up and running) and 1 custom workflow (how to maximally configure the package to make it work…

Hi -- thanks for the kind words. While right now the engine only works with OpenAI, swapping out other LLMs simply from the envars is on the roadmap. Since we use Langchain for LLM calls this logic is already abstracted away so it will be a small code change.

We have some samples (including a quickstart in the README and docs https://dataherald.readthedocs.io/en/latest/quickstart.html) but will be adding more tutorials in the coming days.

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#84

Post by Psyladine »

mellosouls wrote: post #79
neural_thing wrote: post #34

When I look at all the LLM SQL tools, I think: what a cheap and accessible way to get the wrong answers. SQL is easy. Knowledge management is hard. Does the LLM know that there was a bug in June that changed the data? Does it know that this one column is improperly named and confusing? Does it know that you recently released a mobile app the data from which is in a different table? No, of course not, those things are…

SQL is easy If it was that easy you wouldn't have a tradition of devs trying for any alternative. There's obviously a use case for this sort of product, objections appealing to the ease of use of any technical language or toolset are unlikely to be convincing to the majority who are not comfortable with it.

The history of programming is devs frustrated at arbitrary limitations of syntax or modelling and forming new ones, with their own arbitrary limitations of syntax or modelling.

When your only tool is a FOR loop hammer, every set based operation frustratingly looks less like a nail than a screw.

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#85

Post by aazo11 »

fhd2 wrote: post #76
mcqueenjordan wrote: post #74

I want to preface this comment so that I don't detract from the idea. This is a really cool product, and obviously has value. There's just one thing I worry about. It's losing expertise in your data model and gaining organizational false confidence in bad data. Let's consider Bob. Bob is a Product Manager. Bob always used to bother his software engineers to write SQL queries, but now he just uses this tool. Bob didn'…

It's a risk for Bob, I think he should be the one judging whether it's worth it or not, since he'd probably get in trouble over this. What I take away from this is that it'd be helpful if the tool clearly warned him of this risk. Other than that, I'm sceptical for using LLMs for lots of things, but this I find cool. In my consulting work (and even before as a CTO), I usually train PMs, CEOs, really anyone who didn't…

Hi -- the /question endpoint does return the generated SQL under the sql_query field of the JSON response: https://dataherald.readthedocs.io/en/latest/api.question.htm...

You can even see the entire chain of LLM calls in the intermediate_steps field to help debug the agent.

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#86

Post by saigal »

mcqueenjordan wrote: post #74

I want to preface this comment so that I don't detract from the idea. This is a really cool product, and obviously has value. There's just one thing I worry about. It's losing expertise in your data model and gaining organizational false confidence in bad data. Let's consider Bob. Bob is a Product Manager. Bob always used to bother his software engineers to write SQL queries, but now he just uses this tool. Bob didn'…

At the risk of generalizing too much, there's always a chance that layering on a new technology can limit a user's understanding by creating an additional degree of separation. More specifically though, if a PM has been using a BI tool (tableau, looker, etc) or Product Analytics tool (e.g. amplitude, mixpanel) and clicking through dashboards without ability to find their answer, then I don't think this makes them lose additional expertise in the data model.

And perhaps even more importantly, my experience has been that people ultimately wish to understand trends. In other words, an ad-hoc question that is posed and answered in natural language might ultimately be made into a dashboard that lives on the BI tool (so that trends become clearer).

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#87

Post by saigal »

fhd2 wrote: post #78
mcqueenjordan wrote: post #77

Earlier quoted context omitted.

Yes -- I think transparency on the limitations would be a useful step in mitigation. But I ultimately think that it will become a black box for most of those who use it, with a good deal of blind faith in the results. Sounds pessimistic, but grounded in reality, I think. That's not a deal breaker, though! Just a caveat about how much you can trust the output.

I think a tool that allows you to see the generated SQL and maybe edit it should help achieve what I wrote about! And looking at the code, it shouldn't even be a big change in this one. Sure, some people might not bother/think, there's no helping that. But it'd help those who do. I found non-technical people pretty curious and willing to learn once they get over this imagined barrier of "it's too hard". I fix small s…

I agree that allowing a user, particularly a non-technical one, to see the auto-generated SQL (i.e. the SQL created from natural language) might actually encourage understanding of the data. It can create an easier on-ramp for non-tech folks to understand data and get deeper into it.

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#88

Post by saigal »

addisonj wrote: post #44

Congrats on the initial launch! Here a few thoughts, feedback, and questions: * you do a good job in this post of describing why you need more than just ChatGPT to get acceptable quality, but much less of that is in the readme. I wouldn't be afraid to sell the project a bit harder, even in early days * likewise, I think a small visual of the architecture would be helpful, just to make clear what the relationship is t…

I want to double-down on what AAzo11 replied. There is a human in the loop via an admin dashboard that allows quick approval / editing in the event of incorrect queries. This significantly decreases time for a member of the data team to field requests since they typically block out an hour to approve/edit these queries before going back to the question-asker.

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#89

Post by saigal »

bravura wrote: post #49
aazo11 wrote: post #38

Earlier quoted context omitted.

Sort of. Having perfect data engineering is a requirement if you want to connect an LLM straight to your data warehouse. For real world scenarios, you need a way to add context over time (including examples of how to answer questions from messy data). The same way a new team member would need to be on-boarded the tool needs to learn the context of the data and business logic, store it under supervision from an admin…

Personally I think that's a great response. Continuous schema mapping, gotcha-patching, and formalization of undocumented knowledge is imperative, and this can be solved by engineers using an ongoing process. Kudos on the launch. I did an NL-to-SQL startup, but now I think is a much better time to do this.

Curious as to why you pivoted away from the previous NL-to-SQL startup?

Re: Show HN: Dataherald AI – Natural Language to SQL Engine

#90

Post by saigal »

kizum wrote: post #52

Very nice! I added a similar feature to https://sitespeak.ai recently that allows my users to connect their virtual assistant to a Sqlite database to retrieve data to answer visitor questions. Initially this was just a way around the difficulty of "chatting to a CSV" so the solution was to convert any CSV to a Sqlite database and using that, but it's proved very useful. Will definitely look at using Dataherald for ad…

Very cool! is this mostly for unstructured data search? Any interest in adding tabular / structured data NL search?
Post reply on HN