Live data from Hacker News

Show HN: Natural Language to SQL "Text-to-SQL" API

dataherald.com

11–20 of 41 posts

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#11
post #6

The submitted link results in a login wall. It would probably be worth updating the submission to the introducing-dhai page that's mentioned all the way at the end here, as getting greeted by a login prompt that way isn't going to entice people to explore or learn about the product.

We've switched from https://console.dataherald.ai/playground by OP's request.

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#14

Can I trust the generated output SQL? I want to turn something like this into a frontend for my customers to use.

Yes, absolutely. Every AI-generated SQL comes with a confidence score, so that you stay in control. We've had people who set a confidence threshold for returning answers to the users. If confidence threshold isn't met, then there's human-in-the-loop.

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#15
post #14

Can I trust the generated output SQL? I want to turn something like this into a frontend for my customers to use.

Yes, absolutely. Every AI-generated SQL comes with a confidence score, so that you stay in control. We've had people who set a confidence threshold for returning answers to the users. If confidence threshold isn't met, then there's human-in-the-loop.

I see. Just so I'm clear because this is important, if I set the confidence threshold very high, does it mean that my customers cannot create a malicious query? I don't want them deleting data or accessing rows that they're not allowed to.

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#16

I think fine-tuning is too inflexible and costly for working with something as versatile as database schemas, I would recommend looking into RAG, e.g. https://www.sqlai.ai/posts/enhancing-ai-accuracy-for-sql-gen... .

Hi -- we do use Fine-tuning together with RAG. To get best in class performance for NL-to-SQL you definitely need to combine both. The good folk at OpenAI dove into this during the last dev-day https://youtu.be/ahnGLM-RC1Y?si=7fv_JTScpBR9lK1R&t=2370

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#17
post #14

Earlier quoted context omitted.

Yes, absolutely. Every AI-generated SQL comes with a confidence score, so that you stay in control. We've had people who set a confidence threshold for returning answers to the users. If confidence threshold isn't met, then there's human-in-the-loop.

I see. Just so I'm clear because this is important, if I set the confidence threshold very high, does it mean that my customers cannot create a malicious query? I don't want them deleting data or accessing rows that they're not allowed to.

All DML commands are blocked by the engine. You can wrap the returned SQL in a CTE only passing the rows the customer is allowed to access.

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#18
post #17

Earlier quoted context omitted.

I see. Just so I'm clear because this is important, if I set the confidence threshold very high, does it mean that my customers cannot create a malicious query? I don't want them deleting data or accessing rows that they're not allowed to.

All DML commands are blocked by the engine. You can wrap the returned SQL in a CTE only passing the rows the customer is allowed to access.

Wouldn't the AI-generated query need knowledge of the CTE that will be wrapping it? How would the CTE prevent arbitrary joins, or access to tables that use the fully-qualified `schema.table`? And couldn't somebody execute any arbitrary function on the SQL server? Example `pg_sleep(9999999)`.

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#19

What do you mean by complex SQL? How complex?

I've been playing w/ this product self-hosted for a few weeks. It can join across multiple tables, windowing functions. I haven't tried self-joins yet, nor I have put much effort into tuning w/ Golden SQL or other documentation. I would put this at the skill level of a junior data engineer. It's pretty impressive.

Glad you have been liking it. Feel free to reach out at amir (at) dataherald.com if you need any additional help setting up.

Re: Show HN: Natural Language to SQL "Text-to-SQL" API

#20
This looks very polished, and a more self-serve experience than what some other companies have built.

Seeing what has happened with Assistants API, do you expect OpenAI to soon introduce an SQL API as well?

As someone in the position to evaluate integrating a text-to-SQL pipeline in our product, I'm left wondering "why not just wait until OpenAI does this?" Especially when you consider the pipeline ends at OpenAI's model anyway. How long will the current crop of productized text-to-SQL pipelines really be around for?

Post reply on HN