Is there a DSPy module, where I can give a schema, and can start asking sql questions in a structured way?
Not yet, planning to build an open source, local, CLI tool that utilizes natural-sql though!
Show HN: Natural-SQL-7B, a strong text-to-SQL model
11–20 of 171 posts
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#12This is not open source, because you have use-based restrictions. Call it what it is, source available.
Updated the title since it may have been confusing, appreciate the feedback!
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#13Context is 4096? My app db DDL is 19877 tokens (using Llama2 tokenizer) long, so that means we need to do a RAG for handling the DDL prompt injection.
A model like this with a 32k long seq_len, like Mixtral, would be a killer for me.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#14Is this the data that was used for fine tuning?
https://github.com/defog-ai/sql-eval/blob/main/data/question...
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#15Is this the data that was used for fine tuning? https://github.com/defog-ai/sql-eval/blob/main/data/question...
No, those are benchmark, evaluation questions. The fine tune dataset was a custom, synthetically generated dataset of ~20k PostgreSQL Text to SQL pairs covering different SQL categories and question types.
I mention a little more about it here https://x.com/calebfahlgren/status/1754247740291207198?s=20
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#16Context is 4096? My app db DDL is 19877 tokens (using Llama2 tokenizer) long, so that means we need to do a RAG for handling the DDL prompt injection. A model like this with a 32k long seq_len, like Mixtral, would be a killer for me.
Great call out. Will definitely focus on that in the next iteration!
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#17really cool, the license is not really standard, but seems open source. The actual model can be found here: https://huggingface.co/cfahlgren1/NaturalSQL-6.7B-v0 This seems like a great base model, although I wonder if text-to-sql is good use case for small models. We are also building a tool in the space and I regularly wish gpt-4 to be even more knowledgable when answering. Even gpt 3.5 is not good enough for produc…
Thanks! Yes, that was an earlier iteration. The v1 is here https://huggingface.co/chatdb/natural-sql-7b . Plan to push to Ollama soon and build an open source free tool around it. Would love to hear about what you are building!
We are building Dot (https://www.getdot.ai/). We mostly focus on good conversation design and data governance to ensure a great end user experience.
Here is an example conversation about HN data https://eu.getdot.ai/share/c80139c9-13f4-4db4-88f6-6e058ba31...
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#18So you feed it the whole DDL with the prompt. I wonder how it performs on a task like schema normalization or optimization? Say, also include the slow queries log and ask it for the SQL commands to modify the indexes to speed up those queries. Allow it to normalize/denormalize/materialize as options. Give it a way to run the resulting DDL against a test suite and iterate toward some fitness goal. This would save gobs…
That is really interesting. I took note of this. That would be really cool!
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#19This is a big improvement, but I'm not a believer that SQL is the most appropriate query lang here. Personally am more bullish on language models being trained with ORMs, as those normally capture much more information about the fields.
e.g. Passing in some of my more complex table schemas related to flight data and asking about overflights, the model struggles to resolve out information related to aviation. However, GitHub Copilot writes me a perfect call to Prisma with the same single line instruction + information spanning the rest of my codebase.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#20Is this the data that was used for fine tuning? https://github.com/defog-ai/sql-eval/blob/main/data/question...
No, those are benchmark, evaluation questions. The fine tune dataset was a custom, synthetically generated dataset of ~20k PostgreSQL Text to SQL pairs covering different SQL categories and question types. I mention a little more about it here https://x.com/calebfahlgren/status/1754247740291207198?s=20
So this is essentially postgres only? Or how will it handle e.g. MS SQL Schemas and output?