Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Natural-SQL-7B, a strong text-to-SQL model

github.com

Post reply on HN
Print view View original

71–80 of 171 posts

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#71

Post by internet101010 »

rgbrgb wrote: post #21

So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? As a programmer who always has to look stuff up when I SQL, I could definitely see asking something like this for a first draft of a query but it seems like I'm slightly better off asking…

> What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time?

Yeah this is the issue I have with all of the SQL generation stuff. Not only should the SQL be valid, a prompt like "generate a query that pulls sales for the last quarter" should generate the same output for everyone without fail. Vanna's business logic embedding is a good first step but even then it is only correct like 90% of the time with GPT-4.

Even then, it will only work if there are strong standards and data governance structures in place that everyone within an organization is aligned on. For example, "sales" can mean different things to different people and all of that needs to be buttoned up as well.

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#72

Post by Closi »

rgbrgb wrote: post #21

So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? As a programmer who always has to look stuff up when I SQL, I could definitely see asking something like this for a first draft of a query but it seems like I'm slightly better off asking…

> What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time?

The kind of stuff that it is very easy to validate if it works or not :)

I am building a warehouse management system at the moment, and it's great to quickly churn out lots of SQL views (particularly as the schema is changing/evolving slightly as I am writing it, so being able to go back to GPT4 to churn through the changes to some of the 'views' of my pages helps, even if it requires a little testing/validation).

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#73

Post by fifilura »

calvinmorrison wrote: post #68
fifilura wrote: post #34

Earlier quoted context omitted.

1. If you are a a programmer I think you should learn SQL. It will give you a different perspective on programming that I think is invaluable. (I.e. programming without for loops) 2. Combining and slicing data is a craft, and doing it subtly wrong in one step can lead to fatal errors in the outcome. And most importantly, it can be very difficult to notice. Numbers don't smell. That is why I would be very hesitant to…

Honestly I would rather write stuff in code thats inefficient than deal with one more stored procedure. Its like a black box

Just that deeply inside that forest of functions you just wrote is the inner join that the SQL query would do in a couple of lines of code embedded.

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#74

Post by Closi »

internet101010 wrote: post #71
rgbrgb wrote: post #21

So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? As a programmer who always has to look stuff up when I SQL, I could definitely see asking something like this for a first draft of a query but it seems like I'm slightly better off asking…

> What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? Yeah this is the issue I have with all of the SQL generation stuff. Not only should the SQL be valid, a prompt like "generate a query that pulls sales for the last quarter" should generate the same output for everyone without fail. Vanna's business logic embedding is a good first step…

As someone who works with lots of analysts, I can guarantee that they also don't make the correct interpretations all the time, and that you have to sense-check the results back against reality.

In either case, validation is the key step - you can't just trust that your SQL query is correct regardless of if you have manually written it, you still have to go through the data and check it.

That's where the SQL generation stuff can save time - if 50% of the time you can get to an answer in half the time, then it's great! Normally in my experience with current-gen LLM's when they fail they fail quickly, so the other 50% of queries don't take twice as long to write manually.

Then there is the other use case - if you aren't sure why a particular SQL query is erroring, these LLM's are great at telling you why and fixing your code.

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#75

Post by pama »

thecalebf wrote: post #62
zainhoda wrote: post #48

Very cool. Would the license allow for use with Vanna? https://github.com/vanna-ai/vanna

Yes please do! Looks awesome, would love to help any way I can as well.

Not OP, but would it be possible to use a standardized license? Every time a special purpose license is used for a software that gains adaptation, the lawyers of hundreds to thousands of different companies must spend a lot of time and iterations with the team to figure out if they can actually use this model. There is something magical in the GPL, MIT, Apache, etc licenses because these lawyers have already opined on them once and no longer create a bottleneck.

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#76

Post by tillvz »

internet101010 wrote: post #71
rgbrgb wrote: post #21

So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? As a programmer who always has to look stuff up when I SQL, I could definitely see asking something like this for a first draft of a query but it seems like I'm slightly better off asking…

> What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? Yeah this is the issue I have with all of the SQL generation stuff. Not only should the SQL be valid, a prompt like "generate a query that pulls sales for the last quarter" should generate the same output for everyone without fail. Vanna's business logic embedding is a good first step…

Having an LLM be in charge of business logic is madness.

There cannot be any AI involved when processing the definition of a KPI. Otherwise you'll never be able to roll it out to thousands of users when there's always a 90% (or even 99%) chance that the business logic might not get applied correctly.

Check out what we do at Veezoo (https://www.veezoo.com) with the Knowledge Graph / Semantic Layer to mitigate that.

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#78

Post by fwip »

jimmytucson wrote: post #53

In the data world I've worked with tons of folks whose responsibilities include getting questions from execs, knowing their way around the data warehouse enough to write SQL to answer those questions, and delivering the answers back (sometimes formatted nicely). Sometimes they have to predict what followup questions the exec will ask, like "why is that number so low, it obviously shouldn't be that low" so they can pr…

My money is on "overpaid execs will use this to get wrong information, and get mad at their subordinates for correcting them."

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#79

Post by ren_engineer »

whimsicalism wrote: post #25
rgbrgb wrote: post #21

So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? As a programmer who always has to look stuff up when I SQL, I could definitely see asking something like this for a first draft of a query but it seems like I'm slightly better off asking…

do you not trust the setting OAI provides to exclude your conversation from training data?

it's not about trust, they will always be indirectly controlled by the US government who could force them to leak/release your data. OpenAI is already working with the US military and removed their restrictions on allowing their AI to be used for military purposes

https://time.com/6556827/openai-us-military-cybersecurity/

Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model

#80

Post by brightball »

fifilura wrote: post #34
rgbrgb wrote: post #21

So it looks like it scores 76.5% on SQL-Eval [0], a bit behind GPT-4 at 83% and sqlcoder-15b at 78%. What kind of applications would this be useful for? What can you build with an AI data science intern that's right 75% of the time? As a programmer who always has to look stuff up when I SQL, I could definitely see asking something like this for a first draft of a query but it seems like I'm slightly better off asking…

1. If you are a a programmer I think you should learn SQL. It will give you a different perspective on programming that I think is invaluable. (I.e. programming without for loops) 2. Combining and slicing data is a craft, and doing it subtly wrong in one step can lead to fatal errors in the outcome. And most importantly, it can be very difficult to notice. Numbers don't smell. That is why I would be very hesitant to…

I’ve been working in this industry professionally for close to 25 years now. By far the most valuable and useful language I have learned is SQL.
Post reply on HN