Hard to claim success with "complex" questions if you don't account for business context and organizational nuances. For example, "active" listings on Redfin may be a combination of days on market, last open house, last update, etc instead of a Boolean flag called "is_active". How can we expect models to generate correct SQL at the enterprise level without providing a support structure of business context? The model can only be so good.
Show HN: Natural-SQL-7B, a strong text-to-SQL model
141–150 of 171 posts
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#142Earlier quoted context omitted.
I’ve been working in this industry professionally for close to 25 years now. By far the most valuable and useful language I have learned is SQL.
I have been working in this industry for 30 years now. SQL is one of the most ugliest programming language I have ever working with. Like Perl, it's write once, hope you never need to read, language. SQL was created in 60s. It has not really kept up with the pace of modern programming language ergonomics. It was made for a single person executing a batch job query pulling data from the database. On the other hand, I…
> SQL was created in 60s
The IBM System R and SEQUEL paper was 1974, while Oracle 2 was the first commercial database which added it in 1979.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#143Earlier quoted context omitted.
I’ve been working in this industry professionally for close to 25 years now. By far the most valuable and useful language I have learned is SQL.
I actually work in the Data/AI space and SQL has been of limited use to me. Whilst Snowflake is pretty popular the days of elaborately modelled EDWs are long gone. And so typically I find I am doing queries, transformations etc in some abstraction layer e.g. Spark on a data lake, ORM for web applications etc.
> the days of elaborately modelled EDWs are long gone
They're more prevalent than ever in my experience. Consider the popularity of dbt.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#144Earlier quoted context omitted.
For those people who would rather use an ORM, its worth mentioning that ORMs write very bad, un-performant SQL under the hood. They may save you a bit of time initially but when your company gets bigger, the ORMs will become a bottleneck.
"Bad" and "un-performant" are relative terms and as your company gets bigger, you're increasingly more and more likely to have colleagues who write even worse queries than an ORM would. For example I've encountered queries that are not only slow, but they generate several hundred megabytes of output all of which is sent to the user's web browser where JavaScript selects the relevant two kilobytes of data to show the…
I guess once you get to a certain point, you need to hire engineers that have strong SQL skills in order to scale.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#145Earlier quoted context omitted.
Could you give an example?
This will be useful if you have a table with some related history records, e.g., products with product price history, and you want to get the latest price. The lateral join would get all prices for the current product, sort them by date and then pick the top row.
Is a lateral join more efficient than just saying where product = current sort by date desc ?
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#146I don't think any of those text-to-sql models are solving the right problems. The hard part is not syntax or I don't know how to write a group by query. Most data scientists and engineers spend more time on understanding the meaning of the data. One cannot simply look at a 50 columns table in Snowflake and guess what columns are by their names. For example, we have 10 columns in one tables, all named ...price. We hav…
I totally agree that the value is in understanding the data. However, as a tool, do you see value in being able to quickly skeleton out the query? Autocomplete in code is a reasonable analogy to me.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#147Earlier quoted context omitted.
I actually work in the Data/AI space and SQL has been of limited use to me. Whilst Snowflake is pretty popular the days of elaborately modelled EDWs are long gone. And so typically I find I am doing queries, transformations etc in some abstraction layer e.g. Spark on a data lake, ORM for web applications etc.
> the days of elaborately modelled EDWs are long gone They're more prevalent than ever in my experience. Consider the popularity of dbt.
dbt is not elaborately modeled at all, it eschews all the up front work of data modeling with ... well basically a lot of code and poorly maintained balls of mud.
Don't forget the models other's create for you - often hilariously slow code to present a set of facets that often barely align with your business delivery needs; and don't forget to sync it even more slowly with Fivetran, the DE platform of the future!
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#148Earlier quoted context omitted.
1) OpenAI has consistently gone back on commitments it has made 2) Sam Altman has a shady track record publicly, and if you believe the things people say privately he has consistently done business very dishonestly throughout his career. He is the CEO, and virtually the entire executive team are people he brought in from his network. It’s his company. 3) To give one example of many, OpenAI recently changed the terms…
Hm. I disagree that executive level shenanigans translate to real world non-compliance with privacy law when there is an explicit request to delete/not store data.
Even if there weren't shenanigans, it's valid to be concerned. Incentives lead to outcomes. Companies do a cost-benefit analysis, if the legal/reputational costs are less than what they stand to gain, history shows that they'll do the thing and then lie about it. Sam might be uniquely resistant to this due to a personal ethical code, but it's impossible to know for sure given that I can't read his mind.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#149previous palantir & scale ai engineer w/ an ai data science startup here. we found that RAG on a large corpus was the best outcome, not just RAG on queries, but RAG across any data the org uploaded to our system, athena intelligence. semantic search on documents, slack, and other sources provided better outcomes w/ on-prem models too. this has allowed us to unlock some novel analytics workflows including multi-modal, figma style, etc. we also just opened up our platform for public access. currently working with enterprises like anheuser-busch.
Re: Show HN: Natural-SQL-7B, a strong text-to-SQL model
#150Earlier quoted context omitted.
> the days of elaborately modelled EDWs are long gone They're more prevalent than ever in my experience. Consider the popularity of dbt.
dbt is not elaborately modeled at all, it eschews all the up front work of data modeling with ... well basically a lot of code and poorly maintained balls of mud. Don't forget the models other's create for you - often hilariously slow code to present a set of facets that often barely align with your business delivery needs; and don't forget to sync it even more slowly with Fivetran, the DE platform of the future!
> dbt is not elaborately modeled at all
This doesn't make any sense, or I'm guessing you've never actually used it. Modeling is something you can do with dbt, not what dbt does (or is, or can be?). I've used it to create data marts and EDW's with hundreds of tables, no differently than I would have created a decade ago with other tools.