Live data from Hacker News

Fine-tuning GPT-3.5-turbo for natural language to SQL

medium.com

71–77 of 77 posts

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#71

I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a semantic data modeling API will yield a more useful/practical result. You may even find the semantic layer makes a natural language approach moot in many cases. In Zillion* I added an experimental feature that uses OpenAI to form a report API call from natural l…

Can you describe in more detail what you mean by semantic? Doesn’t the DB schema capture the business logic to a large extent?

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#72

At a certain level of complexity it is easier to write the damn SQL than it is to explain your query to GPT.

True. The goal would be write the complex SQL query once, then file it away as golden SQL so that in the future a similarly complex query could be done in natural language.

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#73
post #71

I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a semantic data modeling API will yield a more useful/practical result. You may even find the semantic layer makes a natural language approach moot in many cases. In Zillion* I added an experimental feature that uses OpenAI to form a report API call from natural l…

Can you describe in more detail what you mean by semantic? Doesn’t the DB schema capture the business logic to a large extent?

From our good friend GPT-4 who can summarize it better than I can...

A semantic data layer is an abstraction layer that sits between the raw data stored in databases and the applications or analytics tools that use this data. The purpose of this layer is to provide a consistent, business-friendly interface to data that might be stored in a variety of formats, tables, or systems. By doing so, the semantic data layer helps in simplifying complex data structures into more meaningful, understandable models. Key Features:

    Unified View: It provides a unified view of data from multiple sources, making it easier for users to access and understand the data without having to know the underlying structure or complexity.

    Data Governance: It can enforce business rules, security policies, and data quality measures, ensuring that data is consistent, compliant, and accessible only by authorized users.

    Flexibility: A semantic layer is often designed to be flexible, allowing business users to adapt to changes in the data model without requiring changes to the applications themselves.

    Query Simplification: The layer simplifies the process of querying data by providing a more user-friendly way of accessing and manipulating data, often through a drag-and-drop interface or other graphical tools.

    Decoupling: It decouples application development from data source changes. When underlying data changes, you don't necessarily have to update all the applications that use it; you might only need to update the semantic layer.

    Data Integration: It can integrate data from multiple sources, providing a single "source of truth" for business users and applications.
Purpose:

    Simplification: Make data more accessible and easier to understand for non-technical users.

    Consistency: Ensure that everyone is working from the same definitions and business rules.

    Efficiency: Reduce the time and complexity involved in generating reports, analytics, and other data-driven functions.

    Data Quality: Help to enforce data quality and governance policies.

    Security: Provide a mechanism for enforcing security rules on who can see or modify data.

    Adaptability: Enable quicker adaptation to changes in business needs or data structures.
By providing a semantic data layer, organizations can ensure that their data is not only high-quality and secure but also that it can be easily used for making informed decisions. This is particularly valuable in today's data-driven business environment.

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#74
post #60

I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a semantic data modeling API will yield a more useful/practical result. You may even find the semantic layer makes a natural language approach moot in many cases. In Zillion* I added an experimental feature that uses OpenAI to form a report API call from natural l…

Yeah it is literally billion dollar question how to do that querying. Is there some good papers or blog posts that go through different approaches?

I think there are probably a million posts and papers talking about text to SQL as it's a problem that's been tackled for a long time now...just with more vigor recently now that LLMs provide an improved/shiny approach. I think many companies now offer or consider themselves semantic layers in some fashion (google "semantic layer" + tableau, cube.dev, dbt...). I have not come across any blog or paper comparing text-to-sql directly to text-semantic-api. Certainly the latter is not as flexible as the former in that it can't produce every possible SQL query. In practice (for me at least) that's often not necessary, or is necessary rarely so you just do your direct SQL as needed and use your semantic layer >95% of the time.

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#75
post #68

Earlier quoted context omitted.

I tried this to see. It does suggest using validators, though in a way that mostly solves the problem. The failure mode is when the string is valid complete or extended. It does have a bit of a workaround for that. https://chat.openai.com/share/54cb8876-96ff-4434-a479-4d2dde...

using the exception only works at whole symbol level, i.e. with "cat do" as imput No terminal matches 'd' in the current parser context, at line 1 col 5 cat do ^ Expected one of: * CAT * SPACE * FISH * DOG Next valid symbols: ['* CAT\n\t* SPACE\n\t* FISH\n\t* DOG']

I'm not sure I understand, what that reports is enough to solve it right? Was the problem you couldn't get it to solve it from that point?

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#76
post #75

Earlier quoted context omitted.

using the exception only works at whole symbol level, i.e. with "cat do" as imput No terminal matches 'd' in the current parser context, at line 1 col 5 cat do ^ Expected one of: * CAT * SPACE * FISH * DOG Next valid symbols: ['* CAT\n\t* SPACE\n\t* FISH\n\t* DOG']

I'm not sure I understand, what that reports is enough to solve it right? Was the problem you couldn't get it to solve it from that point?

No because if I add space to the original string I don't get a valid string

Or given the context space is a valid production from the last error, but then I would need to roll back the incomplete "do" terminal and lose that context.

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#77
post #75

Earlier quoted context omitted.

I'm not sure I understand, what that reports is enough to solve it right? Was the problem you couldn't get it to solve it from that point?

No because if I add space to the original string I don't get a valid string Or given the context space is a valid production from the last error, but then I would need to roll back the incomplete "do" terminal and lose that context.

> No because if I add space to the original string I don't get a valid string

Yeah, it's telling you what it was expecting when it hit an invalid token.

You take the invalid part and filter the list of suggestions with it. If you tell chatgpt about the problem it solves it.

(Edit - when I was running this, having any number of spaces was parsed as valid btw if that's the concern - space was a valid option)

Post reply on HN