I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a semantic data modeling API will yield a more useful/practical result. You may even find the semantic layer makes a natural language approach moot in many cases. In Zillion* I added an experimental feature that uses OpenAI to form a report API call from natural l…
Fine-tuning GPT-3.5-turbo for natural language to SQL
71–77 of 77 posts
Re: Fine-tuning GPT-3.5-turbo for natural language to SQL
#72At a certain level of complexity it is easier to write the damn SQL than it is to explain your query to GPT.
Re: Fine-tuning GPT-3.5-turbo for natural language to SQL
#73I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a semantic data modeling API will yield a more useful/practical result. You may even find the semantic layer makes a natural language approach moot in many cases. In Zillion* I added an experimental feature that uses OpenAI to form a report API call from natural l…
Can you describe in more detail what you mean by semantic? Doesn’t the DB schema capture the business logic to a large extent?
A semantic data layer is an abstraction layer that sits between the raw data stored in databases and the applications or analytics tools that use this data. The purpose of this layer is to provide a consistent, business-friendly interface to data that might be stored in a variety of formats, tables, or systems. By doing so, the semantic data layer helps in simplifying complex data structures into more meaningful, understandable models. Key Features:
Unified View: It provides a unified view of data from multiple sources, making it easier for users to access and understand the data without having to know the underlying structure or complexity.
Data Governance: It can enforce business rules, security policies, and data quality measures, ensuring that data is consistent, compliant, and accessible only by authorized users.
Flexibility: A semantic layer is often designed to be flexible, allowing business users to adapt to changes in the data model without requiring changes to the applications themselves.
Query Simplification: The layer simplifies the process of querying data by providing a more user-friendly way of accessing and manipulating data, often through a drag-and-drop interface or other graphical tools.
Decoupling: It decouples application development from data source changes. When underlying data changes, you don't necessarily have to update all the applications that use it; you might only need to update the semantic layer.
Data Integration: It can integrate data from multiple sources, providing a single "source of truth" for business users and applications.
Purpose: Simplification: Make data more accessible and easier to understand for non-technical users.
Consistency: Ensure that everyone is working from the same definitions and business rules.
Efficiency: Reduce the time and complexity involved in generating reports, analytics, and other data-driven functions.
Data Quality: Help to enforce data quality and governance policies.
Security: Provide a mechanism for enforcing security rules on who can see or modify data.
Adaptability: Enable quicker adaptation to changes in business needs or data structures.
By providing a semantic data layer, organizations can ensure that their data is not only high-quality and secure but also that it can be easily used for making informed decisions. This is particularly valuable in today's data-driven business environment.Re: Fine-tuning GPT-3.5-turbo for natural language to SQL
#74I think natural language to SQL is a worthwhile goal and will have its use cases, but perhaps fine-tuning a model on a semantic layer and having it talk to a semantic data modeling API will yield a more useful/practical result. You may even find the semantic layer makes a natural language approach moot in many cases. In Zillion* I added an experimental feature that uses OpenAI to form a report API call from natural l…
Yeah it is literally billion dollar question how to do that querying. Is there some good papers or blog posts that go through different approaches?
Re: Fine-tuning GPT-3.5-turbo for natural language to SQL
#75Earlier quoted context omitted.
I tried this to see. It does suggest using validators, though in a way that mostly solves the problem. The failure mode is when the string is valid complete or extended. It does have a bit of a workaround for that. https://chat.openai.com/share/54cb8876-96ff-4434-a479-4d2dde...
using the exception only works at whole symbol level, i.e. with "cat do" as imput No terminal matches 'd' in the current parser context, at line 1 col 5 cat do ^ Expected one of: * CAT * SPACE * FISH * DOG Next valid symbols: ['* CAT\n\t* SPACE\n\t* FISH\n\t* DOG']
Re: Fine-tuning GPT-3.5-turbo for natural language to SQL
#76Earlier quoted context omitted.
using the exception only works at whole symbol level, i.e. with "cat do" as imput No terminal matches 'd' in the current parser context, at line 1 col 5 cat do ^ Expected one of: * CAT * SPACE * FISH * DOG Next valid symbols: ['* CAT\n\t* SPACE\n\t* FISH\n\t* DOG']
I'm not sure I understand, what that reports is enough to solve it right? Was the problem you couldn't get it to solve it from that point?
Or given the context space is a valid production from the last error, but then I would need to roll back the incomplete "do" terminal and lose that context.
Re: Fine-tuning GPT-3.5-turbo for natural language to SQL
#77Earlier quoted context omitted.
I'm not sure I understand, what that reports is enough to solve it right? Was the problem you couldn't get it to solve it from that point?
No because if I add space to the original string I don't get a valid string Or given the context space is a valid production from the last error, but then I would need to roll back the incomplete "do" terminal and lose that context.
Yeah, it's telling you what it was expecting when it hit an invalid token.
You take the invalid part and filter the list of suggestions with it. If you tell chatgpt about the problem it solves it.
(Edit - when I was running this, having any number of spaces was parsed as valid btw if that's the concern - space was a valid option)