Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fine-tuning GPT-3.5-turbo for natural language to SQL

medium.com

Post reply on HN
Print view View original

61–70 of 77 posts

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#61

Post by gloosx »

TedDallas wrote: post #54

At a certain level of complexity it is easier to write the damn SQL than it is to explain your query to GPT.

In the end, the one who possesses knowledge will turn it into value going the most obvious and straightforward way: writing a piece of useful SQL or code hot from the head.

GPT is a tool for learners, and they will keep shooting them feet until they learn, just the weapon is different now.

Look at the law system, written in natural language. It is misleading and doesn't work well, so we have to gather thousands of people around courts doing non-deterministic work in order to process them. Natural language is a tool for learning, not making systems. You have to shrink your vocabulary down to code at some point in order to make systems, and you can do it much faster and better than GPT.

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#62

Post by disgruntledphd2 »

avereveard wrote: post #55
sillysaurusx wrote: post #44

Earlier quoted context omitted.

GPT 3.5 or 4? Surprisingly it makes a huge difference. I think a lot of peoples’ impressions are with 3.5, but many startups couldn’t have been built on it, whereas with 4 they can. If it was 4 I’d be curious about the specific problem if you’d be willing to link to the chat.

I had similar issue with gpt4, was looking for a library that given a grammar and a string, would produce a list of next valid symbols. Gpt4 only ever suggested grammar validator to the point I had given up and was going to write a grammar generator, and so I started looking for the equivalent of antlr in python, and in three searches I find out nltk.grammar that actually solves the original problem It's not a new li…

If it hasn't been used much/talked about much for this purpose then you wouldn't expect it to, right?

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#63

Post by mejutoco »

sillysaurusx wrote: post #27
bob1029 wrote: post #25

We've been chasing this rabbit since the beginning. It currently seems to be uncatchable for the use cases that would be most valuable to us - writing complex queries we've never seen before. Our use case seems to confound any notion of training or fine-tuning, since the cases we need the most help with are also the ones we have the fewest examples of. Instead of going for generative, few-shot models, I am starting t…

I think you've fallen into the trap of "AIs don't generalize, they memorize." But they do in fact generalize. The reason ChatGPT is so valuable is precisely because it can help out with situations that have never been seen before, not because it merely unlocks old preexisting knowledge. The fella who saved their dog with ChatGPT comes to mind. https://nypost.com/2023/03/27/chatgpt-saved-my-dogs-life-aft...

> With ChatGPT, you cannot realistically explain why you got some output in a way that anyone other than an AI/ML expert would find satisfying

IMHO OP is talking about "explainability" of the results, which is notoriously bad for current AI. For certain applications (idk if SQL would be one but mortgage application might be one) it is required to be able to explain how the computer got to the decision.

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#64

Post by Y_Y »

nl wrote: post #58
PheonixPharts wrote: post #49

Earlier quoted context omitted.

I think you have fallen into the trap of mistaking interpolation for generalization . Working with these models every day, it's clear that they can certainly interpolate between points in latent space and generate sensible answers to unseen questions, but it's pretty clear that they don't generalize . I've seen far to many examples of models failing to display any sense of generalization to believe otherwise. That's…

Given a multi-dimensional latent space with enough dimensions it's hard to imagine cases of generalization that aren't interpolation between points in latent space (given enough data). The one possible exception is logical inference, and this problem seems tractable with tool use or programming.

I object on geometric grounds. You can't interpolate outside the convex hull, if your have an outlier in any dimension then you're going to need to extrapolate, that seems to me like a reasonable way to ask for generalisation that isn't interpolation.

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#65

Post by avereveard »

disgruntledphd2 wrote: post #62
avereveard wrote: post #55

Earlier quoted context omitted.

I had similar issue with gpt4, was looking for a library that given a grammar and a string, would produce a list of next valid symbols. Gpt4 only ever suggested grammar validator to the point I had given up and was going to write a grammar generator, and so I started looking for the equivalent of antlr in python, and in three searches I find out nltk.grammar that actually solves the original problem It's not a new li…

If it hasn't been used much/talked about much for this purpose then you wouldn't expect it to, right?

true, but then it's neither interpolating between the public documentation nor extrapolating from the possible similarities between concepts

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#67

Post by lukev »

This brings up a question I've had about OpenAI's fine-tuning APIs.

The training format is a series of question/answer pairs: i.e, what we might think of as "supervised" learning. It can be challenging to build good data sets for this scenario.

But transformer-based models actually don't require supervised learning: the bulk training of the GPT family is to just throw masses of text at it and have it do next-token prediction.

What's going on with the difference? When I "fine-tune" GPT-3.5 turbo, am I actually training the transformer, or the RLHF model that sits on top? Or both?

More to the point, is there any way to fine-tune OpenAIs models in a n "unsupervised" fashion? I.e, if I want to teach a model SQL, do I need to get a curated data set of question/answer pairs, or I can I just dump in a bunch of schema and SQL queries that will in theory make the model "better at SQL" in a generic way?

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#68

Post by IanCal »

avereveard wrote: post #55
sillysaurusx wrote: post #44

Earlier quoted context omitted.

GPT 3.5 or 4? Surprisingly it makes a huge difference. I think a lot of peoples’ impressions are with 3.5, but many startups couldn’t have been built on it, whereas with 4 they can. If it was 4 I’d be curious about the specific problem if you’d be willing to link to the chat.

I had similar issue with gpt4, was looking for a library that given a grammar and a string, would produce a list of next valid symbols. Gpt4 only ever suggested grammar validator to the point I had given up and was going to write a grammar generator, and so I started looking for the equivalent of antlr in python, and in three searches I find out nltk.grammar that actually solves the original problem It's not a new li…

I tried this to see. It does suggest using validators, though in a way that mostly solves the problem. The failure mode is when the string is valid complete or extended. It does have a bit of a workaround for that.

https://chat.openai.com/share/54cb8876-96ff-4434-a479-4d2dde...

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#69

Post by mistercow »

lukev wrote: post #67

This brings up a question I've had about OpenAI's fine-tuning APIs. The training format is a series of question/answer pairs: i.e, what we might think of as "supervised" learning. It can be challenging to build good data sets for this scenario. But transformer-based models actually don't require supervised learning: the bulk training of the GPT family is to just throw masses of text at it and have it do next-token pr…

My understanding is that there’s no RLHF model sitting on top of ChatGPT. There’s a separate reward model during training, but that reward model is just used to fine-tune the decoder model. This is sort of confusing in the literature, because they will switch from calling it a “language model” to a “policy”, once they’ve used PPO, but if you trace the citations back to [1], you can see that the policy is initialized with a pretrained language model (in that case, GPT-2).

What I don’t know is how they’re applying RLHF after user fine-tuning. Are they redoing the PPO with the original reward model after tuning on your input? Are they just letting it slide, and hoping that the fine-tuning doesn’t cause the model to forget the RLHF? It’s unclear from what I’ve read.

[1] https://arxiv.org/pdf/1909.08593.pdf

Re: Fine-tuning GPT-3.5-turbo for natural language to SQL

#70

Post by avereveard »

IanCal wrote: post #68
avereveard wrote: post #55

Earlier quoted context omitted.

I had similar issue with gpt4, was looking for a library that given a grammar and a string, would produce a list of next valid symbols. Gpt4 only ever suggested grammar validator to the point I had given up and was going to write a grammar generator, and so I started looking for the equivalent of antlr in python, and in three searches I find out nltk.grammar that actually solves the original problem It's not a new li…

I tried this to see. It does suggest using validators, though in a way that mostly solves the problem. The failure mode is when the string is valid complete or extended. It does have a bit of a workaround for that. https://chat.openai.com/share/54cb8876-96ff-4434-a479-4d2dde...

using the exception only works at whole symbol level, i.e. with "cat do" as imput

No terminal matches 'd' in the current parser context, at line 1 col 5

cat do ^ Expected one of: * CAT * SPACE * FISH * DOG

Next valid symbols: ['* CAT\n\t* SPACE\n\t* FISH\n\t* DOG']

Post reply on HN