Earlier quoted context omitted.
Probably not the point, but shouldn't you be able to choose to sample only the tokens for "positive" and "negative" (they're both one token!) instead of (or in addition to) needing to put a request for model to restrict its responses in the context?
Interesting observation :) I guess this is the SQL query you have in mind that uses the LIKE operator: SELECT ChatGPT("Respond to the review with a solution to address the reviewer's concern", review) FROM postgres_data.review_table WHERE ChatGPT("Is the review positive or negative?", review) LIKE "%positive%" AND location = “waffle house”; From a query processing standpoint, both queries should have equivalent perfo…
So like, at the end of all the decoders, the model gives you an output vector; you multiply this by your embeddings to get your token probabilities, then you sample from them to choose a token.
Instead of sampling, you could just look at the probabilities for the tokens "positive" and "negative" and return whichever of those two is highest.