61–70 of 105 posts
Caution. Language models do not know what is salient to a human. They also have a strong bias toward information that they have seen frequently. Research will contain a larger amount of new information and it's that new information which is most valuable to us, but least relevant according to the models.
Earlier quoted context omitted.
I asked it how to do something with an AWS cli tool. ChatGPT invented a new parameter that looked like a programmer would come up with it and it would do exactly what I was looking for (I assumed many people had my problem before). aws cloudfront update-distribution --id --distribution-config --no-reset-origin-access-identity Took me a while to figure out that the parameter --no-reset-origin-access-identity was not o…
Did you not think to check the docs? (Srs question)
With this new class of products based on crafting prompts that best exploit a GPT's algorithm and training data, are we going to start seeing pull requests that tweak individual parts or words of the prompt. I'm also curious how the test suite for projects like this would look for specific facts or phrases to be contained in the responses for specific inputs.
What if the paper is longer than what fits in a prompt?
Disclosure: I work at MSFT but not on Bing
Earlier quoted context omitted.
I like the way you flipped it around. I'm a big believer of using all these new AI services/programs as tools to enhance my workflows, not replacing them.
"The best model was truthful on 58% of questions, while human performance was 94%." https://arxiv.org/abs/2109.07958
It's also clear that RHLF models can be given instructions to reduce this issue. And in many production LLM models, something called few shot is used - in context learning - where you provide several examples and then ask for a new case. The accuracy is again improved this way, because the model "deduces" that you are being serious and humourless, when asking about mirrors breaking, and are not asking in the context of a story.
It's also one of the only datasets that fails to increase with scale (There was a big, cash-paying high incentive challenge to find other datasets, and it pretty much didn't find any that can't be worked around or are not experienced by RLHF models like ChatGPT). So it doesn't represent a wider trend of truthfulness above human accuracy being impossible (dated/data drift on the other hand, clearly remains an issue).
Ive used ChatGPT to make short factual videos for YouTube and honestly it's a bit worrying with supposed 'facts' I would not suggest anyone to use ChatGPT outputs for actual knowledge at this point.
I asked it how to do something with an AWS cli tool. ChatGPT invented a new parameter that looked like a programmer would come up with it and it would do exactly what I was looking for (I assumed many people had my problem before). aws cloudfront update-distribution --id --distribution-config --no-reset-origin-access-identity Took me a while to figure out that the parameter --no-reset-origin-access-identity was not o…
Earlier quoted context omitted.
Abstracts can also be close to a full page long. While this doesn't have to be bad, it's usually more information you're looking for (especially if you were only going to read the abstract anyway).
I can skim several full pages in the time it takes just to send off the abstract and get the response back...
Earlier quoted context omitted.
Length isn't always the indicator. Abstracts can be information dense, and sometimes more words is simpler to understand, or sometimes it just has jargon. For example, this paper on GPT-3: "Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still…
1. What a terrible abstract. That abstract makes me hesitant to bother reading the paper at all. An overly long, overly detailed abstract is a sign of a bad(ly written) paper in my experience. 2. What a useless summary! This summary is so dumbed down it could describe literally any paper on LLMs. This would give me zero information on whether the paper is worth further reading.