241–250 of 303 posts
Ok not bad and pretty damn good performance: > Give me a writing prompt for a sci-fi story. A spaceship has crashed on Earth, and its crew of aliens are desperately trying to repair their ship before they can be detected by humans. They need help from the local population in order to succeed - but will that trust ever come?
Earlier quoted context omitted.
Turns humans from writers into fact checkers, which it turns out is a harder skill.
What sort of facts are you thinking of? There are plenty of surface level facts that another model or a human should easily be able to verify as true or not. Was President Obama the 41st president of the United States, does 2+2 = 5, was Elvis the son of an artist, etc
The former would be easy enough to fix - though it undermines the utility and if not caught could carry ramifications for people, but the latter genuinely concerns me.
Earlier quoted context omitted.
This is one of the things that LLMs seem to be struggling with. They're very good at tone, and feeling right without actually being write. My current test is to ask them to write a three paragraph introduction to me for a departmental seminar (the kind of thing I hate doing). Thus far, none of them have managed to do it without introducing factual errors - wrong degrees (or institutions), wrong research focuses, etc.…
> They're very good at tone, and feeling right without actually being write. The text they generate is probably free of errors like this one as well
Earlier quoted context omitted.
One could imagine training a much smaller LLM on synthetic data that didn't contain much or any real knowledge, but just examples of in-context reasoning. E.g. the training data might look like "a fooble is to a tooble as a fabble is to a" with the answer "tabble". So you feed it tons of these kind of nonsense training data that forces it to only learn the in-context reasoning part of language, and none of the world…
The reason, i suspect, that this method won't work, is because the training dataset has knowledge embedded in it, regardless of whether you think it only contains in-context logic. The evidence is that there are these "glitched" tokens : https://www.youtube.com/watch?v=WO2X3oZEJOA if you watched to the end of that video, it explores a very plausible reason why such glitched tokens exist - it is because the training d…
GPT4 does not suffer from the same glitched tokens as GPT3, presumably because it uses a different tokenizer.
Furthermore, there are LLMs that operate on single bytes instead of multi-character tokens, totally obviating that problem.
Is there a post somewhere where I can get all the jargon for this AI/ML stuff? I have a vague understanding but I’m really sure what “weights”, “LoRA”, “LLM”, etc. are to really understand where each tool and concept fit in.
I was like this a week ago. Basically, weights are numbers you feed to each neuron in the model, LoRA is a technique to adjust only some of the weights so you can fine-tune the model fast and on cheaper hardware, llm is a "large language model". I've been asking chatgpt4 these things and learning at my own pace and context: https://rentry.co/vafkn
My own questioning of chatgpt on LoRA returns "Logistic Regression Algorithm" instead. Looks like it's too new to ChatGPT.
[1] https://arxiv.org/abs/2106.09685 [2] https://rentry.co/vafkn
Earlier quoted context omitted.
The weights aren't the neurons, they're the connection strengths between the neurons.
You mean the same neurons that don't really exist, other than as a connection from weight to weight, so quite reasonable to see as the weights themselves, because what else is available for us to reference as neurons in absentia those connections?
“Neurons” are an abstraction that exists logically “between” the weights, but the weights themselves don’t have the features of neurons. (In that each weight is the weight of a connection between a neurons (or between a neuron and an input/output.) Weights are more synapses than neurons.
Zero-shot translation capabilities (note: doesn't work with all languages well) > translate into japanese: "I am going to school today, but it is raining." 日本語で 「今天は学校に行きますが、雨が吹いている」
I expect translation especially of non-technical sentences to be a solved problem, but can anyone confirme the translations are accurate?
Earlier quoted context omitted.
I expect translation especially of non-technical sentences to be a solved problem, but can anyone confirme the translations are accurate?
Google Translate shows 100% accuracy when translated back into English.
Earlier quoted context omitted.
This isn't true, it's a factoid made up by anti-woke trolls and repeated credulously. "Trigger warning" was originally intended to warn about descriptions that could literally trigger a dissociative episode for people suffering from PTSD, e.g. victims of child abuse or sexual violence. The usage gradually expanded to anything that could be distressing to anyone for any reason, which is kind of belittling to the origi…
I'm afraid that you're wrong. See page 11. https://s.wsj.net/public/resources/documents/stanfordlanguag... . Stanford seems to have beaten the "anti-woke trolls" at their own game.
> The phrase can cause stress about what's to follow. Additionally, one can never know what may or may not trigger a particular person.