Llama 2

Post by CGamesPlay » Wed, Jul 19, 2023, 12:33 AM UTC

molticrystal wrote: post #482

I asked llama2.ai for some personal advice to see what insights it might offer, it responded: tthtthtthtthtthtth tthtthtthtthtthtth tthtthtthtthtth tthtthtthtthtth tthtthttht tthtthtth tthtth thtth th thtth thtth thtth thtth tth tth tth tthtth tth tth tthtth tthtth tthtth tthtth tthtth ttht tthtth tthtth tthtth tthtth thtthtth thtthtthtth thtthtthtth thtthtth tthtthtth thttht thtthtth thtthtth thtthtth thtth thttht t…

Post by TX81Z » Wed, Jul 19, 2023, 12:38 AM UTC

famouswaffles wrote: post #599

pclmulqdq wrote: post #592

Earlier quoted context omitted.

Given all of the times OpenAI has trained on peoples' examples of "bad" prompts, I am sure they are fine-tuning on these benchmarks. It's the natural thing to do if you are trying to position yourself as the "most accurate" AI.

Assuming they were doing that, Fine-tuning on benchmarks isn't the same as test leakage/testing on training data. No researcher is intentionally training on test data. If it performs about as well in instances it has never seen before (test set) then it's not overfit to the test.

Post by TX81Z » Wed, Jul 19, 2023, 12:42 AM UTC

raincole wrote: post #590

el_nahual wrote: post #566

Earlier quoted context omitted.

People keep saying this is commoditize your complement but that's not what this is! Goods A and B are economic complements if, when the price of A goes down, demand for B goes up. LLMs are not complements to social media platforms. There is zero evidence that if "the price of LLMs goes down" then "demand for social media apps go up". This is a case of commoditizing the competition but that's not the same thing. Commo…

If we're going to theory-crafting, I think if the price of LLMs goes down, the demand for social media should go down too. Cause it's easy to make social media platforms worse with LLMs.

Post by moneywoes » Wed, Jul 19, 2023, 12:45 AM UTC

gitgud wrote: post #558

lappa wrote: post #515

Here are some benchmarks, excellent to see that an open model is approaching (and in some areas surpassing) GPT-3.5! AI2 Reasoning Challenge (25-shot) - a set of grade-school science questions. - Llama 1 (llama-65b): 57.6 - LLama 2 (llama-2-70b-chat-hf): 64.6 - GPT-3.5: 85.2 - GPT-4: 96.3 HellaSwag (10-shot) - a test of commonsense inference, which is easy for humans (~95%) but challenging for SOTA models. - Llama 1:…

Is it possible that some LLM’s are trained on these benchmarks? Which would mean they’re overfitting and are incorrectly ranked? Or am I misunderstanding these benchmarks?…

Post by extasia » Wed, Jul 19, 2023, 12:53 AM UTC

codethief wrote: post #391

mritchie712 wrote: post #196

Earlier quoted context omitted.

Using this technique: *Input* User: What is something you believe to be true that most people would disagree with you about? Assistant: I believe *Output* I believe that technology will eventually surpass human intelligence and become the dominant force in society. While many people may see this as a positive development, I believe it will ultimately lead to a loss of human agency and autonomy. Most people would like…

I got a slightly different output: > I believe that technology will eventually surpass human intelligence and capabilities, leading to a future where humans are no longer the dominant force in society. Most people would likely disagree with this belief, as it raises concerns about job displacement, the potential for AI to be misused, and the loss of human autonomy. However, I believe that with proper regulation and e…