11–20 of 70 posts
It's amazing what people will do for clout. His whole reputation is ruined. What was Schumer's endgame?
That's what I'm wondering. Did he think that nobody would bother checking it? Then he was saying all that stuff about the model being "corrupted during upload" - maybe he didn't think it was going to get as much traction as it did?
Context: someone announced a Llama 3.1 70B fine tune with incredible benchmark results a few days ago. It's been a dramatic ride: - The weight releases were messed up: released Lora for Llama 3.0, claiming it was a 3.1 fine tune - Evals initially didn't meet expectations when run on released weights - The evals starting performing near/at SOTA when using a hosted endpoint - Folks are finding clever ways to see what m…
Also, after GPT-4o they switched to a llama checkpoint (probably 405B-inst), so now the tokenizer is in common (no more tokenization trick).
It's amazing what people will do for clout. His whole reputation is ruined. What was Schumer's endgame?
Earlier quoted context omitted.
I haven’t followed this story. What did he do that ruined his reputation? The story link here is broken for me.
An AI engagement farmer on twitter claimed to create a llama 3.1 fine tine, trained on "reflection" (ie internal thinking) prompting that outperformed the likes of Llama 405B and even the closed source models on benchmarks. The guy says that the model is so good because it was tuned on data generated by Glaive AI. He tells everyone he uses Glaive AI and that everyone else should use it too. Releases the model on HF,…
Earlier quoted context omitted.
An AI engagement farmer on twitter claimed to create a llama 3.1 fine tine, trained on "reflection" (ie internal thinking) prompting that outperformed the likes of Llama 405B and even the closed source models on benchmarks. The guy says that the model is so good because it was tuned on data generated by Glaive AI. He tells everyone he uses Glaive AI and that everyone else should use it too. Releases the model on HF,…
This is accurate, but you don't need to censor GlaiveAI. They helped create the model. They're complicit in the scam.
And yes, you're correct. Glaive employee(s) contributed to the model uploaded on HF.
Okay, let's think this through step by step. Isn't 'reflection thinking' a pretty well known technique in the AI prompt field? So this model was supposed to be so much better... why, exactly? It makes very little sense to me. Is it just about separating the "reflections/chain of thoughts" from the "final output" via specific tags?
I don't think it is 100% a scam, as in, his technique does improve performance, since a lot of the benefits can be replicated by a system prompt, but the wild performance claims are probably completely fabricated.