51–60 of 70 posts
Earlier quoted context omitted.
https://xcancel.com/RealJosephus/status/1832904398831280448
First time I see xcancel. Seems to be faster than the x-thread thing. Has it been around for a long time?
Where exactly is this "official API"?
Context: someone announced a Llama 3.1 70B fine tune with incredible benchmark results a few days ago. It's been a dramatic ride: - The weight releases were messed up: released Lora for Llama 3.0, claiming it was a 3.1 fine tune - Evals initially didn't meet expectations when run on released weights - The evals starting performing near/at SOTA when using a hosted endpoint - Folks are finding clever ways to see what m…
Earlier quoted context omitted.
It's also amazing that GlaiveAI will be synonymous with fraud in ML now, because an investor decided to fake some benchmarks. The founder of GlaiveAI, Sahil Chaudhary also participated in the creation of the model.
I wonder if the other investors will sue.
It's amazing what people will do for clout. His whole reputation is ruined. What was Schumer's endgame?
But does reputation work? Will people google "Matt Shumer scam", "HyperWrite scam", "OthersideAI scam", "Sahil Chaudhary scam", "Glaive AI scam" before using their products? He wasted everyone's time, but what's the downside for him? Lots of influencers did fraud, and they do just fine.
Since the current created legal landscape does not punish fraudsters they keep doing it and succeeding. Same thing as society allowing people to fail upward.
It's amazing what people will do for clout. His whole reputation is ruined. What was Schumer's endgame?
But does reputation work? Will people google "Matt Shumer scam", "HyperWrite scam", "OthersideAI scam", "Sahil Chaudhary scam", "Glaive AI scam" before using their products? He wasted everyone's time, but what's the downside for him? Lots of influencers did fraud, and they do just fine.
Okay, let's think this through step by step. Isn't 'reflection thinking' a pretty well known technique in the AI prompt field? So this model was supposed to be so much better... why, exactly? It makes very little sense to me. Is it just about separating the "reflections/chain of thoughts" from the "final output" via specific tags?
Even though this was a scam, it's somewhat plausible. You finetune on synthetic data with lots of common reasoning mistakes followed by self-correction. You also finetine on synthetic data without reasoning mistakes where the "reflection" says that everything is fine. The model then learns to recognize output with subtle mistakes/hallucinations due to having been trained to do that.
Of course if all mistakes are corrected before the final output tokens this is fine, but I could see this method introducing new errors altogether.
At the same time it also seems like it’d already be baked into the model through RLHF? Basically just a different COT flow?