Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

old.reddit.com

Post reply on HN
Print view View original

61–70 of 70 posts

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#61

Post by moralestapia »

throw_me_uwu wrote: post #36
salomonk_mur wrote: post #4

It's amazing what people will do for clout. His whole reputation is ruined. What was Schumer's endgame?

But does reputation work? Will people google "Matt Shumer scam", "HyperWrite scam", "OthersideAI scam", "Sahil Chaudhary scam", "Glaive AI scam" before using their products? He wasted everyone's time, but what's the downside for him? Lots of influencers did fraud, and they do just fine.

This may sound harsh but it's true.

You could do shit things and still come out with people perceiving you as a "winner"; because you got money, status, whatever you wanted, e.g. Adam Neumann. This is "fine" because people want to associate themselves with winners.

Or, you could do pretty much the exact same thing but come out looking as an absolute loser; e.g. SBF, this guy, etc... This is terrible as people do not want to be associated with losers.

IMO, this guy's career is dead, forever.

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#62

Post by JohnMakin »

Would really like to see this float back to the front page rather than getting buried 4+ deep despite its number of upvotes - this is very significant and very damning, and this guy is a real big figure apparently in the AI "hype" space (as far as I understand - that stuff actually hurts my brain to read so I avoid it like the plague).

Evidence I find damning that people have posted:

- Filtering out of "claude" from output responses - would frequently be a blank string, suggesting some manipulation behind the scenes

- errors in output caused by passing in tags in clever ways which the real model will refuse to parse (passed in via base64 encoded string)

- model admitting in various ways that it is claude/built by anthropic (I find this evidence less pursuasive, as models are well known to lie or be manipulated into lying)

- Most damning to me, when people were still playing with it, they were able to get the underlying model to answer questions in arabic, which was not supported on the llama version it was allegedly trained on (ZOMG, emergent behavior?)

Feel free to update this list - I think this deserves far more attention than it is getting.

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#63

Post by Der_Einzige »

petercooper wrote: post #56
throw_me_uwu wrote: post #36

Earlier quoted context omitted.

But does reputation work? Will people google "Matt Shumer scam", "HyperWrite scam", "OthersideAI scam", "Sahil Chaudhary scam", "Glaive AI scam" before using their products? He wasted everyone's time, but what's the downside for him? Lots of influencers did fraud, and they do just fine.

Sure, it's complicated. The core of the AI world right now isn't that large and in many ecosystems it's common for people to speak to each other behind the scenes and to learn about alleged incidents regarding individuals in the space. Such whispering can become an impediment for someone with a "name" in a space, even if not necessarily a full loss of their reputation or opportunities.

Except that bad PR is good PR. Trump proves this daily. Terry A Davis proved this in the context of tech (he coined the term "glowie" in its original racially charged usage in addition to temple OS). If Chris Chan ever learned to code to make the Sonichu game of their dreams, I'm sure that there would be a minor bidding war on their "talent"

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#64

Post by JohnMakin »

JohnMakin wrote: post #62

Would really like to see this float back to the front page rather than getting buried 4+ deep despite its number of upvotes - this is very significant and very damning, and this guy is a real big figure apparently in the AI "hype" space (as far as I understand - that stuff actually hurts my brain to read so I avoid it like the plague). Evidence I find damning that people have posted: - Filtering out of "claude" from…

adding - tokenizer output test showed consistency with claude, this test is allegedly no longer working

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#66

Post by cpthammer »

crimsoneer wrote: post #40

Have we got a "confirmed" from someone reputable/trustworthy yet? Like, it looks pretty compelling to me but I'm not sure I trust this mess of reddit posts/twitter threads/unsourced screenshots from people I don't know yet...

Schumer (or whoever was doing the updates) was continuously changing the api to dodge accusations. I personally replicated the evidence (most critically, the claude meta tag prompt injection) but have no way to prove anything now that it is down.

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#67

Post by petercooper »

Der_Einzige wrote: post #63
petercooper wrote: post #56

Earlier quoted context omitted.

Sure, it's complicated. The core of the AI world right now isn't that large and in many ecosystems it's common for people to speak to each other behind the scenes and to learn about alleged incidents regarding individuals in the space. Such whispering can become an impediment for someone with a "name" in a space, even if not necessarily a full loss of their reputation or opportunities.

Except that bad PR is good PR. Trump proves this daily. Terry A Davis proved this in the context of tech (he coined the term "glowie" in its original racially charged usage in addition to temple OS). If Chris Chan ever learned to code to make the Sonichu game of their dreams, I'm sure that there would be a minor bidding war on their "talent"

Yeah, that can certainly be true! Those sorts of people really seem to lean into their quirky reputation, though. I'm not sure someone could do so well in an academic or engineering discipline and maintain a broad level of professional respect with that approach?

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#68

Post by [deleted] »

baegi wrote: post #58
energy123 wrote: post #31

Earlier quoted context omitted.

Even though this was a scam, it's somewhat plausible. You finetune on synthetic data with lots of common reasoning mistakes followed by self-correction. You also finetine on synthetic data without reasoning mistakes where the "reflection" says that everything is fine. The model then learns to recognize output with subtle mistakes/hallucinations due to having been trained to do that.

But wouldn't the model then also learn to make reasoning mistakes in the first place, where in some cases those mistakes could have been avoided by not training the model on incorrect reasoning? Of course if all mistakes are corrected before the final output tokens this is fine, but I could see this method introducing new errors altogether.

[deleted]

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#69

Post by JohnMakin »

JohnMakin wrote: post #62

Would really like to see this float back to the front page rather than getting buried 4+ deep despite its number of upvotes - this is very significant and very damning, and this guy is a real big figure apparently in the AI "hype" space (as far as I understand - that stuff actually hurts my brain to read so I avoid it like the plague). Evidence I find damning that people have posted: - Filtering out of "claude" from…

He has now basically come to some sort of half assed apology on twitter/x:

https://x.com/mattshumer_/status/1833619390098510039?s=46

Re: Confirmed: Reflection 70B's official API is a wrapper for Sonnet 3.5

#70

Post by JohnMakin »

JohnMakin wrote: post #62

Would really like to see this float back to the front page rather than getting buried 4+ deep despite its number of upvotes - this is very significant and very damning, and this guy is a real big figure apparently in the AI "hype" space (as far as I understand - that stuff actually hurts my brain to read so I avoid it like the plague). Evidence I find damning that people have posted: - Filtering out of "claude" from…

the “full explanation” - https://x.com/csahil28/status/1833619624589725762?s=46
Post reply on HN