Live data from Hacker News

Anthropic’s paper smells like bullshit

djnn.sh

331–340 of 349 posts

Re: Anthropic’s paper smells like bullshit

#331

When I worked at a FAANG with a "world leading" AI lab (now run by a teenage data labeller) as an SRE/sysadmin I was asked to use a modified version of a foundation model which was steered towards infosec stuff. We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box. It helped a little, but it wasn't all that helpful. but in terms of coordination, I can't see how it would be useful.…

[flagged]

I used to work at RL so I instantly knew what he was referring to.

Re: Anthropic’s paper smells like bullshit

#332
post #321
post #291

Earlier quoted context omitted.

I watched this interview when I first heard about Alexandr Wang. I'd seen he was the youngest self made billionaire, which is a pretty impressive credential to have under your belt, and I wanted to see if I could get a read on what sets him apart. Unfortunately he doesn't reveal any particular intelligence, insight, or drive in the interview, nor does he in other videos I found. Possibly he hides it, or possibly his…

Or maybe, just maybe, becoming a billionaire has way more to do with luck than anything else. I don't know about any billionaire in the history of billionaires who appears to have gotten there solely based on special abilities. Being born into the right circumstances is all it really takes.

> Being born into the right circumstances is all it really takes.

You do still need to do the work. People have squandered golden opportunities because they didn't put in the effort.

Re: Anthropic’s paper smells like bullshit

#333
post #58

That whole article felt like "Claude is so good Chinese hackers are using it for espionage" marketing fluff tbh

I'm baffled at the assumption that concrete and specific evidence of international (presumably) hostile espionage that is currently being enacted using X and Y software and Z specific techniques, would be publicly released in real time.

I can't think of a single situation in which it would be reasonable to assume that.

It's not like we even get governments or corporations saying 'oh hey, just raising the alarm that bad people are using this Photoshop feature to create fake cheques which they're then depositing into their accounts, so bank staff, be on the lookout!' Because yeah, that's a Photoshop ad.

And it's not like espionage is new, like the Chinese side have been ramping up for decades now, or like there has ever been an expectation that companies with suspicions or evidence of international subterfuge should... should lay it all out in a public report? Is that really what the article is expecting?

I don't even think the UK has got around to officially acknowledging Funny Business in UK-Argentinian relations in any documents or events during the 80's, and the secret was rather given away around the time we went to all out literal war. We know things must have built up before the day war was declared, but nobody expected every escalation of diplomatic unrest to be communicated to the entire nation in real time. Because that would be deranged.

Idk, maybe I'm misunderstanding something about the article. I feel like it isn't in my field, although I'm not entirely sure what field specific knowledge I'm missing to make sense of this.

I would very much like to agree with the sentiment, I'm always down for some AI-dissing and a bit of tin foil hat Big Tech Analyses.

But I couldn't get much more than "This company is lying because it didn't give me any Chinese State secrets, let alone explain how to get stars secrets using their software,' which feels so censored as to be pointless, or just kinda wildly petty and ill informed

Re: Anthropic’s paper smells like bullshit

#334

Earlier quoted context omitted.

I think the expectation is more that serious people have their work checked over by other serious people to catch the obvious mistakes.

Every time you have your work "checked over by other serious people", it eliminates 90% of the mistakes. So you have it checked over twice so that 99% of mistakes have been eliminated, and so on. But it never gets to 0% mistakes. That's my experience anyway.

Every time you have your work "checked over by other serious people", it only means it's been checked over by other people. You can't attach a metric to this process. Especially when it comes to security, adding more internal eyeballs doesn't mean you've expanded coverage.

One of the things I enjoy about Penn and Teller is that they explain in detail how their point of view differs from the audiences and how they intentionally use that difference in their shows. With that in mind you might picture your org as the audience, with one perspective diligently looking forwards.

Re: Anthropic’s paper smells like bullshit

#335

Earlier quoted context omitted.

If we’re sharing vibes, “our product is dangerous” seems like an unusual sales tactic outside the defense industry. I’m doubtful that’s how it works? Meanwhile, another reason to make a press release is that you’ll be criticized for the coverup if you don’t. Also, it puts other companies on notice that maybe they should look for this?

Yeah. You'd think nuclear power would be incredibly popular, given that "our product is dangerous" is a apparently genius marketing strategy. After all, if it can make a whole region of ukraine uninhabitable and be weaponized to turn people into shadows on pavement, it can surely power your fridge. Yet oddly companies making nuclear reactors always market them as being very safe instead of leaning into the danger.

Are there a lot of commercially available nuclear reactors competing for consumers, or is it more of a niche market, like high end designer goods, custom made spectacles etc, that don't generally rely on public advertising campaigns?

I've seen an absurd amount of AI advertising, and very little nuclear reactor advertising, but maybe your point is valid and I'm just not the target audience.

Re: Anthropic’s paper smells like bullshit

#337

When I worked at a FAANG with a "world leading" AI lab (now run by a teenage data labeller) as an SRE/sysadmin I was asked to use a modified version of a foundation model which was steered towards infosec stuff. We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box. It helped a little, but it wasn't all that helpful. but in terms of coordination, I can't see how it would be useful.…

> the same for claude, you're API is tied to a bankaccount, and vibe coding a command and control system on a very public system seems like a bad choice.

Aside from middlemen as others have suggested - You can also just procure hundreds of hacked accounts for any major service through spyware data dump marketplaces. Some percentage of them will have payment already set up. Steal their browser cookies, use it until they notice and cancel / change their password, then move on to the next stolen account. Happens all the time these days.

Re: Anthropic’s paper smells like bullshit

#338

Earlier quoted context omitted.

[flagged]

I propose a project that we name Blarrble, it will generate text. We will need a large number of humans to filter and label the data inputs for Blarrble, and another group of humans to test the outputs of Blarrble to fix it when it generate errors and outright nonsense that we can't techsplain and technobabble away to a credulous audience. Can we make (m|b|tr)illions and solve teenage unemployment before the Blarrble…

Where do I write the check?

Re: Anthropic’s paper smells like bullshit

#339
post #251

Earlier quoted context omitted.

A broken analog clock will be accurate twice a day despite being of zero use. If someone were to attempt to sell the broken clock as useful because it "accurately returns the time at least twice every day", would Ultimately be causing harm to the consumer.

Depends on what you need the clock for. For example, if it's to serve as an adjustable sign indicating e.g. the closing time of a store, a broken one does the trick just fine :) In other words: Use the right tool for the right job.

You wouldn't market it as a solution to everything (we're still talking about AI here) if it requires you position the hands on the answer you're looking for.

Re: Anthropic’s paper smells like bullshit

#340

Washington has been cold to Anthropic for the wrong bet they made in 2024, hence Anthropic has been desperately screaming all sorts of bullshit to get back attention. Honestly their political homelessness will likely continue for a very long time, pro biz democrats in NY are losing traction; and if newsom wins 2028, they are still at disadvantage with OpenAI who promised to stay California.

Okay but who are you referring to by "pro biz democrats in NY are losing traction"; the NYC mayor election, Hochul retaining governor, Stefanik's ambitions for 2028/32, or else who? It seems the wrong take to read Mamdani's broad-based support as ideological; it was the combination of an affordability crisis plus an awful opponent who should have been replaced (plus a spoiler third candidate). Kind of like a localized 2025 update of "it's the [city] economy, stupid".
Post reply on HN