Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

681–690 of 716 posts

Re: Anthropic drops flagship safety pledge

#681

Post by coldtea »

sarchertech wrote: post #656
coldtea wrote: post #536

Earlier quoted context omitted.

> But other missions would require rare high end high performance high salary specialists who are in demand - and thus expensive. You can't rely on being able to source enough altruists that will put up with being paid half their market worth for the sake of the mission.' That's exactly what a non-profit should be able to rely on. And not just "half their market worth", but even many times less. Else we can just say…

This would shutdown about half the hospitals in the US.

A, US healthcare, that paragon of value-for-money and not-for-profitness...

Re: Anthropic drops flagship safety pledge

#682

Post by toss1 »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

Excellent news. I was seriously worried they would cave when I saw the earlier news they'd dropped their core safety pledge [0].

It is entirely reasonable to not provide tools to break the law by doing mass surveillance on civilian citizens and to insist the tool not be used automatically to kill a human without a human in the loop. Those are unreasonable demands by an unreasonable regime.

[0] https://news.ycombinator.com/item?id=47145963

Re: Anthropic drops flagship safety pledge

#683

Post by sarchertech »

coldtea wrote: post #681
sarchertech wrote: post #656

Earlier quoted context omitted.

This would shutdown about half the hospitals in the US.

A, US healthcare, that paragon of value-for-money and not-for-profitness...

Yeah I’m sure the fix for that is to shutdown or transition all of the remaining non-profit hospitals to a for profit model.

Re: Anthropic drops flagship safety pledge

#684

Post by afavour »

ACCount37 wrote: post #657
afavour wrote: post #640

Earlier quoted context omitted.

I think you're falling victim to survivorship bias there, or something like it. In 1940 I might have said "fusion power is possible" based entirely on what advanced psychics knowledge I had. And I would have been correct, according to the laws of physics it is possible. We still don't have it though. When watching Neil Armstrong walk on the moon I might have said "moon colonies are possible", and I'd have been right…

Those two things are prevented by economics more than physics. For AI in particular, the economics currently favor ongoing capability R&D - and even if they didn't favor AI R&D directly (i.e. if ChatGPT and Stable Diffusion never happened), they would still favor making the computational inputs of AI R&D cheaper over time. Building advanced AIs is becoming easier and cheaper. It's just that the bar of "good enough" h…

> But the path there is getting shorter over time, clearly.

Why do you assume there is no hard limit we’ll hit with the current tech that prevents us from reaching AGI?

Re: Anthropic drops flagship safety pledge

#686

Post by moomoo11 »

slfnflctd wrote: post #546
moomoo11 wrote: post #522

Earlier quoted context omitted.

He’s an e/acc guy. That should tell you everything. And maybe the incredibly awkward behavior and demeanor.

"Y'know, like, the thing is, like, y'know, here's the thing..." I totally feel for people with speech pathologies or anxiety that makes it harder for them to communicate verbally, but how is this guy the public face of the company and doing all these interviews by himself? With as much as is at stake, I find it baffling.

tin foil hat on

I wouldn’t be surprised if the e/acc freaks have some secret society or cabal lol

Re: Anthropic drops flagship safety pledge

#687

Post by keeda »

shimman wrote: post #636
keeda wrote: post #623

Earlier quoted context omitted.

> It's well established that AI just turns devs into AI babysitters that are 10% more productive and produce 200% the bugs, and in the long-term don't understand what they built. It's not well established at all. In fact, there is increasing evidence to the contrary if you look outside the HN echo chamber. The nuanced take is that AI in coding is an amplifier of your engineering culture: teams with strong software di…

If these tools were so great they wouldn't be struggling so hard to sell them. Great sign that the company has to mandate a "productivity" tool that the workers hate. Hence why all these LLM companies love government contracts, they can't sell to consumers so they'll just steal from tax payers instead.

Cursor and Claude Code are amongst the fastest selling products in history.

Cursor: 1 Billion ARR in 24 months -- https://andrew.ooo/posts/cursor-fastest-growing-saas-1b-arr/

Claude Code: 2.5 Billion ARR in 10 months -- https://www.anthropic.com/news/anthropic-raises-30-billion-s...

Re: Anthropic drops flagship safety pledge

#688

Post by votepaunchy »

isodev wrote: post #182
latexr wrote: post #151

> “We felt that it wouldn't actually help anyone for us to stop training AI models,” How magnanimous! They are only thinking of others, you see. They are rejecting their safety pledge for you . > “We didn't really feel, with the rapid advance of AI, that it made sense for us to make unilateral commitments … if competitors are blazing ahead.” Oops, said the quiet part out loud that it’s all about money. “I mean, if al…

Indeed, Anthropic can’t afford to be the ones that impose any kind of sense in the market - that’s supposed to be the job of the government by creating policy, regulations and installing watchdogs to monitor things. But lucky for the AI companies, most of them are based in place that only has a government on paper and everyone forgot where that paper is.

> that’s supposed to be the job of the government by creating policy, regulations and installing watchdogs to monitor things

But that government cannot trust the other government on the other side of the world to implement the same restrictions, so we find ourselves in this Nash equilibrium.

Re: Anthropic drops flagship safety pledge

#689

Post by keeda »

overgard wrote: post #668
keeda wrote: post #612

Earlier quoted context omitted.

The problem is, the low hanging fruit, the stuff it's good at, is 90% of all software. Maybe more. And it's getting better at the other 10% too. Two years ago ChatGPT struggled to help me with race conditions in a C++ LD_PRELOAD library. It was a side project so I dropped it. Last week Codex churned away for 10 minutes and gave me a working version with tests.

I think that typescript is a language uniquely suited to LLMs though: - It's garbage collected, so variable lifetimes don't need to be traced - It's structurally typed, so LLMs can get away with duplicating types as long as the shape fits. - The type system has an escape hatch (any or unknown) - It produces nice stack traces - The industry has more or less settled styling issues (ie, most typescript looks pretty unif…

I wouldn't say TypeScript is 90% of all software exactly, but tons of apps on all kinds of technologies like Python / Django, Ruby on Rails, PHP, Wordpress, "enterprise" Java and the like, primarily doing CRUD and data plumbing especially for niche applications and internal LoB sites that we never see on the open Internet.

I agree C++ is harder, and I still occassionally find a missing free(), but Codex did crack my problem... including fixing a segfault! I had a bunch of strategically placed printfs gated behind an environment variable, it found those, added its own, set the environment variable, and examined the outputs to debug the issue.

I cannot emphasize how mindblowing this is, because years back I had spent an hour+ doing the same thing unsuccessfully before being pulled away.

Re: Anthropic drops flagship safety pledge

#690

Post by keeda »

bdangubic wrote: post #635
keeda wrote: post #616

Earlier quoted context omitted.

If you mean "us" on this forum, I would believe that. I would bet the number of engineers working on stuff "outside the distribution" is overrepresented here. If you mean "us" as in all software engineers, not at all. The challenge we're facing is exactly that, reskilling the 90% of engineers who have been working on CRUD apps to the 10% that is outside the distribution.

> 90% of engineers who have been working on CRUD apps I am a 30-year "veteran" in the industry and in my opinion this cannot be further from the truth but it is often quotes (even before AI). CRUD apps have been a solved problem for quite some time now and while there are still companies who may allow someone to "coast" doing CRUD stuff they are hard to find these days. There is almost always more to it than building…

I agree, I'd say we're talking about the same thing, just in different terms. When I said CRUD apps, it was a crude stand-in for what you call the "closing" work. Over-simplifying, but it's unglamorous, not too complicated, somewhat mechanical, mostly a translation into working code from high-level designs that come down from the "innovators."

But I am concerned precisely because AI is usurping that closing work, which accounts for the bulk of the team. Realistically the innovators will be the only people required. But the innovators are able to do the hard stuff by learning through a lot of hands-on experience and painful lessons, which they typically get by spending a lot of time in the trenches as closers.

And we're only talking about coding here, but this pattern repeats ALL over knowledge work: product, legal, consultancy, finance, accounting, adminstration...

So now the problem is two-fold: how do we get the closers to upskill to innovators a) without the hands-on experience b) faster than AI can replace them?

I can see where Dario is coming from.

Post reply on HN