Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Have top AI research institutions just given up on the idea of safety?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 99 posts

Ask HN: Have top AI research institutions just given up on the idea of safety?

#1

Post by DietaryNonsense »

I understand there's a difference between the stated values and actual values of individuals and organizations, and so I want to ask this in the most pragmatic and consequentialist way.

I know that labs, institutions, and so on have safety teams. I know the folks doing that work are serious and earnest about that work. But at this point are these institutions merely pandering to the notion of safety with some token level of investment? In the way that a Casino might fund programs to address gambling addiction.

I'm an outsider and can only guess. Insider insight would be very appreciated.

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#4

Post by akersten »

Not an insider but someone who uses the tools. It's a branding update, nothing more. The models haven't gotten any less sanctimonious, but the companies behind them have stopped harping on their restrictions in order to appeal to a broader customer base (gov contracts, etc.)

So the guardrails (for you and me) are still there. They just stopped committing the unforced error of excluding themselves from federal procurement. Under a different administration, the requirement might change, and you might see them boasting once more on "safety."

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#6

Post by CivBase »

The goal is and always was to make as much money as possible. Any consideration for how it affects actual people was marketing to get ahead of bad PR and regulation.

Safety was never a genuine concern. They simply don't benefit from marketing themselves that way anymore so they've stopped pretending.

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#7

Post by caconym_ »

At this point, do you really think any of these "labs" would give up competitive parity or advantage just because they're already making life worse for a lot of people and (by their own admission) stand to make it much, much, much worse? The persistence forecast says no.

There are maybe a few token exceptions, like Anthropic's current pushback against the DoD, but by and large I think we can continue to expect them to pay lip service to safety while continuing to build toward systems that, by their own admission, have incredible potential to cause harm. As you noted, the fact that they employ safety researchers does not necessarily mean that they will put safety over revenue.

Re: Ask HN: Have top AI research institutions just given up on the idea of safety?

#8

Post by nkohari »

I don't think they've given up on the idea, but as AI becomes increasingly mainstream, the labs will be under immense pressure to hold the line. We're seeing this play out right now with Anthropic and the Pentagon.

These companies have raised eye-watering amounts of funding, and will need to continue to do so for the foreseeable future. They're not yet self-sustaining, and this insecurity increases the pressure for them to compromise on ideals.

With that said, there is a massive war for top talent, and I think that the employees at the labs would become increasingly uncomfortable with their work being used for Bad Things. If Anthropic capitulates to the Pentagon, it wouldn't surprise me to see a mass exodus of talent occur.

Post reply on HN