Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The people writing AI alignment policy are not whose work is being replaced

danieltan.weblog.lol

Post reply on HN
Print view View original

61–70 of 73 posts

Re: The people writing AI alignment policy are not whose work is being replaced

#61

Post by sometimelurker »

This might be related to the fact that fully automating AI safety can't be meaningfully done. And a lot of work is put into automating parts of it. Circuit-finding algorithms and SAEs are automated algorithms for interpreting parts of LLMs, and RLAIF (RL with AI feedback) for alignment requires an LLM to judge if another LLM is visibly misaligned. (Claude says 'genuine' a lot due to this. Its harder to look misaligned when you use the word 'genuine' a ton) And there's work on having AIs write cute little stories in which AIs are ethical, and putting those stories in the pretraining corpus.

So there's a ton of work being done already on automating parts of alignment, but since the core premise of alignment being that its hard to encode human values into the reward function, automating it fully would be equivalent to solving it.

Re: The people writing AI alignment policy are not whose work is being replaced

#62

Post by customguy »

kranke155 wrote: post #28
andai wrote: post #26

Well, what are we aligning it with? Civilization is already a misaligned superintelligence (aligned mostly with Moloch, these days). Civilization accelerated by AI just moves in the same direction faster. Moloch on speed. https://www.youtube.com/watch?v=KCSsKV5F4xc Another angle to this is that superintelligence requires supermorality. Super morality looks unpleasant from below. My dad won't let me have more candy, w…

it depends whether you think humanity / civilization are stable systems meant to exist in equilibrium, which they might not be.

"meant to"? What does that mean?

Re: The people writing AI alignment policy are not whose work is being replaced

#63

Post by binarysolo »

throw-the-towel wrote: post #32

“I think, and my thoughts cross the barrier into the synapses of the machine, just as the good doctor intended. But what I cannot shake, and what hints at things to come, is that thoughts cross back. In my dreams, the sensibility of the machine invades the periphery of my consciousness: dark, rigid, cold, alien. Evolution is at work here, but just what is evolving remains to be seen.” — Commissioner Pravin Lal, “Man…

I love SMAC -- I wish they had a real sequel to this complete with storyline. Most Civ clones really don't nail the narrative feel of SMAC as you explore the planet and grow your settlements.

Re: The people writing AI alignment policy are not whose work is being replaced

#64

Post by moffkalast »

saltcured wrote: post #45
bethekidyouwant wrote: post #40

Earlier quoted context omitted.

Ah yes the -2nm node.

I'd like to see a full development of this idea. Something like a CPU that runs at -3 GHz. Or perhaps it generates power while it undoes computation? It's too bad node size is a linear dimension rather than area. If it were area, we could get into its many complex/imaginary properties.

Ha well one that does computation with no power draw is theoretically possible, since processing as such does zero actual "work" physics-wise, it's all just various mechanical losses.

https://en.wikipedia.org/wiki/Reversible_computing

Re: The people writing AI alignment policy are not whose work is being replaced

#65

Post by tim333 »

renjimen wrote: post #52

This is a bit of weird article. On one hand, I understand what they're getting at: AI is a transformative technology, but the people whose lives will be most transformed aren't included in the conversation. On the other hand... of course that's how it is while AI is in the hands of literal profit seeking corporations. That won't change until the labs are nationalised under a government that cares about its citizens'…

I don't think nationalising the AI labs is happening.

I agree the article is a bit odd. Alignment is mostly about making AI helpful and not wanting to kill people unless it's told to (https://www.forbes.com/sites/davidkirichenko/2026/05/12/ukra...).

The article is talking more about people like translators being replaced by AI translation. I don't think any of the labs have a department of making it worse so it can't do people's jobs.

The normal way of dealing with tech doing peoples jobs is to help them get different jobs. I've got a translator friend who did a government paid course to train as a tour guide - that sort of thing.

Re: The people writing AI alignment policy are not whose work is being replaced

#66

Post by redanddead »

Supermancho wrote: post #6
redanddead wrote: post #2

Love the writing style and perspective

I dont appreciate using quotes from individuals to extrapolate to groups and ethos.

Well hold on, the author is using contrasts as a stylistic choice, it's not exactly journalism, there are no journalistic standards to hold him up to, it's a blog post, he can write whatever he wants

I personally wouldn't police his style

Re: The people writing AI alignment policy are not whose work is being replaced

#67

Post by Supermancho »

redanddead wrote: post #66
Supermancho wrote: post #6

Earlier quoted context omitted.

I dont appreciate using quotes from individuals to extrapolate to groups and ethos.

Well hold on, the author is using contrasts as a stylistic choice, it's not exactly journalism, there are no journalistic standards to hold him up to, it's a blog post, he can write whatever he wants I personally wouldn't police his style

> Well hold on,

No thank you. You said you liked the style and I said I don't.

Re: The people writing AI alignment policy are not whose work is being replaced

#68

Post by kranke155 »

customguy wrote: post #62
kranke155 wrote: post #28

Earlier quoted context omitted.

it depends whether you think humanity / civilization are stable systems meant to exist in equilibrium, which they might not be.

"meant to"? What does that mean?

I thought the comment above implied that somehow, humanity and its constructed civilisations ought to be stable really, if it wasn’t for the bad actors in the system making such bad decisions, and how it all ends up throwing the system into periods of chaos that end up threatening the system itself with wars, disasters, etc.

My counter to that is that you might be describing a naturally cyclically unstable system (like a a weather system that naturally produces hurricanes or a tornado) and hoping that it somehow becomes stable again as it was back when winds were 5 mph. My counter to that is that it’s just wishful thinking to believe that any system will forever run at low entropy type conditions. It’s very possible that all systems are inherently just cyclical. I don’t see any evidence yet that we are so different from ie any other dominant branch of life in this planet, by which I mean - the dinosaurs went away.

Also - Our obsession with stability often ends up causing more instability not less, because fudging against the natural trend lines of instability and chaos necessary to the system just lead to a repressed energy exploding in our face - sometimes that’s just what happens.

Re: The people writing AI alignment policy are not whose work is being replaced

#69

Post by renjimen »

tim333 wrote: post #65
renjimen wrote: post #52

This is a bit of weird article. On one hand, I understand what they're getting at: AI is a transformative technology, but the people whose lives will be most transformed aren't included in the conversation. On the other hand... of course that's how it is while AI is in the hands of literal profit seeking corporations. That won't change until the labs are nationalised under a government that cares about its citizens'…

I don't think nationalising the AI labs is happening. I agree the article is a bit odd. Alignment is mostly about making AI helpful and not wanting to kill people unless it's told to ( https://www.forbes.com/sites/davidkirichenko/2026/05/12/ukra... ). The article is talking more about people like translators being replaced by AI translation. I don't think any of the labs have a department of making it worse so it can…

Dario and Demis have called for nationalisation at some point. They know if AI reaches what they believe its potential to be, it needs to be democratically governed. It will upend the markets, but AI already threatens to do that. It feels like wishful thinking given how entrenched we are in neoliberalism, but it makes sense.

In the mean time there are various avenues of regulation and redistribution to lessen the effects, including retraining programs, though that job creation will keep pace with job losses is a big unknown.

Re: The people writing AI alignment policy are not whose work is being replaced

#70

Post by rexpop »

tinfoilhatter wrote: post #53
fluffybucktsnek wrote: post #30

Earlier quoted context omitted.

While their comment is abbrasive, that's a lot of assumptions about OP. Do you really know what their information source is? The core point remains valid, you could've just skipped the play on "alt-man" and you wouldn't have muddied your argument.

They called me a schizo in two separate comments and the only thing I did in my original post, was point out that his name was Altman which could be interpreted as alternate man. I never claimed that it was the etymological root of his name - the folks making the replies got upset for whatever reason and inferred that. Pointing things out that I find interesting to potential readers of my comment, doesn't necessarily…

Going forward, I'll consider using the term "apophenic", instead of "schizoid", but my point remains the same—you are burying the lede, re: very real, dangerous ideologies of Yarvin et al. with unserious associations like "alternate man", and even the bizarre, esoteric epithet "Luciferian."
Post reply on HN