Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

twitter.com

Post reply on HN
Print view View original

91–100 of 264 posts

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#91

Post by edot »

drcode wrote: post #46
rocmcd wrote: post #38

It's hard to read Gwern's accompanying article without seeing this for what it is, which is a kind of mania. I'm sure he means well and is genuine in his aspirations, but what's outlined for GA is framing LLM's as quasi-gods, which they absolutely are not. I wish him the best, and look forward to being proven wrong.

An LLM just solved 10 top tier math problems this week. It seems very likely that in 6 months they will solve 100 top tier math problems, maybe 1 millenium math problem, maybe 10 top tier physics problems, etc. It's only going to get crazier.

Math is verifiable. LLMs will continue to make strides in such search spaces where all they need to do is try->verify->repeat. You can't train an LLM on how to end the war in Iran or Ukraine, nor how to solve hunger, nor climate change, etc. Real issues.

And LLMs still suck at art and writing.

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#92

Post by thin_carapace »

honestly i dont blame smart people for cashing in, because there is no inherent reward for behaving goodly/smartly. in this instance gwern is a particularly smart individual and has contributed a lot so i especially shouldnt judge him. straight up ripping off a black mirror episode (s02e07) is a bit heinous for my liking though. digital twins are inevitable but that doesnt make this any less wrong.

"AI poses threats of its own ... a nuclear bomb can’t think for itself and make choices, but AIs do, and current LLMs have proven themselves untrustworthy as they regularly reward-hack and betray their users ... how can you trust them to handle ecosystems of combined-arms for an AI-centric military during a war? But widespread deployment of GAs offer some hope of meaningful supervision, as long as the GAs are sample-efficient enough ... or there is some chance of the principals being able to “catch up” later and correct any errors before events have spun too far out of control."

gwern openly admits ai behaves erratically, then hand waves the issue away 'as long as we double check things [sic]'. gwern is smarter than this, ergo i feel like i am being bullshitted.

is anything in this world worth knowing that a version of you will suffer for eternity? is anything in this world worth copying yourself such that you may be enslaved by anyone with terminal access?

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#93

Post by kashyapc »

> What would it take for LLMs to make me 100× more productive? Without this, I am doomed to irrelevance.

Are you, though? You will only be "doomed" if your place your value system squarely on "productivity". Then what is to differentiate you from a machine?

    * * *
Edit: I'm not sure how I can feel confident about a proposal that puts so much value on "productivity". How can you reconcile this with "self-actualization"? (Don't get me wrong, I like my LLM-based productivity gains as the next person, but I care more about wisdom than becoming "100x more productive".)

Edit 2: "the goal of GA is to preserve individual human cognitive liberty and flourishing" — so the proposal is to do that by overlaying a software bot that continuously mimics your "self"?

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#95

Post by jephs »

Only, if he had instead fallen in love with the version of this idea in which a community acts as the principal, rather than an individual.

(I would like to be known to the agent serving my family, that serving my friends, my team at work, the PTA at my kids' school.)

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#96

Post by MichaelDickens »

edot wrote: post #91
drcode wrote: post #46

Earlier quoted context omitted.

An LLM just solved 10 top tier math problems this week. It seems very likely that in 6 months they will solve 100 top tier math problems, maybe 1 millenium math problem, maybe 10 top tier physics problems, etc. It's only going to get crazier.

Math is verifiable. LLMs will continue to make strides in such search spaces where all they need to do is try->verify->repeat. You can't train an LLM on how to end the war in Iran or Ukraine, nor how to solve hunger, nor climate change, etc. Real issues. And LLMs still suck at art and writing.

Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#97

Post by artyom »

dataviz1000 wrote: post #74
artyom wrote: post #41

Earlier quoted context omitted.

Leaving AI completely aside, it still amazes me that people finds "novel" the idea of removing highly-paid white collar intellectual workers (software developers or otherwise) completely out of the loop. No-code platforms date back to the 80's. Getting rid of engineers in general is even older [*]. Even relational databases and SQL were initially promoted as "ways to get rid of those expensive programmers to access y…

How many people here started writing code because of no code Drupal?

I'm quite a bit older than that, so no. But I 100% get your point.

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#98

Post by wolvoleo »

symfoniq wrote: post #52
sillysaurusx wrote: post #18

(I'm not a part of GA.) I've known gwern for the better part of a decade. Working with him has been great. We've done quite a few projects together, including being the first ones to demonstrate that GPT-2 could play chess (or rather, can be used for actual useful work instead of just being an autocomplete). He's a great person. I've wanted to do a writeup on it for some time, but what surprised me the most is his hu…

I can nod along when I hear and read people talking about the dangers of these technologies and the endgame of their creators; and then suddenly very weird when everyone except me is nodding along that the answer must be similar (but different) technology! It's as if technologists are stuck in a room of mirrors, unable to imagine a world in which "solutions" don't ultimately just continue to feed technology's increas…

Technologists don't drive decisions though. The shareholders do.

Edit: I mean technically shareholders don't decide any operational choices but their shallow interests are what everything is decided around.

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#99

Post by bigyabai »

MichaelDickens wrote: post #96
edot wrote: post #91

Earlier quoted context omitted.

Math is verifiable. LLMs will continue to make strides in such search spaces where all they need to do is try->verify->repeat. You can't train an LLM on how to end the war in Iran or Ukraine, nor how to solve hunger, nor climate change, etc. Real issues. And LLMs still suck at art and writing.

Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."

> they don't understand the notion of truth so they can't do verifiable work like math proofs.

Nobody that understands automated proof checking was claiming that.

Re: I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel

#100

Post by jdub »

MichaelDickens wrote: post #96
edot wrote: post #91

Earlier quoted context omitted.

Math is verifiable. LLMs will continue to make strides in such search spaces where all they need to do is try->verify->repeat. You can't train an LLM on how to end the war in Iran or Ukraine, nor how to solve hunger, nor climate change, etc. Real issues. And LLMs still suck at art and writing.

Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."

You understand that those are different kinds of "truth", right?
Post reply on HN