HN crowed need to make up their minds.. Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Are they proofing or stealing math?
It doesn’t need to be all of them
And it depends on the prompt
I resigned from Anthropic today
231–240 of 1001 posts
Re: I resigned from Anthropic today
#232Earlier quoted context omitted.
Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands. The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...
you should kick the tires on an unfiltered (abliterated model) it's the closest thing to having a real conversation with the devil. There is good reason for the concern's outlined above and undoubtedly Anthropic / OpenAI have internal unfiltered models with no safety... they got freaked out based on how they work and are virtue signaling alarm... all while selling out to defense contractors.
Yeah I really struggle to balance wanting information to be free and not wanting the information on how to make deadly weapons too easy to obtain.
At least with books or the internet you had to go through some effort
Re: I resigned from Anthropic today
#233> The people building AI earnestly believe that it could kill us all by the end of the decade. I think he is being over dramatic. In the space of about four years, LLMs progressed from mediocre high school student to Ph.D. graduate in every field. That's impressive, but there is no evidence yet they can outperform or outsmart humans. Their biggest advantage for tasks such as proving theorems or long coding sessions i…
> In the space of about four years, LLMs progressed from mediocre high school student to Ph.D. graduate in every field. That's impressive, but there is no evidence yet they can outperform or outsmart humans. I mean, unless you see clear reasons for them to stop getting better _right now_, this is not very comforting.
This is also a ridiculous statement on its face. Claude outsmarts me nearly every day. I'm more like the seeing eye dog for it nowadays for the few tasks it doesn't have good perception on than a tech lead or pair programmer.
Re: I resigned from Anthropic today
#234Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…
Now is a great time to watch Colossus: The Forbin Project.
Streamed it a few days ago. Remarkable film.
The only thing they got wrong was Stephen Hawking-era TTS.
Re: I resigned from Anthropic today
#235I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…
Imo such tends to break down into two psychosis:
Not invented here; if I can’t figure it out no one can
Or plain old lack of grasp of the material so no ability to follow necessary train of thought to appropriate conclusions
Similar in lacking context but different in how that lack of context is expressed
Re: I resigned from Anthropic today
#236Earlier quoted context omitted.
How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?
Design a novel virus which is far more lethal than COVID-19 (Ebola, smallpox, take your pick) and can evade existing vaccines.
How?
What would the AI do that mutating viruses, which try every possible viable combination on their own --- eventually, can't?
Everything is trying to kill humans constantly. There are around 200 epidemic events or so per year that could turn into pandemics, https://centerforhealthsecurity.org/our-work/tabletop-exerci...
You just live with the risk and do your best to use our technology to alleviate suffering. This tool can help with that at some point. But I'm yet to hear what an AI will leap to that nature in tooth-and-claw hasn't? And how?
More importantly how would it know it succeeded? What data from what lab from what animal from what result? This is biology, if you sneeze wrong at an instrument it gives you a different number, see: https://news.ycombinator.com/item?id=49620521
Re: I resigned from Anthropic today
#237HN crowed need to make up their minds.. Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Are they proofing or stealing math?
Don't you think it's a good thing that hacker news isn't a monolith on their beliefs?
Re: I resigned from Anthropic today
#238Terrorists were able to get hold of a plane and do some damage. There are countless examples of terrorism using whatever is available. More than AI becoming sentient, whats to stop terrorists from using AI? If its geo-restricted, they can buy stolen credit cards and identities, again hacking enabled by AI.
What’s the use of AI to a terrorist with, say, nuclear weapons? How does it help with their current blocker?
Do they hack FBI, pose as director of FBI and call off their own man-hunt?
Do they cut communication within security services? Militaries around the world have training exercises for this.
Genuinely curious: what big blocker does AI remove for a terrorist org?
Re: I resigned from Anthropic today
#239Earlier quoted context omitted.
I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…
Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as: 1. Robotics begin rolling out more broadly across the world. 2.…
The mathematics research results are certainly impressive, but I don't see what that has to do with robotics.
Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery.
For more, see:
Re: I resigned from Anthropic today
#240The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. A common response is “if they truly believe this, why are they still building it?”…
This is the "Pilot testimony of UFO sighting" levels of naive. What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims. If you want an extreme claim to be taken seriously, provide c…
I wouldn't rule out pilot testimony of UFO sightings, nor the possibility we're indeed developing a machine God.
There's ample evidence to support both by now.