121–130 of 161 posts
I think when chinese models figure out how to partner with SOC 2 providers in the USA in a way that offers a $20-$100-$200 subscription is an easy convenient way, anthropic and open ai will be screwed.
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.
The only examples of companies voluntarily putting up warnings I can think of are those obviously tongue-in-cheek warnings "this book/game/series may cause addiction because it's just so good" etc.
Somehow the Chinese phrases 内卷 https://en.wikipedia.org/wiki/Neijuan and 摆烂 ("let it rot" https://zh-yue.wikipedia.org/wiki/%E6%93%BA%E7%88%9B ) are quite appropriately used, even in stylistic tone. Don't know how much Chinese geohot actually speaks but it's a nice touch.
The English article for "let it rot" (or a related phrase) seems to be here: https://en.wikipedia.org/wiki/Tang_ping
Earlier quoted context omitted.
With the reality-bending financial incentives that these people have, I simply do not trust your or anyone else's assessment of the genuineness of their beliefs. And anyway, it's quite ironic that from godless San Francisco of all places, we are supposed to simply accept that the harmful acts of religious extremists are justified because, well, they do believe in their cause.
The point here is not the correctness of their beliefs. It's about what the actual content of their beliefs is. If someone says that Donald Trump is a secret Shia supremacist and using that to explain his actions, pointing out that that belief about Trump's belief is wrong is not a statement that Trump's beliefs are correct, but that your model of Trump's belief is incorrect.
Personally, I do think some of the people in this field have really drunk the Kool-Aid and still believe in the paperclip monster. But for many others, I do not think the past couple of years of plateauing progress has escaped their notice. ESPECIALLY at the leadership level. I think they're larping and they know it. I just think they want the money and the power.
Actually, that's not what usually happens after a failed prediction. The weak-believers leave but the rest end up having their beliefs reinforced by the event (or lack thereof). It's like an exercise in doublethink, and those who pass are now deeper than ever in the cult.
Earlier quoted context omitted.
What's "exponential" about AI development? Model parameter counts? Anthropic doesn't publish those for their own models, last I checked. Datacenter buildouts? Water consumption per request? There just isn't enough evidence that AI smarts is growing all that much, once you account for the scaling of inputs. That's what OP probably means by "nonsensical hype".
What's "exponential" about AI development? The METR task-completion time horizons, for one. https://metr.org/time-horizons/
Earlier quoted context omitted.
What's "exponential" about AI development? The METR task-completion time horizons, for one. https://metr.org/time-horizons/
Lousy benchmark, they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.
Make your own then. It can go on the pile with all the others that keep getting saturated too fast to be useful.
> they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.
What?
They made the benchmark last year, and included a bunch of models going back as far as 2019.
When they first announced it, the top end of their tests were things AI could not actually automate, and even now only does erratically. Examples of the tasks SOTA models are now saturating (at the 50% success level, not at 80%) include:
"Prune attention heads of a BERT language model while minimizing accuracy loss on text classification tasks."
"Implement a Python library for the ACE-OAuth standard that can generate and parse messages in CBOR format and encrypt/decrypt access tokens with COSE according to RFC specifications."
"Debug a PyTorch machine learning library with gradient calculation and memory optimization bugs until all tests pass."
"Finetune a large language model to reduce the accuracy of a truth detection probe while maintaining performance on standard benchmarks."
- https://arxiv.org/html/2503.17354v1
Earlier quoted context omitted.
[dead]
Regardless of whether you believe in doom or not, it's this kind of wild, willful misunderstanding that makes people roll their eyes and dismiss you. Yes, people at Anthropic (and OpenAI, for that matter!) really, genuinely do believe AI poses a potential existential threat to humanity; every single one that I know personally believes that. This has been pointed out hundreds of times, and to simply dismiss it as mark…
Calling them scammers is literally giving them benefit of the doubt and trying to avoid conspiratorial thinking. You frame the belief that "these are normal people with normal motivation" to be paranoid conspiratorial thinking. Meanwhile, you claim they are literal conspiracy and cult and that is somehow less paranoid.
Now, dangerous cults and scammers are severely overlapping categories. Most if not all cults scam people. They lie to get more powerful whenever it suits them. They have true believers (like your friends) fully willing to cause any amount of harm for the visionary in the center. These people may have started as a well meaning do-gooders, but they usually just end up being co-conspirators in crimes.
I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.
Whether or not that level of power exists, that is definitely how they're pitching it.