Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

871–880 of 946 posts

Re: We must pace the frontier

#871

Post by adsharma »

tclancy wrote: post #864
adsharma wrote: post #785

Earlier quoted context omitted.

The real threat is that we uncritically adopt language such as alignment. Implicit in this is the idea that AI is a inscrutable matrix and going to remain that way and we'll need expert interpreters to make sense of it. We need to insist on building tech that's explainable by design.

That last line needs a lot of workshopping. A guillotine with instructions on the bottom of the blade conforms to your request.

If there is guillotine in the weights of the model, it needs to be properly labeled so you can look it up by name using a database index (or a graph-vector index).

It helps both the bad guys and good guys. Like responsible disclosure in cyber security, we need to have a conversation around it.

Re: We must pace the frontier

#872

Post by politician »

esafak wrote: post #401
politician wrote: post #394

Earlier quoted context omitted.

I downvoted you because I think the OPs model is far simpler to implement and suffers from fewer conflicts of interest. If we go down the regulation of alignment route, we'll have to ask experts to create those regulations and monitoring regimes. And which experts will the government ask? Oh, right, the parties that stand to benefit the most from regulation: OpenAI and Anthropic! When "the experts" make certification…

> And which experts will the government ask? Oh, right, the parties that stand to benefit the most from regulation: OpenAI and Anthropic! You've never heard of academia, NGOs, and intergovernmental organizations? Sure, the AI labs will have a say but not all of it. You don't ask the fox to guard the henhouse.

[flagged]

Re: We must pace the frontier

#873

Post by adsharma »

mapontosevenths wrote: post #855
adsharma wrote: post #785

Earlier quoted context omitted.

The real threat is that we uncritically adopt language such as alignment. Implicit in this is the idea that AI is a inscrutable matrix and going to remain that way and we'll need expert interpreters to make sense of it. We need to insist on building tech that's explainable by design.

> We need to insist on building tech that's explainable by design. You realize this means insisting on terrible tech that humans can understand right? It essentially caps human progress at some point about 4 years ago. If you are old and happy with the way things are this might sound like a good idea. It does not to me.

Oh you want tech that helps discover new science instead of parroting existing wisdom?

There is little evidence that the RSI we are discussing is capable of inventing the theory of relativity (or the more advanced equivalent). All we have seen is pattern matching in a much larger space than humans can, with some human provided verification tech.

I would argue that human-AI collaboration with explainable tech has a better chance. Continuous learning can be done in a way that doesn't violate IP or privacy.

Re: We must pace the frontier

#875

Post by amazingman »

academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

This smacks of "all information should be free" dogma. I for one prefer not to be ground into dust in service of producing more paperclips.

Re: We must pace the frontier

#876

Post by abustamam »

epihelix wrote: post #691
cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

> At what point do we stop engaging with Anthropic’s leadership in good faith About two years ago? I would also note that Dario's post appears to be LLM written. Maybe... maybe ... he's read so much Claudeish that it's all he can speak now himself. But I wonder if he's becoming a bit of a meat proxy. (It's funny, I thought "pace the frontier" was going to mean something similar to "patrolling the frontier". But no, i…

> But I wonder if he's becoming a bit of a meat proxy.

A joke I recently encountered:

---

A CEO proudly announced he'd bought an AI system designed to identify the company's most replaceable employee.

It spent the night analyzing five years of emails, meetings, salaries, performance reviews, and productivity data.

The next morning, it came back with one name:

The CEO.

The IT guy was fired for installing defective software.

Re: We must pace the frontier

#877

Post by amazingman »

michaellee8 wrote: post #465
cardamomo wrote: post #450

Earlier quoted context omitted.

These do not seem to be mutually exclusive. Should we not do both?

Do you really think we can really get every country to truly pace the frontier? Pretty sure China won't give a f until they catch up Anthropic and OpenAI. It is an arm race. We had nukes for like 70 years and still haven't figured out how to make every single country follow those nuclear treaties, with an increasingly non-interventionist US I don't think we can get every single country to the table and agree to a pau…

We don't need every country to pace the frontier, just a certain few. And yes, I believe it's possible and the prior art is nuclear non-proliferation. Non-proliferation wasn't perfect, of course, but it was good enough (so far) to pull back from the brink of extinction.

Re: We must pace the frontier

#878

Post by spopejoy »

InsanityCheck wrote: post #222
anon84873628 wrote: post #43

Earlier quoted context omitted.

Yeah, I don't get it. Are they imagining this happening just with the open weights models running on however many GPUs the bad actors can cobble together? For now, all the scary hacking things still require an API key to one of the LLM providers. Surely they should take some responsibility for how to turn off the tap.

Currently Qwen3.8 27B is roughly on Opus 4.6 level. In at most a year given the current pace, you could probably run such hacking bot nets out of a reasonably small local server, bootstrapping by hacking or acquiring login credentials for more compute.

And GLM 5.3, and deepseek 4.1 ... Hyperscaler fanbois have their heads in the sand

Re: We must pace the frontier

#879

Post by itsmek »

lelanthran wrote: post #639
villish wrote: post #414

Earlier quoted context omitted.

> Qwen 3.8 and GLM 5.3 Flash were not just cheaper, the results were significantly higher quality How can they be higher quality than models they were distilled from?

> How can they be higher quality than models they were distilled from? The word "distillation" is not specific to AIs, has been in use for 100s of years, and does not mean the same thing as "dilution". It means "getting a more concentrated form of the original product".

Very shallow answer to a valid question. The vast majority of people know that word btw, that's not some revelation.

Re: We must pace the frontier

#880

Post by taneq »

solenoid0937 wrote: post #757
icantevenhold wrote: post #744

Earlier quoted context omitted.

There are worse fates than extinction, for example living forever, for I have no mouth and I must scream

I dunno, being a Culture Mind sounds pretty damn good. Or even just a death-optional citizen in the Culture.

The Culture isn’t perfect but it seems like the best possible outcome. I think I’d want to be a fast picket and hang out with Special Circumstances.
Post reply on HN