U.S. Department of Energy Launches the Genesis Open Models Initiative

> I think the distinction between political appointee and scientist/researcher stands.

I agree with this.

I'm a civil servant and I know (personally; my work is nowhere near the labs) a number of people who work or have worked in that weird contracting DOE/DOD structure that includes the labs.

The difference I've seen is far more from the nature of work rather than the employment details.

Since you asked my opinion, I will say that it is nuanced and have thought a lot about these topics.

People need to have the power to influence their government and the government largely needs to operate in the interest of the people. It doesn't have to do what the people want, but I think governance needs to understand what the people want and interpret how best to address it. Kind of like how developers think of what users want.

The right to bear arms is critical. That is a form of power and self defense which can save your life, your neighbors life, or millions of lives from some kind of tyranny. The governmental structure of the US is so good, there is no comparison anywhere else in the world and we're not even remotely close to some sort of totalitarianism like China has.

At the same time, we do have surveillance capitalism accelerating and privacy is a form of power too. Even though I dislike it, in the current moment we're in I feel like it is unavoidable. When the threats against the state increase (whether the power of the people, or otherwise), the defenses increase too. I think most people who gravitated to HN understand the risk of threats leading to safety solutions that kill freedom and privacy a little more each time.

Iran built out a huge camera surveillance network to track their people, then Israel hacked it and used it to track them back. Surveillance capitalism is a double edged sword. You catch some types of crime, terrorism, whatever. That is great. At the same time, it opens up a huge vulnerability allowing the destruction of your whole state.

So then what about open weight models? People really do not understand the enormous scale of the threat. We do not let regular civilians run around with nuclear bombs or develop biological weapons or any number of things. It's not that the people want them and the government doesn't let us, it's that basically universally people do not want any other people to have that power either.

AI is like... mass manufacturing someone smarter than the smartest human that ever lived and allowing an infantile 16 year old with raging hormones to send a swarm of them off to cause chaos like some kind of necromancer. There are things these models know how to do that the citizens of any given country should want to largely be kept in responsible hands.

This is actually a double sided issue too, because you don't simply give everyone infinite power so they can defend against tyranny. If you've ever seen ideological activists, then you know people can be tyrannical too. Silencing you, cancelling you, ending your career, livelihood, disturbing the peace and so on. The government isn't the only threat. If the potential power of AI causes too much chaos, then the government has little choice but to crack down on society in more ways and AI can be the very thing that caused what you wanted to avoid.

I think open weight models are great, up to a point. People should own a gun for self defense and a car to get where they need to go. It's great to be able to ask private health questions to an open weight model in an era where everything you tell your doctors goes into some online database to be stolen by China. AI can help people be better at the essential things and fill in gaps where they're lacking. There are measurable points though, where models are just force multipliers beyond any reasonable norm for problematic types of tasks.

We don't need nukes. I don't need a carrier group and spy satellites. The people who control those swore to defend the constitution, which defends the people. Some people disagree that AI can ever be good enough that these scale of threats are even comparable. It's ok, they're just actually wrong in a fully logical, serious and non-rhetorical sense. The problem is that with open weight models you only have to be wrong once. That floppy someone copied in the 1990s is still floating around somewhere. In that sense it may be inevitable, but if we allow ourselves a head start then perhaps we can manage it better in the future when we're more ready.

There are a couple current mitigating factors, for now. One is that a lot of safety training and filtering is occurring, so even if some companies distill from the big companies they are getting filtered results. Another is that any model big enough to be dangerous is hard enough to run that the threat can't easily scale up in a residential or private company scenario.

None of this is going to help us from countries like China, Russia, Iran, North Korea and so on using powerful models to crack down on their people while accelerating chaos around the world if they choose. So long as countries have nukes and can maintain ways of accurately measuring interference, there will be red lines we tell each other not to cross.

Do you think the current (or future) US governments will apply the DMCA to these trillion dollar companies with the same gusto that they use it to, for example, push Aaron Schwartz into killing himself, or threatening teenagers with lifelong federal prison sentences?

dystopian sci fi movie narrator voice in 2020, she was branded an Essential Worker shows her in chains at a grocery store checkout counter. In 2026, she was... dun dun dun, EXPORT CONTROLLED shows her getting those words stamped on her neck Coming this summer, American Worker, a new sci fi horror movie

It is not slower than DS4. That's crazy talk. It's much smaller, with smaller active parameters, and thus runs faster.