The language of these LLM posts makes me think they're considering the option to be a defense subsidiary. It doesn't seem like they're actually constraining or limiting the models in any way. They don't seem to understand how the model actually works. Poke the beast and see what happens. Also the use of passive language as-in AI is becoming more and more of a threat as opposed to the reality where they're making the model more and more aggressive and useful for military is very hypocritical.
An Alien Mind
381–390 of 490 posts
Re: An Alien Mind
#382> a lot of the model’s capability comes from a verbalized reasoning process I call bullsh*t. There is no verbalisation of any reasoning process. Verbalisation, e.g. putting reasoning etc. into words requires some reasoning to exist. These LLMs have nothing but the words. That's why they are language models not e.g. reason models.
And I have nothing but neurons firing, I'm just a neuron meat-sack, no reasoning going on. Yes, their reasoning is different from ours, and both considerably weaker in lots of ways, and stronger in other ways. Playing with a coding agent now, they do think through problems and make sensible decisions. It's a mess to read, of correcting itself and second guessing, and verbiage. But... It works decently well these days…
> There is also reasoning happening internally - e.g. look at the steps in the J-Space paper from earlier in the year
> That's the "reasoning process" that leads to the words,
That's not reasoning. Its just the words at intermediate LLM layers.
The paper's very title is clickbait. Its "global workspace" reasoning is delusional fantasy.
Re: An Alien Mind
#383Earlier quoted context omitted.
The point of the comment you're responding to is that it is at this point hardly an arms race: The frontier of ai development is happening completely inside 2 American companies, the most significant results outside America mostly involve (impressive!) distillation, which means their progress is conditional on progress of the big closed source models. So if the race here is between 2 American companies, this is obvio…
Development and improvement of nuclear weapons was an entirely American project until the technology was exfiltrated and then it became an instant arms race. That cat is already out of the bag with LLMs. Distillation is just the fastest way to keep pace but that in no way prevents other countries and actors from doing it the hard way. An arms race doesn’t imply one side winning, it’s not a race with an end goal, it’s…
>an entirely American project until the technology was exfiltrated
No it wasn't. https://en.wikipedia.org/wiki/Tube_Alloys
Post WW2 the USA (for a bunch of quite interesting reasons) excluded the British. But, of course, the British had acquired a lot of knowledge from the program and were able to develop their own bomb.
Re: An Alien Mind
#384Earlier quoted context omitted.
My opinion is that serious repercussions for lying would fix the world overnight. Everything bad stems from lying, it is the root of all evil. It creates distrust, fear, paranoia. It re-inforces bad ideas and groupthink. It creates delusions and delusional people. It makes weaker people, too. People don't get an opportunity to learn to deal with criticism. People don't get an accurate reflection of how others see the…
"I can't really think of a single example where lying is actually a good thing" Lying to save a life or rape Lying to preserve a childhood myth like Santa Claus. Lying to avoid hurting someones feeling when knowing the truth could only bring pain Lying to create shared cultural myths to strength society. Lying isn't the harm you make it out to be.
> Lying to preserve a childhood myth like Santa Claus.
FWIW from the very beginning, I told my son that Santa Claus, the Tooth Fairy, and the Easter Bunny were just a game we all played, and it's seemed just as fun to me. I don't think being lied to about Santa Claus hurt me, but still I'm not in favor of it.
I'd lie to a Nazi without a second thought though.
Re: An Alien Mind
#385Earlier quoted context omitted.
Maybe, maybe not. It's not unreasonable that these systems cap out at some point, or perhaps fizzle away entirely. The businesses that create these systems are not profitable and run at a massive historical and go-forward loss. New data centers required to operate these systems are facing increasing pushback at local levels. New construction is not guaranteed. Energy and power grid constraints exist as well. Governme…
> It's not unreasonable that these systems cap out at some point, or perhaps fizzle away entirely. Yeah, like computers and mobile phones did. Things that have utility, even if not immediate or initially obvious, don't fizzle out.
That's survivor bias to the extreme.
Re: An Alien Mind
#386Earlier quoted context omitted.
Alien mind is in my view the best mental model - LLMs are not merely stochastic parrots, are not like humans, are not like animals. They're maybe nearest to Cthulhu, but that's fictional. In terms of existing mental models "alien minds" feels the best can do. I agree that "teach how to love" is off and perhaps excessively anthropomorphic. But we don't have good words or concepts for what we really need to do - hence…
pretending that it's something like a mind at all is what's misleading. it's more like a cast of a bunch of overlapping/entangled thinkprints, and pushing activation through it produces new prints. it can already "love" because that behaviors in the data along with hate and everything else. acting like the behavior is alien or unexplained is the dishonest part. they know exactly where the behavior comes from - why el…
the behaviour is alien because it is different from human, they don't know in detail how those trillions thinkprints work together to produce output
openai "knew" about scaling law for a decade now and still can't fully explain it, why you think they are dishonest here?
Re: An Alien Mind
#387Earlier quoted context omitted.
The results we've been seeing internally on our physics and circuit design environments are expert-level and beyond-expert-level results from models that Astra completely outclasses across the board on our evaluation suite (Fable 5+/Opus 5/Grok 4.6 were all worthy of being called AGI in my opinion). That's hard tech that will translate to real product innovation. But you don't need any kind of insider information to…
I mean honestly, that's the problem. I'm actually not seeing the world changing. What specific advances in robotics, unsolved maths, and software have LLMs provided? What is the finished result that affects everyday life? In all categories, it's been hype with little actual real results. The robots are still doing the things they did before 2022. The maths are a handful of fairly insignificant proofs that have no sig…
The number of bug fixes to important programs has skyrocketed. Look at what Google are saying about how many Chrome security bugs they've been fixing lately. Other big software firms have been doing the same thing - AI has been finding and fixing a ton of bugs. I know of one big program where thousands and thousands of security bugs are being found and fixed.
It may not feel like this to you because a lot of the dollars right now are going into security bugs which you can't perceive. But it's definitely happening.
At the company I own I've got AI employees autonomously triaging backlogs and fixing long tail bugs. The software is definitely getting better, although by definition long tail bugs aren't ones you are likely to encounter. The subjective "feel" of how robust the software is won't change quickly.
New tech is always applied in apparently boring ways because we are imagination constrained and people harvest the low hanging fruits first. Remember claims there was worldwide demand for only about four computers? When Gates said he wanted a computer on every desk and in every home people laughed at him. What would people do with all those computers, they asked. But he was right about where the world was heading.
Re: An Alien Mind
#388Earlier quoted context omitted.
Software engineers have been trying to put themselves out of a job ever since the profession first came into being. Whenever an engineer gets a task their very first thought is "how can I automate this?" Going by mainstream consensus we should all have been unemployed by now. Yet every new leap into automation opens up a whole new tree of possibilities with an order of magnutude more jobs. So no, the profession will…
This time it is different. Because in the past, setting up that automation needed a, drumroll, qualified engineer. Now you can get a 14 year old halfway around the world who knows how to prompt alright enough to ship. There is no more moat.
"Low code" has been a dream of the industry for longer than I've been alive. There are reasons SQL and COBOL look superficially like English even when it's inefficient to do so. There are reasons Excel is the most popular programming language. Programmers have always been trying to enable non-programmers to write software.
Re: An Alien Mind
#389Earlier quoted context omitted.
Right, but currently the USA is still pretty far ahead. If you're in a race and the person who's in the lead by a large margin says "this is getting out of hand, how about we take a break?" isn't it obviously in the interest of the disadvantaged party to agree? And if one of the parties can put a stop to the race like that at any time, is it really an arms race? The current balance between the US and China when it co…
> If you're in a race and the person who's in the lead by a large margin says "this is getting out of hand, how about we take a break?" isn't it obviously in the interest of the disadvantaged party to agree? Only if you trust the other party (which very clearly doesn't hold in this case) or monitoring the break can be done independently and reliably (it can't), and you expect the gap to narrow rather than widen durin…
> For nuclear weapons it has become quite clear that even for small players, being in the race and having at least a few nukes is far more rational than having none. Ukraine found out the hard way that giving them up in exchange for promises of good behavior just sets you up for getting stabbed in the back.
I've heard a different perspective on this: Nuclear weapons need maintaining, and even maintaining them was probably beyond Ukraine's capability. Qaddafi gave up nuclear weapons after determining that they were just too expensive to be worth it; Iran damaged its economy to the tune of trillions of dollars trying to get nuclear weapons and so far failed; NK managed to get them but impoverished their nation to do it.
Re: An Alien Mind
#390Earlier quoted context omitted.
Actually "France, the UK and The United States have all declared that they would never allow AI to control decision-making on the use of nuclear weapons." [0] I also expect AIs never be in control of nuclear weapons. AIs can never fully be trusted. On a lighter note, Wargames gave us an insight of a computer having access to thermonuclear missiles. [0] https://www.icanw.org/are_there_specific_international_agree...
All official statements are literal and fragile. Basically, this means that France, the UK, and the US will use AI in the deployment of conventional weapons.
Why would AI ever be useful in nuclear weapons decisions? There is no need to be faster or more efficient at making that decision since if we need to make the decision all is already lost.