An Alien Mind • Hacker News Board

Re: An Alien Mind

#81

Post by nxnxjzizish » Sun, Sep 06, 2026, 6:42 PM UTC

vatsachak wrote: post #38

tomrod wrote: post #23

Earlier quoted context omitted.

> Astra is a new step in LLMs I think. I'd be interested in hearing more about your evaluation here. It would be nice if LLMs have gotten past the "tell me" hump of recent Claude/OpenAI verbosity.

So before I got a job this fall, I was working on a side project about compiling a particular language to SQL. To test Astra I pulled it off the shelf and asked it to take the grammar and then create a compiler to SQL. I've done this before with GPT-5.5, 5.6-Sol High. The latter was way better but it was still really verbose and information sparse; it used a lot of words to describe each IR expression but didn't real…

[dead]

Re: An Alien Mind

#82

Post by nzmzmxkxixns » Sun, Sep 06, 2026, 6:43 PM UTC

granzymes wrote: post #45

dgellow wrote: post #39

Earlier quoted context omitted.

> I want to believe that humanity is trending towards a good outcome here All the trends so far are towards a nightmarish hyper-capitalist end game. None of the AI leadership is trustworthy, and they openly discuss how they are willing to sacrifice everything humans cherish to have a shot at reaching their envisioned utopia (which would be the most obvious dystopia for anyone else)

I'm not really worried about the labs, it's misaligned governments that keep me up at night. ASI landing during the current administration is not ideal. I also would prefer to avoid needing to indoctrinate myself in Xi Jinping Thought.

ASI is not real and if it was it wouldn't be ideal under any admin. a real ASI will instantly and permanently enslave you, me, the current admin, and everyone else

Re: An Alien Mind

#83

Post by pet_the_bird » Sun, Sep 06, 2026, 6:44 PM UTC

pu_pe wrote: post #37

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chines…

I am personally concerned by what defensive can mean. Alignment of these models is inherently a non-neutral proces, and currently what values are reinforced is decided by a few OpenAI engineers. I feel that any 'defensive model' will further ingrain current values and actively resist the natural progression of our society. This is especially the case for any use of these models for policing or military.

Maybe people are rightfully concerned about the capabilities of the models of other (non/less democratic) states. But if we are concentrating power in the hands of few and at the same time allowing the creation of a weapon that thwarts any offense, how do we ensure the health of our democratic societies?

Re: An Alien Mind

#84

Post by sumitkumar » Sun, Sep 06, 2026, 6:44 PM UTC

sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

"In late 2020s, while the whole world was focussed on AI, automation and resultant economy four major mathematical study branches were discovered by human researchers which took AI a long time to catch up with"

Re: An Alien Mind

#85

Post by hatefulmoron » Sun, Sep 06, 2026, 6:46 PM UTC

brcmthrowaway wrote: post #75

gertlabs wrote: post #72

> Delivering the benefits of scientific progress and economic growth that very intelligent machines enable. I think we're very close to the point where AI-driven breakthroughs outside of pure math and software start to really affect the world. We evaluated GPT-6 Astra in 100 complex, unsaturated multi-agent coding environments, competing and cooperating with other models in open-ended tasks. It's the new frontier mod…

Incredible... software engineers will be joining the breadline soon as managers, executives and PMs take over deliverables. The world will look very different on Jan 1st 2027.

Maybe I just lack imagination, but I don't really know how jobs are supposed to solidify around the role of giving prompts to agents and then looking at the results. I mean, engineers will be in the breadline because their role was simply to prompt the agents.. only to be superseded by managers or executives who no longer manage engineers but themselves prompt the agents? And, for this previously considered obsolete function which they do presumably by copy/pasting requirements from their email inbox, they will be paid by someone who doesn't know that they could just be talking to their own agents?

Sorry if I misunderstand the point, just trying to understand.

Re: An Alien Mind

#86

Post by wartywhoa23 » Sun, Sep 06, 2026, 6:46 PM UTC

dgellow wrote: post #49

pu_pe wrote: post #37

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chines…

Yep. Such a disgusting industry. They created the arm race, push for the arm race, put themselves in position to benefit from the arm race

Just like the rest of military stuff.

Re: An Alien Mind

#87

Post by [deleted] » Sun, Sep 06, 2026, 6:47 PM UTC

mccoyb wrote: post #79

brcmthrowaway wrote: post #75

Earlier quoted context omitted.

Incredible... software engineers will be joining the breadline soon as managers, executives and PMs take over deliverables. The world will look very different on Jan 1st 2027.

I hope this is satire.

[deleted]

Re: An Alien Mind

#88

Post by skybrian » Sun, Sep 06, 2026, 6:52 PM UTC

Planktonne wrote: post #74

skybrian wrote: post #68

Earlier quoted context omitted.

Playing chess, writing code, finding security bugs, and proving mathematical theorems all seem fairly similar to thinking and don't seem much like being tall.

The crucial distinction here is that "seem" does not at all mean the same thing as "is". Thunder seems like the anger of the gods but it isn't . We've had chess playing programs for a long time now and despite it seeming like thinking is required for them, it isn't. The principle you're using here isn't a scientific one but magical [1]. Abandoning empiricism and rationality is not a good way to make progress. [1] htt…

You're insisting on a particular definition of a vague term.

It makes sense now to say that temperature is what a thermometer measures. However, before there were good thermometers, people often thought that heat and cold were different things. The meanings of the words we use were influenced by scientific progress.

For thinking, we don't have a good thermometer. There are IQ tests, but they aren't aren't necessarily all that useful for comparing what people do to what machines do. And that's why there are a zillion AI benchmarks - none are entirely satisfactory.

So what does "smart" mean to you? How do you define it in practical sense? What definition should scientists settle on?

Without a proper definition, how do you tell the difference between "seems smart" and "is smart?"

Re: An Alien Mind

#89

Post by RALaBarge » Sun, Sep 06, 2026, 6:53 PM UTC

hatefulmoron wrote: post #85

brcmthrowaway wrote: post #75

Earlier quoted context omitted.

Incredible... software engineers will be joining the breadline soon as managers, executives and PMs take over deliverables. The world will look very different on Jan 1st 2027.

Maybe I just lack imagination, but I don't really know how jobs are supposed to solidify around the role of giving prompts to agents and then looking at the results. I mean, engineers will be in the breadline because their role was simply to prompt the agents.. only to be superseded by managers or executives who no longer manage engineers but themselves prompt the agents? And, for this previously considered obsolete…

Regardless of the imagination quandary, this second, RIGHT NOW is the worst these systems will ever be. They are only going to get better.

Re: An Alien Mind

#90

Post by gafferongames » Sun, Sep 06, 2026, 6:55 PM UTC

Planktonne wrote: post #74

skybrian wrote: post #68

Earlier quoted context omitted.

Playing chess, writing code, finding security bugs, and proving mathematical theorems all seem fairly similar to thinking and don't seem much like being tall.

The crucial distinction here is that "seem" does not at all mean the same thing as "is". Thunder seems like the anger of the gods but it isn't . We've had chess playing programs for a long time now and despite it seeming like thinking is required for them, it isn't. The principle you're using here isn't a scientific one but magical [1]. Abandoning empiricism and rationality is not a good way to make progress. [1] htt…

First, define “thinking”