Live data from Hacker News

An Alien Mind

openai.com

101–110 of 490 posts

Re: An Alien Mind

#101
Am I naive to not understand the "delivering the benefits" part?

Industrial revolution worked that way because it replaced something very finite and unscalable - manual labor. LLMs just make intellectual work faster, so we can do more intellectual work. With labor we somehow decided that NOT doing too much of it is best. Will we decide to reduce intellectual labor because LLM made it more efficient? I doubt that.

On the other side, as I see in software engineering, the same models are available to everyone, some people are better at it and some people are not. "Software developer" is here to stay, we'll just always be better at it than people who are experts in, say, chemistry. Same works for most other fields.

So we'll just end up in the same situation, with same intellectual labor baseline, just more output requirements. Before, you spend 2h per day coding, deliver a software in 1 month, later, you spend the same 2h per day in intense Claude-herding sessions, deliver a software in 1 week. Ok. Next task.

Fundamentally, there's finite number of desirable resources, and if the models are available to everyone, humanity will just continue about the same, bickering here and there, war here and there, politics, homelessness, poverty, - normal human state.

And if the models are only available to elites, even worse.

Re: An Alien Mind

#102

Earlier quoted context omitted.

You're insisting on a particular definition of a vague term. It makes sense now to say that temperature is what a thermometer measures. However, before there were good thermometers, people often thought that heat and cold were different things. The meanings of the words we use were influenced by scientific progress. For thinking, we don't have a good thermometer. There are IQ tests, but they aren't aren't necessarily…

People in the past being wrong doesn't mean you have to repeat the same mistakes, and it definitely doesn't mean you should throw your hands up and declare that all similar things must be identical. Quibbling over commonly-understood definitions is not a strong argument. If you're genuinely struggling to understand that El Ajedrecista [1] did not meet any definition of thought, then the solution is not to demand that…

I have a vague understanding, enough to know that tallness isn't thinking. I think I can usually use the word correctly. (I don't think El Ajedrecista qualifies, but it was starting to play chess, so it's closer to "smart" than a rock is.)

That doesn't mean I know whether "smart" should be applied to what AI's do, and I suspect nobody else knows either. This is the sort of thing philosophers debate about, not common sense.

Turing invented the imitation game because he didn't really know either.

Re: An Alien Mind

#103

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. Yikes! I really wonder about the cognitive dissonance necessary to work at OpenAI these days. They’re in an arms race to build a machine god, knowing full well that it could end humanity.

Money me. Money now. Me a money needing a lot now.

Re: An Alien Mind

#104
post #20
post #16

Earlier quoted context omitted.

[flagged]

1. The grandparent commentator is describing strategic behavior of dangerous technologies. Game theory / mechanism design primitives. 2. If there is competition for resources among autonomous agents, the "strongest" agent wins (conceptually the most adaptive / evolutionarily fit). 3. Computer programs serve up webapps today, but they also run utility companies, dams, nuclear arsenals, factory production floors, autom…

OK so if it's a computer security issue we're worried about, and there is a credible threat, then probably the answer is to build more secure systems? We know how to do it but choose not to because it's very expensive and usually the threat isn't severe enough to warrant it.

If we decide we can't or would rather not build secure computerized systems, then the answer could be don't incorporate computers into those systems. There's no essential reason to have utility companies, dams, nuclear arsenals, factory production floors, mines, cars, planes, ships, etc all be computerized. If it became necessary from a safety standpoint to uncomputerize them we could probably do it quickly (if not necessarily smoothly) with the stroke of a legislator's pen. Some of those things would actually be made better from a functional standpoint in the long run by doing so. Computerization breeds non-essential complexity like nothing else, eliminating it from some systems could be an extremely worthwhile exercise.

The whole "paperclip apocalypse" fantasy rests on some really strange assumptions about how things actually work in the real world. How much friction there is setting up a factory/mine/smelter/whatever, how much manual labor it takes to build it, let alone make it run (even the most computerized ones). It all seems like total bunk to me, but I'm not a philosopher.

To be convinced this outcome is even remotely possible I'd need to see some clear evidence that a computer program was successfully exhibiting agency and successfully using that agency to manipulate large numbers of people into doing its bidding. Mobilizing massive nation-scale manual labor is the only way it could possibly achieve some nefarious ends like "turn everything into paperclips" and I'm sorry but that just seems way too far fetched. Nations full of people can barely ever agree on anything. My money is not on that changing anytime soon.

And none of that pie in the sky shit has anything to do with language models. OpenAI is a company that sells language models. Not clear why they're talking about all this stuff, or why any of us should be either. It's a bunch of low quality fan fiction sci-fi drivel, and I think we can all surely agree language models are not the thing that'll make it real... right? Maybe if they show us some major technical improvements it would be interesting, but right now it all just sounds like more of the same snake oil.

[edit] not sure why my parent comment was flagged? That seems excessive.

Re: An Alien Mind

#105

Earlier quoted context omitted.

Incredible... software engineers will be joining the breadline soon as managers, executives and PMs take over deliverables. The world will look very different on Jan 1st 2027.

Maybe I just lack imagination, but I don't really know how jobs are supposed to solidify around the role of giving prompts to agents and then looking at the results. I mean, engineers will be in the breadline because their role was simply to prompt the agents.. only to be superseded by managers or executives who no longer manage engineers but themselves prompt the agents? And, for this previously considered obsolete…

There are two things SOTA LLMs fundamentally cannot do. They cannot take financial or legal responsibility for mistakes, and they cannot learn new things without forgetting things (except to a limited degree by adding it to their context). This is clear to anyone who has used even the smartest models for tasks requiring domain knowledge outside of math and coding, for which it's not possible to generate an infinite amount of synthetic training data: they still make stupid mistakes, and have limited ability to learn from those mistakes.

Humans also have a limit on the amount of domain knowledge they can acquire, albeit a much larger one. Executives hence cannot just replace all knowledge workers with LLMs, because executives have neither the domain knowledge to prompt and check the LLMs' work nor the bandwidth to keep on top of such a large volume of ongoing work.

Re: An Alien Mind

#106
post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

The museum will have a scrap of paper that will say "Pound pastrami, can kraut, six bagels bring home for Emma".

Re: An Alien Mind

#108
>> We need to find ways to preserve human agency and enshrine an intrinsic value to being human..

Evey politician, salesman and conmen alike, utter some lofty ideals as goals for "We", just to obscure their private goals that go exactly in opposite direction.

Just like how Nations talk about climate change while increasing pet capita energy consumption and waste production.

Re: An Alien Mind

#109
post #53

Sometimes I wonder if the people working at frontier AI labs even talk to other humans anymore. Reading this little essay started out normal, but soon felt like a look into a disturbed and worrying mind, and if you find yourself taking it at face value, I urge you to step away from chat bots and spend some time with friends and family.

Frontier AI labs I don't know, but I know for a fact that the company I work for has been experiencing "its pivotal moment" (with strongly negative connotation), per the sentiments of both its longest-serving employees and the newcomers baffled at the number of idiotic instructions and fines, since the emergence of LLMs the company's founder has been spending entire nights chatting with. People have been fleeing like…

Wait. Fines?
Post reply on HN