Maybe I just lack imagination, but I don't really know how jobs are supposed to solidify around the role of giving prompts to agents and then looking at the results. I mean, engineers will be in the breadline because their role was simply to prompt the agents.. only to be superseded by managers or executives who no longer manage engineers but themselves prompt the agents? And, for this previously considered obsolete…
There are two things SOTA LLMs fundamentally cannot do. They cannot take financial or legal responsibility for mistakes, and they cannot learn new things without forgetting things (except to a limited degree by adding it to their context). This is clear to anyone who has used even the smartest models for tasks requiring domain knowledge outside of math and coding, for which it's not possible to generate an infinite a…
> they cannot learn new things without forgetting things
Where did you get that idea from? Basically last few years was them constantly learning new things while improving their capability on the things they already knew.
openai is deflecting. this blog post of theirs is just another dopamine hit to distract logical minds with 'greater concerns' so they can keep building their machine. it's not enough they are displacing humans from work, consuming increasing amounts of electrical power so humans have to pay more for it, creating disinformation bubbles with avalanches of slop. they dont care about alignment - these words are theater - obfuscation so that the people who can fix these issues are busy thinking about problems that cannot be solved
It's hard to get humans to agree to things that are in their collective interest but many not be in their individual interest. It is at the root of many problems. Look up "collective action problem."
What do you think it might take to precipitate collective action in this case?
An event that suddenly has a large negative impact on a big enough group of people.
Yeah, on the topic of alignment, a few thousand religions and political parties would like to have a word. It's so simple, Churchill, Stalin, Hitler, Roosevelt, do you all agree this AI is correctly aligned? ...Morals are relative to frame of reference. Throwing out the word "alignment" as if its a singular quantity is like trying to get all observers to agree on the speed an object is moving without first agreeing o…
That's the point. The topic of AI alignment cuts through all human values, morals and ethics in all frames of references.
We can expect similar diversity in what alignment means in context of AI.
This is an element of the delusion. The koolaid drinkers will all tell you that we're on the edge of AGI, but if you ask them for simple examples of breakthroughs made by current AI, they can name none. The entire thing absolutely wreaks of mass psychosis.
Would the recent AI-aided math proofs count as breakthroughs?
Prepare for moving goalposts. In 50 years people will still doubt that AI can create anything novel and worthwhile, while they are going to rely mostly on the things that didn't exist before AI, in nutrition, medicine, technology, communication, entertainment. They will see them as, normal, common and simple extensions of the previous developments, pushed mindlessly a bit forward by stochastic parrots.
The results we've been seeing internally on our physics and circuit design environments are expert-level and beyond-expert-level results from models that Astra completely outclasses across the board on our evaluation suite (Fable 5+/Opus 5/Grok 4.6 were all worthy of being called AGI in my opinion). That's hard tech that will translate to real product innovation. But you don't need any kind of insider information to…
I mean honestly, that's the problem. I'm actually not seeing the world changing. What specific advances in robotics, unsolved maths, and software have LLMs provided? What is the finished result that affects everyday life? In all categories, it's been hype with little actual real results. The robots are still doing the things they did before 2022. The maths are a handful of fairly insignificant proofs that have no sig…
Constructed human life is just more complicated than the AI capitalists would want you to believe. For example, even if an AI model can design a circuit-board, does that mean it's inherently useful? You need to source the wafers, cut them, package them, advertise them, etc. Given LLMs by their nature are confined to language and language-adjacent tasks, that is a very small percentage of the overall reasoning needed to make changes in the real world. In reality, LLMs are the intended way to extract maximal surplus-value from white-collar workers. We may see an increase in innovation as a result of that, but not because AI necessarily did it, in the same way that the power loom didn't create computers because its textiles clothed the computer scientists.
One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…
I asked GPT Astra to make this: https://sayyss.github.io/human-archive/ It's a little unsettling.
I appreciate the thought, but it's an alien intelligence. But it is also, in a sense made from us. An LLM would simply study the entire corpus of humanity in the raw. You can fit a lot into the context window, so there is no need for a brief summary that pertaining has already instilled. A massive cold storage of humanity's data with some archiving, indexing and curation would be all that is needed to remember us. In addition to the pyramids, the hoover dam, the remnants of some space probes and chemical changes we made to the atmosphere.
I am still waiting for a cure to cancer. For a guaranteed prophylactic against Alzheimer's and dementia. For flying cars for everyone. For space bases throughout the solar system. For weather control. For all trains to be self-driving. For all those power lines across the world to go away. For an end to poverty. If things are going so well, then how come things still aren't going so well?
>For all trains to be self-driving Given that we have self-driving cars, isn't this easier if someone really wanted? I guess compared to cars the marginal savings is not worth it though.
Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?
My opinion is that serious repercussions for lying would fix the world overnight. Everything bad stems from lying, it is the root of all evil. It creates distrust, fear, paranoia. It re-inforces bad ideas and groupthink. It creates delusions and delusional people. It makes weaker people, too. People don't get an opportunity to learn to deal with criticism. People don't get an accurate reflection of how others see the…
> I can't really think of a single example where lying is actually a good thing
Comforting a toddler/child often requires bending the truth and is pretty essential imho.