[1] https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
When AI Builds Itself: Our progress toward recursive self-improvement
201–210 of 738 posts
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#202We've had self-improving AIs before, and they tended to get lost after a while. That's going to be a problem. LLMs are stable because they return to a ground state with no history for a new job. Systems with persistent state have a problem with that state not being sane. Remember Microsoft's 2016 chatbot that learned from Twitter? [1] [1] https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#203Earlier quoted context omitted.
This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".
As someone who maintains a roguelike with a terminal-like UI that: 1. Maintains an internal representation of what the game thinks is on screen. 2. Runs the game for one frame which updates that representation. 3. Generates a diff to see how that differs from what's actually on screen. 4. Executes the minimum set of draw calls to get the screen to match the internal representation. It's really not that hard. It's a f…
> -> rasterizes them to a 2d screen
Also you forgot "render to a framebuffer, then parse the framebuffer back to chars".
Anyway, I'm off to construct the new `ls` command. It will render the list of files to a mesh of billions of polygons in a GPU with advanced shaders, 16x oversampling, HDR and all the graphic acronyms I don't understand, then read the resulting image, find the nearest character in the ANSI charset and use that one.
It will be _glorious_ (and profoundly stupid)
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#204We've had self-improving AIs before, and they tended to get lost after a while. That's going to be a problem. LLMs are stable because they return to a ground state with no history for a new job. Systems with persistent state have a problem with that state not being sane. Remember Microsoft's 2016 chatbot that learned from Twitter? [1] [1] https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#205> We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research—in collaboration with many others—and take actions to help build the systems that a credible slowdown or pause would require. Interesting - they're commiting…
i don't want to be a negative nancy but i'm sure this "slowdown" will only be in effect until the infrastructure buildout is done or largely done. If they weren't hardware constrained there'd be no slowdown at all. Whoever gets there first wins everything ("there" being defined as AGI or a similar scale leap in capability).
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#206Earlier quoted context omitted.
> For each frame our pipeline constructs a scene graph with React then > -> layouts elements > -> rasterizes them to a 2d screen > -> diffs that against the previous screen > -> finally uses the diff to generate ANSI sequences to draw Yup. Overengineering.
What is "frame" in this context? Video frame, or something else?
> We have a ~16ms frame budget so we have roughly ~5ms to go from the React scene graph to ANSI written.
It looks like video frame, full framebuffer, generated and parsed at 60fps. It surprises me they haven't introduced GPU shaders, 16x oversampling and raytracing. Maybe for next release.
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#207Earlier quoted context omitted.
Sorry for nitpicking, but: > If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? Arguably, yes.
Is the idea to keep the world in balance via MAD? I could see that, though it's a dangerous gamble. From Richard Rhode's "The Making of the Atomic Bomb", I got the impression that most scientists involved thought they could manage a US or UN monopoly on nukes after the war. General Groves attempted to buy up all of the world's uranium ore. Unfortunately, it is only high grade ore that is rare; many countries have low…
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#208Okay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages wit…
Indeed... why is Anthropic even employing people at all if this AI magic story is true?
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#209I find any and all claims like this ridiculous from a company who can't build a terminal application that uses less than a gigabyte of RAM.
Re: When AI Builds Itself: Our progress toward recursive self-improvement
#210Earlier quoted context omitted.
This might explain it, in the opposite way it was meant to: https://fxtwitter.com/trq212/status/2014051501786931427 > Most people's mental model of Claude Code is that "it's just a TUI" but it should really be closer to "a small game engine".
> For each frame our pipeline constructs a scene graph with React then > -> layouts elements > -> rasterizes them to a 2d screen > -> diffs that against the previous screen > -> finally uses the diff to generate ANSI sequences to draw Yup. Overengineering.