71–80 of 159 posts
Ah good, we're getting closer and closer to Venus, Inc. every day. /s
AI improving itself (or at least the architecture it runs on), the singularity is near as they say. Do we have other examples of AI being used to improve the LLMs, apart for the creation of synthetic data and the testing of the models?
A more efficient transformer just costs less to run.
"AI improving AI" would be if one generation of AI designed a next-gen AI that was fundamentally more capable (not just faster/cheaper) than itself. A reptilian brain that could autonomously design a mammalian brain.
Even when hooked up into a smart harness like AlphaEvolve, I don't think LLMs have the creativity to do this, unless the next-gen architecture is hiding in plain sight as an assemblage of parts than an LLM can be coaxed into predicting.
More likely it'll take a few more steps of human innovation, steps towards AGI, before we have an AI capable of autonomous innovation rather than just prompted mashup generation.
This reminds me of Antirez's "Don't fall into the anti-AI hype" [0] In a sentence: These foundation models are really good at optimizing these extremely high level, extremely well defined problem spaces (ie multiply matrices faster). In Antirez's case, it's "make Redis faster". There have been two reactions: "Oh it would never work for me" and "I have seen months of my life accomplished in an hour", and I think they'…
AI improving itself (or at least the architecture it runs on), the singularity is near as they say. Do we have other examples of AI being used to improve the LLMs, apart for the creation of synthetic data and the testing of the models?
Shameless plug: https://huggingface.co/spaces/smolagents/ml-intern It’s a simple harness around Opus, but with tight integration to Hugging Face infra, so the agent can read papers, test code and launch experiments
Re: hyperparameter tuning and autoresearch: https://news.ycombinator.com/item?id=47444581
Parameter-free LLMs would be cool
This reminds me of Antirez's "Don't fall into the anti-AI hype" [0] In a sentence: These foundation models are really good at optimizing these extremely high level, extremely well defined problem spaces (ie multiply matrices faster). In Antirez's case, it's "make Redis faster". There have been two reactions: "Oh it would never work for me" and "I have seen months of my life accomplished in an hour", and I think they'…
>I think the rest of us should rest easy knowing that LLM's can't (and maybe were never meant to) tackle the tacit-knowledge-filled, human-system-centric, ambiguously-defined-problem-space jobs most mortals work I don't believe that anymore, to be honest. Models are starting to get good at ambiguity. Claude Code now asks me when something is ambiguous. Soon, all meetings will be recorded, transcribed and stored in a…
Earlier quoted context omitted.
>I think the rest of us should rest easy knowing that LLM's can't (and maybe were never meant to) tackle the tacit-knowledge-filled, human-system-centric, ambiguously-defined-problem-space jobs most mortals work I don't believe that anymore, to be honest. Models are starting to get good at ambiguity. Claude Code now asks me when something is ambiguous. Soon, all meetings will be recorded, transcribed and stored in a…
So self chosen total surveillance and transparency so your fav LLM can be better?
Earlier quoted context omitted.
Advanced Machine Intelligence (AMI), a new Paris-based startup cofounded by Meta’s former chief AI scientist Yann LeCun, announced Monday it has raised more than $1 billion to develop AI world models. LeCun argues that most human reasoning is grounded in the physical world, not language, and that AI world models are necessary to develop true human-level intelligence. “The idea that you’re going to extend the capabili…
Now check how much OpenAI got in their last funding round, and you have your answer.
I would be interested to see how exactly the agent helped. How was it used, where did it lead to the given improvement and in how far would it have taken a human to come to the same solution.
Earlier quoted context omitted.
Why not both? You don’t need 1trillion allocated before you have a proof of concept to demonstrate your non-LLM model, and once you have a PoC you will definitely have the larger investors interested
You will need 100s of billions to make a viable POC.
This reminds me of Antirez's "Don't fall into the anti-AI hype" [0] In a sentence: These foundation models are really good at optimizing these extremely high level, extremely well defined problem spaces (ie multiply matrices faster). In Antirez's case, it's "make Redis faster". There have been two reactions: "Oh it would never work for me" and "I have seen months of my life accomplished in an hour", and I think they'…