I'm skeptical of any Engineering loop that doesn't include reality (as in touch grass) feedback. Pure logic and reasoning is the domain of Maths and Science (philosophy). Surely it will work, but it will not "be able to solve any learning loop".
Discovery Loop
11–20 of 625 posts
Re: Discovery Loop
#12As LLM coding agents plateau— at least for the average engineer without tens of thousands of dollars or swarms of agents to run —I’d say that, from here on it’s going to be about ASICs, specialized LoRA/or-equivalent models, or a Ruby on Rails for LLM context engineering and orchestration, which LangChain and others seems well position, including Google as they own the entire stack. LLM free lunch has been over for a…
we definitely haven't hit plateau yet. I think a lot of people latch on to anti-LLM narratives without really thinking things through.
Re: Discovery Loop
#13Did the ycombinator podcast which included giving advice to startup founders just a few days ago:
https://www.ycombinator.com/library/Vy-jeff-dean-the-1-rule-...
Re: Discovery Loop
#14Oh wow, that's a blow to Google, what's with the talent scarcity in ML. Though if this goes anywhere Google will likely buy them back.
Re: Discovery Loop
#15Is this a joke? Site is not loading for me.
Re: Discovery Loop
#16I’ve always felt that the idea that science is bottlenecked and therefore needs more automation only works for a very narrow definition of what science is, and entails a very specific view on what it should be.
Re: Discovery Loop
#17> we have pioneered massive scale computing and led the creation of critical infrastructure, products, and foundational AI advances that the world relies on, including multiple generations of Google Search, Google Ads, Google News, Google Translate, Google File System, MapReduce, BigTable, Spanner, TensorFlow, Pathways, TPUs, AlphaChip, AlphaStar, AlphaCode, AlphaFold, Gemini, model distillation, mixture-of-experts model architectures, word2vec, sequence-to-sequence models, chain of thought reasoning, neural architecture search, and multiple generations of Large Language Models (LLMs) among others.
holy shit. I've known this, but...
Re: Discovery Loop
#18From Jeff's twitter post:
> Our general approach is to automate the experimental loop. We think this approach is broadly applicable across many different fields of science and engineering. We’ll initially focus on ML research and engineering, but believe the approach can help with important subproblems in nearly every one of the fourteen NAE Grand Challenge problems. We think doing this well requires strong expertise in machine learning as well as large-scale systems.
See also: https://www.nae.edu/20782/grand-challenges-project
Those 14 are:
NAE Grand Challenges for Engineering
1. Make Solar Energy Economical
2. Provide Energy from Fusion
3. Develop Carbon Sequestration Methods
4. Manage the Nitrogen Cycle
5. Provide Access to Clean Water
6. Restore and Improve Urban Infrastructure
7. Advance Health Informatics
8. Engineer Better Medicines
9. Reverse Engineer the Brain
10. Prevent Nuclear Terror
11. Secure Cyberspace
12. Enhance Virtual Reality
13. Advance Personalized Learning
14. Engineer the Tools of Scientific Discovery
Re: Discovery Loop
#19As LLM coding agents plateau— at least for the average engineer without tens of thousands of dollars or swarms of agents to run —I’d say that, from here on it’s going to be about ASICs, specialized LoRA/or-equivalent models, or a Ruby on Rails for LLM context engineering and orchestration, which LangChain and others seems well position, including Google as they own the entire stack. LLM free lunch has been over for a…
Sounds like you get your news from 2024 when people thought things were plateauing after GPT4?
Re: Discovery Loop
#20I wonder if Jeff Dean facts [1] (I hope people remember the reference) will carry over to the new startup. [1] https://github.com/LRitzdorf/TheJeffDeanFacts
«Jeff Dean's PIN is the last 4 digits of pi.»
I had not read this before, but told many students the same about my PIN code and I a quiz about the last digits. Love it.