Earlier quoted context omitted.
> But also more of our time thinking about higher level, hard, technical problems (e.g., how do we use math to build a system that dynamically optimizes itself for whatever metric we care about?). It’s likely that a near-future AI system can suggest suitable math and implement it in an algorithm for the problem the user wants solved. An expert who understands it might be able to critique and ask for a better solution…
This kind of optimization is what I did for the last few years of my career, so I might be biased / limited in my thinking about what AI is capable of. But a lot of this area is still being figured out by humans, and there are a lot of tradeoffs between the math/software/business sides that limits what we can do. I'm not sure many business decision makers would give free rein to AI (they don't give it to engineers to…
Yes, unless models are being live fine-tuned, but generally yes.
>What we call AI today does not use reason
I don’t think this is correct- I think it’s more accurate to say it reasons on its priors rather than from first principles.
For the most part, I agree with the rest of your post.