you don't need 1000 packages and abstractions on top of abstractions just to interact with some DOM elementsI don't think many developers believe that you do. Most of the complexity around 'modern web dev' isn't about achieving the basics of moving DOM nodes around. If that's what you're doing then it's hard to argue that a framework adds much benefit.
Frameworks bring two benefits:
Firstly, they push you down a specific path around the shape of the code. When you're on a team of 20 working on part of an app that shares data across n other components then you need something to keep the code from turning into a swamp. Frameworks bring that experience. You don't need it on a small app or if you're a lone dev, but even then it kind of helps if you're not especially disciplined. If you want an example, have a look at some of the demos from the react-three-fiber team. It's so much nicer to work with a declarative API than imperative vanilla Three.js code.
Secondly, frameworks used well enable you to eek out additional perf. Building an app that renders a complex page in under 16ms isn't that easy if there's a lot going on, and leaning on a scheduler like the one in React makes it simpler. It's still far too easy to get it wrong and kill all your perf even in a framework though.