Do I need to build my projects without a framework/library to understand more?

I've been having struggles starting my projects and sticking to it till I finish, especially when I try to use frameworks like React. Working with state in React is one thing I still struggle with. When a library like Redux comes in, I get a lot more caught up with mastering the framework/library before I start my project.

Since I didn't have much experience making projects without a framework,(e.g did not work with DOM much before React either, I only had a few projects using a basic HTML/CSS/JS file), do I need to focus on projects without a framework to help with my understanding of React and/or any other framework/library?

Just forget frameworks, they make things a LOT easier but they really don't help you learn the fundamentals.

Once you've got the fundamentals down, i.e you know the language and can do whatever you need without a framework, then you can move on and learn one and for that I'd do a complete structured learning course, something like https://www.fullstackreact.com/ then once you've mastered the language, and then mastered the framework, you're good to go. Otherwise you're just going to flounder and produce crappy code.