Basic linear algebra can be taught in elementary school. It definitely doesn't require a lot of background.
Most linear algebra courses and textbooks do. Without a lot of background, you'll run into a hard brick well.
A few good starting points
- Project Algebra's Road Coloring curriculum
- 3B1B (on Youtube) has a series on the geometry of linear algebra. You should have the mechanics of adding/multiplying matrices, but not much more
- Game theory is a nice place to explore early
Linear algebra is a many, many year progression. It's quite deep and interesting. But you don't need to start with singular value decompositions and whatnot. Start with those sorts of things, and then apply it in different domains:
- 3D graphics
- Image processing
- Control theory
- Quantum computing
- Data analysis
... and whatever else suits your interests.
After a decade or so, if rather fascinating deep dives, you'll have a deep understanding not just of linear algebra but of some rather interesting domains.