Sheldon Axler's "Linear Algebra Done Right" has my highest recommendation if you want expertise in linear algebra.
As a followup, Paolo Aluffi's "Algebra: Chapter Zero" is the best synthesizing text for abstract algebra for a beginning graduate student. The thing that makes it so amazing is the writing style: it introduces and demystifies category theory, and then discusses groups, rings, modules, linear algebra, fields, r-modules, and advanced topics (toward the end) with the unifying theme of how they work and relate as categories. It is very much a book that focuses on the why over the what and how. And there are many many many juicy exercises.
I would recommend against Spivak's Calculus on manifolds. It's too dense and too focused on advanced topics (unless you're an ivy league undergrad, you don't learn cohomology). That being said I don't have an analysis reference that fits your request.
As a CS person I can continue with book recommendations related to computing-related topics in math (computational algebraic geometry comes to mind). Let me know if you're interested.