When I was in school I taught a physics lab to non-physicists, and this was a conversation professors and I had. Granted this was at university level so most people had an idea about whether they (thought they) were a math person or not. I went in with a completely different philosophy to that class than with physicists. Because here's the thing, a physicist will care about different things than the non-physicist.
What I focused on was less of the physics and mathematics and more on making sure they gained an intuition about how nature works and how to think critically. These people didn't need the same toolbox as the ones trying to get a degree in physics or engineering. I could care less if these students knew the equations for Newton's laws, but I did care if they had an intuition. I didn't care if they had Bernoulli's principle memorized, but I did care if they could analyze an experiment.
The difference is that these people needed different skills in their lives. You must know it is popular to say things like "School taught me the Pythagorean theorem but not how to do my taxes." These people aren't realizing that math is giving them a toolbox that can help them do their taxes and other things. I teach my young nephews math whenever I visit them. They don't care about it in school but they like what I teach them because I make it fun and challenging. You can get a lot of topics covered and a lot of ideas and principles conveyed if you aren't worried about them being able to do every case. An example of this being that the basic principles of calculus can be used in your daily life and could benefit everyone (thinking about things like rates of change, tangents, series, limits, and squeeze theorem), but they won't need to be able to take the derivative of a function unless they are going into a job that requires that.
So I guess what I'm getting at is that there is a lot of benefit from math to the average person without actually requiring them to be fluent in the language. Think about this like being a moderate speaker in a second language but not being able to write or hold a deep conversation. They have a lot of advantages over someone who might know more about the written language and grammar, but don't know many words. I think the same argument of learning basic phrases in a second language would apply here as well. Most people don't need to be fluent, but we are teaching them like they are.