Earlier quoted context omitted.
Thank you for taking the time to answer the question. In principle, I know how it ought to work. But when would I need to use it? For instance, if someone is making a web api, and you're pulling data from a database, should one think about it? If one is doing some data analysis, should one think about it? I always thought that it is a "low level" program that takes care of it. So tensorflow might worry about it, but…
I don't know your experience so I'll do my best to draw on examples _I suspect_ you might be familiar with. I'll try to keep this simplified but deep enough. I'm taking a shot in the dark and hoping you're more familiar with the one of the python/node/ruby world. You should worry about it when you want to do something faster than you are already or want to do more things at once. Maybe someone's framework/service has…
This was the best part if you want to expand on it in a longer post: "You should worry about it when you want to do something faster than you are already or want to do more things at once. Maybe someone's framework/service done it for you maybe not.
For a web api, the workload _usually_ looks like: {database call} -> {do some work} -> {maybe more calls} -> {more local work} -> {done}.
In the python/node/ruby world where you have mostly single process workers. You get some parallelism by doing work locally while the database call is made asynchronously. If you want to handle more requests on a host you'll just run more processes and load balance across the front (nginx, gunicorn, etc). In languages with threads you usually do that INSIDE your program. A single process load balances across worker threads internally.
At first blush, multiple threads are pretty similar to just running multiple programs. It gets more complicated when you start talking about how you want to communicate between them or share resources across them."