It isn't a science podcast per se, but he actually has a lot of scientists/science educations on. Neil deGrasse Tyson has been on four times, Sean Carroll three times, Brian Cox twice, Brian Greene once, Roger Penrose once, etc. (those are just the science people I remember off the top of my head, there have been lots) He's had Richard Dawkins, Elon Musk, John Carmack, Sam Harris on if you're into them. But it's
very hit or miss. He has about as many quacks on as legitimate scientists/educators.
He does like 3-5 episodes a week, and they're each 2-4 hours. So there's a shitload of content, much more than I could ever sit through. When I was commuting ~3 total hours each day, podcasts were my medium of choice, I would generally listen to maybe one of his podcasts per week and skipped the other 2-4 after listening long enough to know that I didn't give a shit about the guest, whether it's because they were a quack, some random boring comedian, or some meathead MMA person.
He is (rightly) criticized for having lots of shitty guests on, and that criticism is fair. But the fact that the podcast is often bad doesn't mean that it's always bad. When the podcast is at its best, it is very, very good.
On the website the episodes are broken down by category, so if you want science content you just click "scientists". Unfortunately Joe is too dumb to tell the difference between the real scientists and the fake ones, so most of the quacks are lumped in there too.
https://www.jrepodcast.com/episodes/scientists/