Agree. If you're selective about which bits and pieces you move away from Excel, and not religious about getting rid of Excel entirely, you can often have a great result.
What trips people up is that Excel models can be deceptively inviting of replacement ideas. They may have many workflows within them that are obviously much easier to accomplish with a different tool. E.g., grouping and applying aggregate functions; filtering; joining; filling missing data, especially with constraints such as filling forward EPS estimates but only into the same fiscal quarter; etc. You could look at an ocean of INDEX(,MATCH(...)) functions in Excel and discover that the same manipulation in Pandas would require just two lines of code. But it is easy to overlook the importance of seemingly simple calculation flows, in the same model, that are uniquely suited for being expressed in Excel, the universal language for describing calculations.
For example, I have seen data analysts move from Excel to Python, replacing their models as they go -- after being wowed by Pandas' data manipulation capabilities -- and then get bogged down for months trying to recreate what Excel's =RTD("BLOOMBERG.RTD","",...) already did for them. And when the portfolio manager tells them, "this number looks wrong", which happens often, they spend half a day dumping data into Excel and building a sheet that's illustrative for the PM.