Wouldn’t be not parsing incoming media unless it’s for someone from contact is a first step that can reduce largely the thread ? So it will stay the same for people in your contact list but a new touch to load for message from unknown person
Many services from banks to healthcare utilize SMS as a main way of communicating with end-users. many rely on dynamic numbers.
Moreover, spoofing SMS messages is not that hard.
Messaging apps whether it is SMS or alternatives like whatsapp, telegram etc. will always offer a powerful vector to infect devices.