Some of the stuff I've seen our cat eating, it's both obvious and a blessing that he doesn't have much of a sense of taste. Goes nuts for the plastic wrappers you get around Pepsi cans and won't stop licking them. Little weirdo.
But the "taste" you're talking about is the one we tend to use in general language, and it's a complicated mix of touch/texture/temperature sense in the mouth and tongue, taste bud chemical reception, and more than anything else smell, which "fills in the chemical blanks" that the coarse taste senses miss.
Cats miss one but have the others in spades. When they lick and eat random junk, it's for a very good sensory reason. Still weirdos, obviously, but ones with a good excuse.