That could be the case to a degree -- it's likely very hard to rule out simple miscommunication in general as a part of the equation here. But I have a friend who administered this test (to be spoken to its subject):
Close your eyes. Imagine a ball resting on a table. The ball begins to roll, quicker now, and it goes over the edge of the table. It bounces, once, twice, three times, and rolls along the floor, then comes to a stop again.
What color was the ball?
Did you have to make that detail up just now when I asked you, or was that what you saw?
This last question is the decider: a visual detail like that is very difficult to omit while _picturing_ the situation, but is very easy to leave out if your brain doesn't work that way. If you had to choose a color at the end, you may have the true "no picture at all" form of aphantasia, as opposed to just failing to understand that other people don't see a real, solid image.