Carve out some time and see if you can push yourself to finish something without the constraint of making it high quality.
If you can do this when you intentionally tried, it means you're a perfectionist.
If you cannot no matter how you try, it means you're bad.
It's important that you try this out and actually find out who you are, unless you can't do it because you're too scared.
If you find out that you're a "perfectionist" and that's why its slowing you down, try to learn how to balance. People never doubt you unless you're consistently slow. I am sure what happened to OP was he was consistently slow. It's extremely hard for a manager to just tell people they're slow, and this guy heard it multiple times, so it probably means he IS very slow.
Every programmer knows it's always not as straightforward as it looks from outside. Sometimes sou would come across a task that looks like it would take 30 minutes just to find out it's much more complex than that and it takes up your entire week. But sometimes it looks like it will take a week but turns out you can finish in 30 minutes. That's why most people tend not to judge you for being slow sometimes. It's only when you're ALWAYS slow that people doubt you.
Think about this for a moment: you say you do everything by the book but everyone else does more--have you ever thought if it maybe the case that these other people also do things by the book?
If you have doubts about yourself, you should really proactively figure out ways to tackle this. In most cases it's not because you're stupid, it's because you have bad habits and you just think it's natural.