If you use pencils a lot, do yourself a favor and get a dedicated eraser. I personally like the Sakura SumoGrip block erasers, but even a $3 Pentel Clic from your local big box store will be an upgrade from what is typically on the back of a pencil.
The harder the eraser is the more prone it is to destroying your paper or making smudges. The disadvantage of a soft eraser is that it wears more quickly. I don't buy expensive enough erasers to where that's a problem.
Most pink erasers are fine, but I've had more than once instance of them working more like a crayon than an eraser.
Recently I got recommended trying an edding R20 eraser. It's a weird one. It's much more plastic-y than the average eraser. It's quite firm too, but still manages to not smudge and keep itself clean. Haven't used it enough to form a proper opinion yet however.