Earlier quoted context omitted.
You'd imagine the pilots would notice if the auto-pilot started flying erratically. I'm sure it could be used to confuse the hell out of the pilots though.
Instrument failure isn't always detected by the crew. In the case of Air France 447 in 2009, even the instruments that were working correctly weren't properly understood. The black box transcript from AF 447 is quite an interesting read: http://www.popularmechanics.com/technology/aviation/crashes/...
I remember reading that - it is quite terrible but fascinating. It seemed pretty clear that the the crash was 100% human error due to the co-pilot basically panicking and losing his mind. Though it was initiated by an instrument failure.