Update - I am been rated above average in the current appraisal system and quite happy with the work I do. Changing job would be resetting all the good will and trust that I have built here. But after advent of this colleague, I see everything declining as exceptions are changing, sometimes portaged as "irresponsible". I can work like donkey but I like to continue to spend tine with friends and family and follow all my hobbies unlike the peer.
I don't really want to portray my peer as bad(He is mostly nice on the face), just need honest opinion if this job is still a right fit for me.