My youngest brother has schizophrenia, which started when he was 19. He just turned 31, and were it not for his family supporting and caring for him, I'm sure he would be homeless or worse. He's one of the most sensitive, gentle people I have ever known. My wife and others who have gotten to know him would agree. Although he's "stable", thanks to an injection every 3 weeks, he's not going to ever get better. In fact…
It is human nature to want to associate with people who display attributes you wish to see in yourself, and to avoid the converse. The aphorism "you are the average of your 5 closest friends" is based on that idea.
If your brother is self-aware of his illness and doesn't like its effects on him then his response is perfectly understandable to me.