Scientists have discovered that being poor actually impairs our cognitive abilities. http://www.huffingtonpost.com/2013/10/28/the-lasting-impacts...
Relevant quotes from the abstract:
Adults with lower family income at age 9 exhibited reduced ventrolateral and dorsolateral prefrontal cortex activity and failure to suppress amygdala activation during effortful regulation of negative emotion at age 24. ... The findings demonstrate the significance of childhood chronic stress exposures in predicting neural outcomes during emotion regulation in adults who grew up in poverty.