All these had a huge impact on me. The descriptions are inadequate, I've just tried to mention the subject matter.
Deborah Tannen, You Just Don't Understand - how males and females talk different languages.
Lakoff & Johnson, Metaphors We Live By - how our language and thoughts are built from a fabric of conceptual metaphors. Philosophy In The Flesh is about the conceptual metaphors that philosophy is built from.
Jonathan Glover, Humanity: A Moral History of the Twentieth Century - WWI, WWII and other wars, nazism, communism etc
Plutarch's Lives - biographies and stories from famous ancient Greeks and Romans. Amazing how little's changed.
Lin Yutang, The Importance of Understanding - introduced me to ancient Chinese philosophers. The Importance of Living - introduced me to ancient Chinese writers, poets and the Chinese way of life.
Hofstadter, Gödel, Escher, Bach - read it when I was 14, and I was into music, art and programming, so it blew my mind.
Susan Faludi, STIFFED - men, work, jobs, masculinity, 20th C
Walter Lippmann, Public Opinion - media, war, propaganda, democracy
Noam Chomsky's political books
E.F. Schumacher, Small is Beautiful - the world, development, government, planning, organization, humanity, sustainability
J.R. Saul, Voltaire's Bastards - power, history, democracy, technocracy, reason, rationality, history from late 18th C until today.
Raymond Williams, Culture and Society 1780-1950 - the anti-industrialist tradition, politics, culture, society and the new language describing these things
Clifford Pickover, Computers, Patterns, Chaos, and Beauty - programming, mathematics, art, fractals, dynamical systems etc
Ben Zander, The Art of Possibility - hard to explain, kind of advanced self-help, the magic of changing attitudes, expectations, habits.
My favourite non-fiction books of all time, though, are the essays of Emerson, Hazlitt, RL Stevenson, GK Chesterton, Santayana, Bertrand Russell. And the books of Nietzsche, SARK and Robert Fulghum.