Hmm. No one has mentioned Elements of Computing Systems: Building a Modern Computer from First Principles by Nissan and Schocken (which is absolutely superb).
Great suggestion. I flipped through Computer Architecture, A Quantitative Approach ( http://www.amazon.com/Computer-Architecture-Fifth-Quantitati... ) at the bookstore the other day and it looked interesting, but that's all I could gather in the short time I had.
If this is absolutely the first time you are looking at architecture, http://www.amazon.com/Computer-Organization-Design-Fifth-Arc... by the same authors might be a easier entry point.