Hopefully it's clear that I know the dictionary meaning of 'atheist'. I'll expand the point I was trying to make slightly and address your use of 'faith'.
In a very strict sense of the word, atheism is an assertion that there is no god/supernatural being. It is clear that this assertion is impossible to prove, only possible to prove wrong. For any god you prove does not exist it is trivial to imagine one that does and which can't be disproved in the same way.
This means that simply stating there is no god is not very useful or meaningful. Much more meaningful is to state that even if there is a god, they take no part in our world. This is the interventionist god that Nick Cave does not believe in.
You are correct in pointing out that the word faith can be used to collect beliefs together and give them a name. That is a reasonable interpretation of the parent's use of the term as well. Too often, however, the phrases "faith of atheism" or "faith in science" are used to draw parallels with other faiths, specifically faiths that require "blind faith" as part of their belief system. Take the whole line in context:
> To me such ideas as the naive assumption that we know the limits of knowledge go hand in hand with the faith of atheism.
The thrust of this statement, as I read it, is identical to the one Asimov discusses in that essay:
"If I am the wisest man," said Socrates, "it is because I alone know that I know nothing."
That is, it is impossible to know what you don't know and believing you do is foolish. He continues:
The implication was that I was very foolish because I knew a great deal.
The argument in the parent is that people who believe in atheism are foolish for believing they know the limits of knowledge. My counter argument is that atheists - like most people - don't tend to claim they know the limits of knowledge. Their distinguishing characteristic is that at no point does an explanation come to rest on blind faith. That is why saying "Atheism isn't a belief that there is no god" is a meaningful one.
There are probably other words that could be used instead of atheism to label this belief system that arises when you discard blind faith. No one needs a label for their belief that there isn't a teapot circling the sun in the exact same orbit as the earth, but 6 months the other side. Atheism is a useful word precisely because there is a long seated assumption in the consciousness of mankind that there must be a god.