Earlier quoted context omitted.
Don't be alarmed citizen. They only 'know' about your movements if they actually look at them. Until then, they don't 'know' anything as long as it sits in their archives untouched.
It's a bit more Orwellian than that. The NSA claims it doesn't "collect" data until it looks at the data. Somehow the data magically appears in its databases, but it isn't collected.
These caches of information exist all over the place. While I'd prefer NDA/GCHQ to not slurp and store this stuff at least they have a duty to keep it secret. I am a lot more bothered by my ISP keeping that stuff. They're a lot less competant and have many fewer access controls.