There is also another problem on how much and how often is Googlebot indexing your site because your site speed is one of the factors of so called Google index budget. My users are in Germany so my VPS is also in Germany to be fast for local user (~130ms for http reply), but for US Googlebot is my site slow (~420ms for http reply). So you are penalized also for this.
Hi - I know some tools report a slow site in this case, but these reports are not accurate - don't believe them! Google is not that stupid ! :D I am currently working as a dev in an SEO-Agency (in Austria), and we never believed this hypothesis - so we tested this once with a bunch of our sites: When moving sites with a German speaking audience to a VPS in America, your rankings at google.de/google.at will decrease (…
A bit off-topic, but out of curiosity, have you run any other interesting experiments like this? I would love to read a blog post about them.