We also had significantly more deaths from things like school shootings and other violence not related to terrorism from Islamist groups, and yet we are pulling out funds for counter-extremism from all causes not related to Islamic terrorism (say, white-supremacist terrorism). By the numbers, the current counter-terrorism policies are an over-reaction, poorly thought out and unfairly target an enormous population that doesn't seem to pose in expectation a more significant risk than others:
http://www.vox.com/world/2017/1/27/14412420/terrorism-muslim... : "The study found that only 46 Muslim Americans (defined as “Muslims who lived in the US for an extended period”) were linked to violent extremism at home or abroad in 2016. The total Muslim American population is 3.3 million."
and, from https://sites.duke.edu/tcths/files/2017/01/Kurzman_Muslim-Am... : "The 54 fatalities caused by Muslim-American extremists in 2016 brought the total since 9/11 to 123. More than 240,000 Americans were murdered over the same period."
So, to prevent 46 or so crimes, affecting 54 or so people (say 300 if you think there are large numbers of wounded), we are de facto abrogating the rights of 3.3 million people, plus visitors. Nevermind that terrorism represents only 0.05% of violent crime, that violent crime itself is at its lowest in decades and that we never felt the need to become a surveillance state over said amount of crime. The only reason why people think it reasonable to do so now is that they associate Muslim with terrorist at a visceral irrational level, not because it makes sound logical sense. Hence, racism and xenophobia is a perfectly reasonable explanation.
Also keep in mind that, over time, treating any group better both at home and abroad reduces incidence of terrorism from that group. But the point is that the current incidence itself is not anywhere near the point where putting millions under surveillance is worth it.