I think one of the major reasons for our disagreement is you perceive that there is an unlimited risk with GMO organisms - the paper you linked to explicitly states this [1]. Quoting from it:
Ecologically, in addition to intentional cultivation, GMOs have the propensity to spread uncontrollably, and thus their risks cannot be localized. The cross-breeding of wild-type plants with genetically modified ones prevents their disentangling, leading to irreversible system-wide effects with unknown downsides. The ecological implications of releasing modified organisms into the wild are not tested empirically before release.
The problem with this analysis is their assumption that GMOs have a propensity to spread uncontrollably. This is totally false. The problem with GMOs is they don’t spread very effectively unless under human selective pressure. The reason why is really simple - all organisms have a fixed metabolic budget which they can allocate to different functions (grow, reproduction, protection against predation, repair, etc). When you introduce new genes into an organism you have to take away some of this fixed budget to allocate to the new function. This means that when the GMO competes with other organisms in the wild they are at a disadvantage. If you make an argument based on a false assumption then you will get a faulty conclusion.
It is a real shame with this paper as the rest is quite good and an excellent argument for not doing things where ...the consequences can involve total irreversible ruin, such as the extinction of human beings or all life on the planet. GMOs just don’t happen to be one of these things (I am leaving out of GMOs things like creating new pathogens, but you don’t need genetic engineering to create these as the USA and Russia showed in the 1950 and 1960s).
The bottom line is GMOs are not that scary. In fact I know of no serious harm that has come from any other than to make people fearful for no reason and their absence to drive up the price of food.
1. http://arxiv.org/pdf/1410.5787.pdf