> why did the new form suddenly edge out the old in the mid-1990s?
Based on the fact that the years listed in the first section were the birth-years of the speakers, I would guess that the main cultural transition actually took place around 2000-2005, when speakers born after 1995 would be developing their dialects.
I suppose it could be argued that the shift must have happened earlier based on the mixed usage among speakers born as early as 1970, but at any rate, the complete takeover seems to have happened around the millennium change.