Traditionally the age is considered to have come to an end with the Mongol Siege of Baghdad, wherein the invaders sacked the city. Baghdad was the center of learning, and the fact that scholars from all around the empire (including non-Arab, and even non-Muslim) were able to travel to and, more importantly, find institutional support in Baghdad was one of the main drivers behind the age.
But institutional support had already begun to wane as a consequence of a theological shift in Arabic Islam from Mu'tazilism to Ash'arism. Ash'arism is often described as pro-rationalism, but compared to Mu'tazilism it was a significant regression, especially in terms of the motivation for official, institutional support for the study of natural philosophy. Despite the sacking, Baghdad recovered relatively quickly, but institutional support for natural philosophy never did recover.
Work in mathematics, medicine, and natural philosophy in the Islamic world didn't end. But it found much less support, and often came from the periphery, further away from the now more theologically conservative Arab world, and fragmented. Proximity is everything, especially before the age of telecommunications. There were no fewer geniuses in the Islamic world, but now they were cut off from each other. It's much like the Dark Ages in Europe--there was no shortage of smart people doing smart things, but they lost networking benefits and enjoyed less patronage.