> But there were not many orcs and goblins for a long time in many areas, as we learned from the books. They only came back and multiplied with the slow return-growth of Sauron.
Sauron had already returned c1000 TA to Dol Guldur. We're specifically told that evil things, such as Orcs, begin multiplying in number by c1300 TA, which triggers the migration of the Hobbits to the Shire (already we have an example of why there's less people than we expect.) In c1400 the Nazgul begin their long destruction of the disunited (perennially entangled in civil war) kingdoms of Arnor. In c1500, Gondor is embroiled in devastating wars with Harad. In the 1600s a Great Plague depopulates the North and West: "many parts of Eriador become desolate"; not long before, Gondor has had a horrible civil war. During the plague, even the Shire suffers "great loss." In the 1800s Gondor is again involved in a costly war, this time against the Wainriders, and loses its eastern territories. In 1981 TA Moria falls. In the 2400s Gondor is again under a major attack by (though they do not know it) Sauron, and Sauron is able to use the Orcs in the Misty Mountains to effectively cut off Eriador from the rest of the continent. in the 2500s, the Gondorians are bailed out of a major disaster by the appearance of the Rohirrim, but shortly after, the dragons appear and decimate the dwarves. In the 2700s, after centuries of Orc attacks in the region, the Hobbits themselves defeat a band of Orcs; shortly after, Dale and the Iron Mountain fall to Smaug, Rhoah is nearly destroyed, and Gondor is attacked by fleets of Corsairs. In addition, there is a devastating winter that causes "great suffering and loss of life"; in the same century, Smaug annihilates Erebor and Dale, and there's a major, costly war between the dwarves and the Orcs.
In the following centuries, there are continual major Orc attacks. In 2900 Men finally abandon Ithilien. In 2911 there is another devastating winter, and the year after, great floods that result in the desertion of Tharbad and Minhiriath.
The Hobbit begins in 2941. Orcs have been in force for 1600 years.