There are a few little details elided.
Pastoral lands are pastoral, and not arable, usually because they are not suitable for arable cropping. There is either too little rainfall, or the wrong kind/timing of rainfall, or the soils are wrong. For rainfall reasons forests are out too.
Articles that go on about how ${large_number} percent of the world's non-desert, non-mountain, non-arctic land is used for grazing animals, all omit these inconvenient facts.
The "Sand Hills" region in Nebraska is typical. Dry, could in theory support arable farming, for a while, by drawing down the Ogallala aquifer. But for the fact that that the soil is just sand, and it doesn't give good crop yields. Pouring on the amounts of fertilizer that would be required for cropping would poison the aquifer for everyone else. (In the Sand Hills, the aquifer is highly connected to the surface because of the porosity of the, er, sand.)
Leaving these pastoral lands fallow (without herds of wild ruminants, with their global warming emissions) would just result in continual grass fires.
That said, feeding ruminant animals grains (soy, maize/corn, wheat) is a Bad Idea environmentally as far as I can tell. So is bulldozing the Amazon rainforest to grow soy and cattle. And it might be a Good Idea to re-wild quite a bit of pastoral land. I don't know. Persuading people to leave money on the table is difficult.[1]
1. Major Major's father notwithstanding. https://www.goodreads.com/quotes/771699-his-specialty-was-al...