While these discussions are always loaded with sentimental intepretation, and complex questions of what "rightful owner" after hundreds or thousands of years even means, I think more of an emphasis should be put on impact for the population. After all, the British Museum, the main example for restitutions, is located in a global city, given completely free access to its huge collection on display and pays for preserv…
The UK has been very stable for a long time, however they are profiting indirectly from the museums, since it s a driver of tourism.
Should the UK become less stable, we should have a hard look at ensuring the continuity of the collection. As others have mentioned, a lot of these things would have been destroyed or forgotten had the British not decided it was important to keep it - and as time goes on, those things become even more irreplaceable.