I think the position of Gebru, et al, can best be expressed as a form of "human exceptionalism". As a teenager, my english teacher shared this writing by Faulkner, which he delivered as his Nobel Prize speech. I complained because the writing completely ignores the laws of thermodynamics about the end of the universe. """I believe that when the last ding-dong of doom has clanged and faded from the last worthless rock…
This is partially reflected by later observations from training LLMs where we see that the performance of LLMs increases even on purely language tasks when adding extra modalities such as computer code or images, which, in a sense, bring the model closer to different aspects of reality; and we observe that adding tiny quantities of "experimental interaction" through RLHF can bring features that additional humongous amounts of pure surface form training data can't, and it certainly seems plausible that making a qualitative leap further would require some data from actual causal interaction with the real world (i.e. not replay of data based on "someone else's" actions but feedback from whatever action the model currently feels is the most "interesting" i.e. the outcome is uncertain to the model but with potential for surprise data), where relatively tiny amounts of such data can enable learning what large amounts of pure observations can't - just as the hypothetical octopus from the stochastic parrot paper thought experiment.