Earlier quoted context omitted.
Quote Investigator traces variants back as far as 1626, though it evolves over time. A "rocks all the way down" variant dates to 1838, "tortoises all the way down" to 1854: https://quoteinvestigator.com/2021/08/22/turtles-down/ > The version I'd first heard attributed the story to a lecture by Bertrand Russell and an audience Q&A, though it seems clear that that couldn't have been the first instance.
Your source story sounds like the story I heard/read Kurt Vonnegut tell,
I've run across the Vonnegut variant more recently. I don't recall where or when I heard the earlier version for the first time, though I suspect it came up in conversation without attribution. Likely sometime ~1980 -- 1999.
That variant may well trace to Vonnegut, though I suspect it had been passed through numerous mouths and ears by the time I heard it.