Logo with a physical turtle?
Can something be done with an overhead camera, and a bunch of cardboard cutouts using QR codes? I remember one of the (big name?) universities had a project along these lines. In their case they used different patterns of coloured dots on objects, and every object in the room was mapped to a virtual equivalent. Interaction with the computer was solely by moving objects around in the room. I can't find it on Google. It might have had a connection to the Dynabook project? They probably have code ready to go.
Edit: Found it: Dynamicland
https://dynamicland.org/
No source code available?
It has appeared on Hacker News:
https://news.ycombinator.com/item?id=26725370
Also: https://paperprograms.org/
https://github.com/janpaul123/paperprograms
Maybe you could have different shaped bits of paper which implement blocks using javascript? Add a "print" function and the programmer could consolidate something they have written using multiple blocks/sheets into a single block/sheet, so allowing more complexity in a smaller space? Alternatively, can it be mapped directly to scratch instead of javascript?