Here's a video of Dynamicland.[1] The textual description doesn't tell you much.
It's still at the cool demo level, though. How do you scale this thing?
What is Realtalk’s relationship to AI? (2024)
11–20 of 98 posts
Re: What is Realtalk’s relationship to AI? (2024)
#12I've never experienced dynamicaland in person (only seen videos). However, one concern I have about it's demos so far is that they use a projector. So you need a room dark enough to for the projected light and you need to keep your heads, hands, and body out of the way of it.
Re: What is Realtalk’s relationship to AI? (2024)
#13Such an amazing project.
I've made a lot of progress recently working on my own homebrew version, running it in the browser in order to share it with people. Planning to take some time soon to take another stab at the real (physical) thing.
Progress so far: https://deosjr.github.io/dynamicland/
Re: What is Realtalk’s relationship to AI? (2024)
#14> we aim for a computing system that is fully visible and understandable top-to-bottom I mean even for something that is in theory fully understandable like the linux kernel it is not feasible to actually read the source before using it. To me this really makes no sense. Even for traditional programming we only have so powerful systems because we use a layered approach. You can look into these layers and understand t…
You're right, and when the behaviour of large codebases violates expectations, the intertwined webs of code are problems. Alan Kay's Viewpoints Research studied this, and famously Kay proposed "t-shirt sized algorithms", where short rules can be used to make even fancy desktop word processing and presentation software. There are also projects like From NAND to Tetris that show understanding a full stack is achievable. Could this go broader, and deeper? Of course, and this is what Bret Victor is getting at. Not just for when code goes wrong, but to make it more modifiable and creative in the first place (See Maggie Appleton's essay/talk "Home-Cooked Software and Barefoot Developers").
Projects like SerenityOS show how powerful "small" software can be. For instance, its spin-off project, the Ladybug browser has vastly fewer lines of code compared to Chromium, and yet it seems that the Ladybug team is able to implement one specification after another. Last I saw, they were close to meeting the minimum feature set Apple requires for shipping as a browser on iOS.
Re: What is Realtalk’s relationship to AI? (2024)
#15I've never experienced dynamicaland in person (only seen videos). However, one concern I have about it's demos so far is that they use a projector. So you need a room dark enough to for the projected light and you need to keep your heads, hands, and body out of the way of it.
This is true, but modern laser projectors are very, very bright. I use one as my main computer display with no problems with the blinds open, and the sun shining in.
Occlusion is definitely a problem.
Re: What is Realtalk’s relationship to AI? (2024)
#16I'm only just now reading about Dynamicland for the first time, so maybe I'm not understanding something obvious. The text description is not very helpful, as far as I can tell from pictures it's a place where you can move around physical objects and papers to do computer programming type stuff?
Under visibility they say:
>To empower people to understand and have full agency over the systems they are involved in, we aim for a computing system that is fully visible and understandable top-to-bottom — as simple, transparent, trustable, and non-magical as possible
But the programming behind the projector-camera system feels like it would be pretty impenetrable to the average person, right? What is so different about AI?
Re: What is Realtalk’s relationship to AI? (2024)
#17I've never experienced dynamicaland in person (only seen videos). However, one concern I have about it's demos so far is that they use a projector. So you need a room dark enough to for the projected light and you need to keep your heads, hands, and body out of the way of it.
Projectors have been strong enough to be visible in decently lit rooms for ages. The reason you want the room extremely dark for most projector setups is for contrast, because the darkest thing you can make on a projected image is the ambient surface illumination (and the brightest is that surface under full power from your projector [0]). If you accept that compromise you don't need a super dark room, the recommendation for tight light control is mostly for media viewing where you want reasonable black levels.
Do still need to keep hands out of the light to see everything but that can also be part of the interaction too. If we ever get ubiquitous AR glasses or holograms I'm sure Bret will integrate them into DL.
[0] Which leads to a bit of a catch 22 you want a surface that looks dark but prefectly reflects all the colors of your projector so you need a white screen which means you ideally want zero other light other than the projector to make the projector act the most like a screen.
Re: What is Realtalk’s relationship to AI? (2024)
#18I've never experienced dynamicaland in person (only seen videos). However, one concern I have about it's demos so far is that they use a projector. So you need a room dark enough to for the projected light and you need to keep your heads, hands, and body out of the way of it.
>you need to keep your heads, hands, and body out of the way of it.
I've seen systems like this that use multiple projectors from different angles, calibrated for the space and the angle. They're very effective at preventing occlusion, and it takes fewer than you'd think (also see Valve's Lighthouse tech for motion tracking).
Unfortunately, doing that is expensive, big, and requires recalibrating whenever it's moved.
Re: What is Realtalk’s relationship to AI? (2024)
#19I'm only just now reading about Dynamicland for the first time, so maybe I'm not understanding something obvious. The text description is not very helpful, as far as I can tell from pictures it's a place where you can move around physical objects and papers to do computer programming type stuff? Under visibility they say: >To empower people to understand and have full agency over the systems they are involved in, we…
Dynamicland is bootstrapped in a sense, [0] the same way you write the first compiler/interpreter for your code in another language then later write it in it's own language. The code running the camera and projector systems is also running from physically printed programs in one of the videos you can see a wall that's the core 'OS' so to speak of Dynamicland.
I think the vision is neat but hampered by the projector tech and the cost of setting up a version of your own, since it's so physically tied and Bret is (imo stubbornly) dedicated to the concept there's not a community building on this outside the local area that can make it to DL in person. It'd be neat to have a version for VR for example and maybe some day AR becomes ubiquitous enough to make it work anywhere.
[0] Annoyingly it's not open sourced so you can't really build your own version easily or examine it. There have been a few attempts at making similar systems but they haven't lasted as long or been as successful as Bret's Dynamicland.
Re: What is Realtalk’s relationship to AI? (2024)
#20I'm sympathetic, but I do think Realtalk could be improved with some simple object recognition and LLMing. One of the challenges I found when I played with RealTalk is interoperability. The aim is to use the "spacial layer" to bootstrap people's intuitions on how programs should work, and interact with the world. It's really cool when this works. But key intuitions about how things interact when combined with each ot…
That's similar to the issue with the whole NFT craze where you'd "take items from one game to another", it requires everything to work with everything.
For Dynamicland I get the issue though putting the whole thing through an LLM to make pointy and sharp both trigger the same effects on another card would just hide the interaction entirely. It could or couldn't work for reasons completely opaque to both designer and user.