Yeah absolutely. I've been working on my rendering engine on and off for the past 20 odd years. It has been used in commercially in my gigs. However, it's what's considered an offline renderer (like 30+ minutes to render a frame). Last few years, on and off again, I've been slowly working towards a limited in scope realtime version of it. One of the components I've managed to kind of bring almost to the finish line is for it to work with a limited VRAM budget and streaming geometry, where camera has a (I'm simplifying here) bubble around you (around, not a frustum in-front because I need semi global visibility since I'm doing realtime raytracing). This bubble as it moves dictates what geometry and resources are loaded, sometimes even ahead of time because I also do a form of motion prediction. Far out things, outside of the bubble, and things on the border of the bubble go through a coarse to fine representation. That way I can render distant stuff which I don't need much detail for it. This part is a bit buggy, to say the least, but also viably measurable so it's a prime candidate for an LLM to loop/goal over it. Or at least I though so. With max I can see now the progress, but with max I'm now 21+ hours into it and not a clear solution ahead yet. I think I'm running out of weekly quota on ChatGPT Pro 20x for this. I actually bought subscription exactly to drive this home since I don't use LLMs much anymore except gemini in lieu of googling.
I haven't told it how to do stuff, but what I want to do and it picked up my tests to see if it's making progress and I can see it's tuning the movement prediction and caching logic and it fixed few minor bugs. So, it seems to be doing the right approach.. it just takes an awful lot of time. Question is if it's a viable approach even though, but it didn't flag it as wrong. It actually agreed with me and unlike Claude I haven't gotten the impression it's much of a yesman.