Live data from Hacker News

Genesis – a generative physics engine for general-purpose robotics

genesis-world.readthedocs.io

41–50 of 54 posts

Re: Genesis – a generative physics engine for general-purpose robotics

#41

Earlier quoted context omitted.

Depends where your boundary for "100% Anything" is I suppose. It seems to use GPU accelerated kernels written in Python via the Taichi library for most of the physics calculations. At some point, sure, the OS+GPU driver+GPU firmware you need to run the GPU accelerated kernel are not written in Python (and if you run it on CPU instead it will be slow, but more because you're using the CPU than you're not using C or so…

The doc emphasizes "100% Python" and that backend is natively in Python. I'm reading this as "you don't need anything else than Python interpreter." Given a large number of packages aren't in Python under the hood, that's a big, unnecessary hyperbole. It's Ok to acknowledge that there's a heavily reliance on non-python code, e.g. Taichi or Numpy. I also think that the distinction isn't particularly useful. Just pedan…

It’s particularly useful if it is an open source project and you want to communicate to people who might want to hack on it (either in a fork or the main project) what languages they will need to work directly with to do so.

It’s not important to end users, but they aren’t the only audience.

Re: Genesis – a generative physics engine for general-purpose robotics

#42
post #7

What method is Genesis using for JIT compilation? What subset of Python syntax / operations will be supported? The automatic differentiation seems to be intended for compatibility with Pytorch. Will Genesis be able to interface with JAX as well? The project looks interesting, but the website is somewhat light on details. In any case, all the best to the developers! It's great to hear about various efforts in the spac…

> What method is Genesis using for JIT compilation?

Taichi and Numba are both in the pyproject.toml

Re: Genesis – a generative physics engine for general-purpose robotics

#43
post #10

In the sizzle reel, the early waterdrop demos are beautiful but seem staged, the later robotics demos look more plausible and very impressive. But referring to all these "4D dynamical worlds" sounds overhyped / scammy - everyone else calls 3D space simulated through time a 3D world. > Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym an…

> But referring to all these "4D dynamical worlds" sounds overhyped / scammy - everyone else calls 3D space simulated through time a 3D world.

In the research community, "4D" is a commonly used term to differentiate from work on static 3D objects and environments, especially in recent years since the advent of NeRF.

The term "dynamic" has long been used similarly, but sometimes connotes a narrower scope. For example, reconstruction of cloth dynamics from an RGBD sensor, human body motion from a multi-view camera rig, or a scene from video, but assuming that the scene can be decomposed into rigid objects with their individual dynamics and an otherwise static environment. An even narrower related term in this space would be "articulated", such as reconstruction of humans, animals, or objects with moving parts. However, the representations used in prior works typically did not generalize outside their target domains.

So, "4D" has become more common recently to reflect the development of more general representations that can be used to model dynamic objects and environments.

If you'd like to find related work, I'd recommend searching in conjunction with a conference name to start, e.g. "4D CVPR" or "4D NeurIPS", and then digging into webpages of specific researchers or lab groups. Here are a couple interesting related works I found:

https://shape-of-motion.github.io/ https://generative-dynamics.github.io/ https://stereo4d.github.io/ https://make-a-video3d.github.io/

All that considered, "4D dynamical worlds" does feel like buzzword salad, even if the intended audience is the research community, for two main reasons. First, it's as if some authors with a background in physics simulation wanted to reference "dynamical systems", but none of the prior work in 4D reconstruction/generation uses "dynamical", they use "dynamic". Second, as described above, the whole point of "4D" is that it's more general than "dynamic", using both is redundant. So, "4D worlds" would be more appropriate IMO.

Re: Genesis – a generative physics engine for general-purpose robotics

#44
So we can run AI agents with RL in molecular level simulations for replacing product designing,machanical engineering, electrical engineering, aerospace engineerig and everything else right!!? If we can combine protein folding too then we could possibly solve any disease and poverty with fully automation

Re: Genesis – a generative physics engine for general-purpose robotics

#46
post #32
post #22

I suspect that the actual generation and simulation/rendering takes several minutes for each step.

The simulation/rendering is actually pretty fast since it's all done by heavily optimized gpu-based physics and graphics engines. The "generative" part is that they have some LLM stuff that's finetuned for generating configurations/parameters for the physics engine conditioned on some text. Ie, the physics and graphics are classical clockworky simulations, with a generative frontend to make it easier (but less precis…

That's what I meant about the LLM.

If you are saying it's a real time simulation and rendering then I did not realize that.

Re: Genesis – a generative physics engine for general-purpose robotics

#47
post #32
post #22

I suspect that the actual generation and simulation/rendering takes several minutes for each step.

The simulation/rendering is actually pretty fast since it's all done by heavily optimized gpu-based physics and graphics engines. The "generative" part is that they have some LLM stuff that's finetuned for generating configurations/parameters for the physics engine conditioned on some text. Ie, the physics and graphics are classical clockworky simulations, with a generative frontend to make it easier (but less precis…

man i love what u writing even as far of this post ur 2012 post about SVM, i am 2nd year CS student from indonesia.

are you a phd on computer science?

Re: Genesis – a generative physics engine for general-purpose robotics

#48

Earlier quoted context omitted.

> "4D dynamical worlds" Its a feature of that field of science. I'm currently working in a lab that is doing bunch of things that in papers are described $adjective-AI. In practice its just a slightly hyped, but vaguely agreed upon by consensus in weird science paper english term, or set of terms. (in the same way that guassian splats and totally just point clouds with efficient alpha blending[only slightly more comp…

They could be implying a little bit of computer graphics in the mix. Rotation, shear, and transformation matrices have a dimension of 4.

I mean yeah the transformation matrix is 4x3.

Re: Genesis – a generative physics engine for general-purpose robotics

#49
post #15

I saw this on twitter and actually came on HN to see if there was a thread with more details. The demo on twitter was frankly unbelievable. Show me a water droplet falling...okay...now add a live force diagram that is perfectly rendered by just asking for it? What? Doesn't seem possible/real. And yet it seems reputable, the docs/tech look legit, they just "aren't released the generative part yet". What is going on he…

My understanding is they built a performant suite of simulation tools from the ground up, and then they expose those tools via API to an "agent" that can compose them to accomplish the user's ask. It's probably less general than the prompt interface implies, but still seems incredibly useful.

Still doesn’t seem possible with current technology? It would have to access those apis while it generates video.

Re: Genesis – a generative physics engine for general-purpose robotics

#50
post #32

Earlier quoted context omitted.

The simulation/rendering is actually pretty fast since it's all done by heavily optimized gpu-based physics and graphics engines. The "generative" part is that they have some LLM stuff that's finetuned for generating configurations/parameters for the physics engine conditioned on some text. Ie, the physics and graphics are classical clockworky simulations, with a generative frontend to make it easier (but less precis…

man i love what u writing even as far of this post ur 2012 post about SVM, i am 2nd year CS student from indonesia. are you a phd on computer science?

Yeah, I finished a PhD in machine learning around 2016 and have been working professionally as a researcher since then, though I'm currently between jobs. It's a fun gig, but the "publish or perish" aspect of academic-ish research roles gets old fast.
Post reply on HN