what for world models be equivalent of ChatGPT for LLM to really blow up in utility?
text to roblox maybe?
Genie 2: A large-scale foundation world model
261–270 of 436 posts
Re: Genie 2: A large-scale foundation world model
#262Earlier quoted context omitted.
> the squealing carcass called Gemini Have you used Gemini? It seems every bit as good as ChatGPT.
I have, and it's terrible in exactly the way GP describes it. It won't talk to me about anything involving the word "president" or anything related to the US political system, even very procedural/hopefully uncontroversial questions such as "who appoints , and is the appointment confirmed in congress or not". That's only one example; it generally refuses so many things (and often even lies about "not being able to",…
Yeah that's somewhat of a special case - the Gemini API even has a specific CIVIC_INTEGRITY flag in its safety filters: https://ai.google.dev/gemini-api/docs/safety-settings. They literally put "election-related queries" on the same table column as "sexual acts" or "hate speech".
It's not exactly explained how answering who the current president is would be considered harmful to civic integrity, but it is something very specifically filtered out and not really the result of the general RLHF lobotomy.
Re: Genie 2: A large-scale foundation world model
#263I don't understand this space very well, but this seems incredible. Something I find interesting about generative AI is how it adds a huge layer of flexibility, but at the cost of lots of computation, while a very narrow set of constraints (a traditional program) is comparatively incredibly efficient. If someone spent a ton of time building out something simple in Unity, they could get the same thing running with a s…
> Will traditional programming be used for most "production" workloads with gen AI being used to aid in the prototyping and development of those traditional programs
I mean we're already there with Copilot, Cursor and other tools that use LLMs to assist in coding tasks.
Re: Genie 2: A large-scale foundation world model
#264Earlier quoted context omitted.
we have reliable infinite story generation in PvP multiplayer. if the matchup is fair, every game can be different and exciting. see chess
is PvP multiplayer considered a "story"? Is a football game a "story"? I guess if all you consider for story is "things happen", then a PvP match can be a story, but that's stretching what I would consider "story" for a game. That is the story of the match, but it's not in and of itself a plot story.
I agree, the parent would've been much better suited with the example of PVE/PVP Roleplaying. People make up stories all the time
Re: Genie 2: A large-scale foundation world model
#265Earlier quoted context omitted.
This is really not the avenue for house-hold robots. Interacting with the actual physical world is very different from creating a video game.
> Interacting with the actual physical world is very different from creating a video game The major difference being the former scales very poorly for generating training data compared to the latter. Genie 2 is not even a video game and has worse fidelity that video games, the upside is it probably scales even better than video games for generating training scenarios. If you want androids in teal life, Genie 2 (or si…
How does turning an image into a game help with robots? Robots don't need to guess what they can't see, they would have sensors to tell them exactly what is there (like a self driving car).
Re: Genie 2: A large-scale foundation world model
#266Forget video games. This is a huge step forward for AGI and Robotics. There's a lot of evidence from Neurobiology that we must be running something like this in our brains--things like optical illusions, the editing out of our visual blind spot, the relatively low bandwidth measured in neural signals from our senses to our brain, hallucinations, our ability to visualize 3d shapes, to dream. This is the start of addin…
This is akin to navigating a lucid dream, nothing more. Conscious inputs to a visual stream synthesized from long term memory.
> nothing more.
Consider the use where you seed the first frame from a real world picture, with a prompt that gives it a goal. Not only can you see what might happen, with different approaches, and then pick one, but you can re-seed with real world baselines periodically as you're actually executing that action to correct for anything that changes. This is a great step for real world agency.
As a person without aphantasia, this is how I do anything mechanical. I picture what will happen, try a few things visually in my head, decide which to do, and then do it for real. This "lucid dream" that I call my imagination is all based on long term memory that made my world view. I find it incredibly valuable. I very much rely on it for my day job, and try to exercise it as much as possible, before, say, going to a whiteboard.
Re: Genie 2: A large-scale foundation world model
#267While this is very (very) cool, what is the upside to having a model render everything at runtime, vs. having it render the 3D assets during development (or even JIT), and then rendering it as just another game? I can think of many reasons why the latter is preferable.
Re: Genie 2: A large-scale foundation world model
#268OpenAI launches Sora (quite a while ago now), Google needs to fire back with something else groundbreaking. I love the advancement of the tech but this still looks very young and I'd be curious what the underlying output code looks like (how well it's formatted, documented, organized, optimized, etc.) Also, this seems oddly related to the recent post from WorldLabs https://www.worldlabs.ai/blog . Wonder if this was t…
I also find the timing vs World Labs demo disturbing.
What's disturbing? In all likelihood the close timing was world labs rushing to get their demo out the door knowing this was coming because they wouldn't get nearly the hype they did if this came before.
Re: Genie 2: A large-scale foundation world model
#269Related recent project you can try out yourself (Chrome only) which hallucinates new frames of a Minecraft style game: https://oasis.decart.ai/ That one would reimagine the world any time you look at the sky or ground. Sounds like Genie2 solves that: "Genie 2 is capable of remembering parts of the world that are no longer in view and then rendering them accurately when they become observable again."
This blows Decart's Oasis (which raised $25 million at $500 million valuation) and World Labs (which raised $230 million in complete stealth) out of the water. Google is firing warning shots to kill off interest in funding competing startups in this space. I suspect that in 6 months it won't matter as we'll have completely open source Chinese world models. They're already starting to kill video foundation model compa…
The game has indeed become brutal for foundational model companies.
I am less worried for AI research+product companies: they have likely secured revenue streams with real customers and built domain knowledge in the meantime.
Re: Genie 2: A large-scale foundation world model
#270While this is very (very) cool, what is the upside to having a model render everything at runtime, vs. having it render the 3D assets during development (or even JIT), and then rendering it as just another game? I can think of many reasons why the latter is preferable.
To me, keeping a world state in sync with rapidly changing external state is the most compelling application. Something like dockercraft: https://github.com/docker/dockercraft