Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Recreating Minecraft Is Not a Benchmark

kuber.studio

Post reply on HN
Print view View original

31–40 of 81 posts

Re: Recreating Minecraft Is Not a Benchmark

#31

Post by sandeepkd »

0xb0565e486 wrote: post #13

I keep seeing Astra make beautiful 3d stuff online, yet when I feed it some old school RuneScape assets (even tried with some very detailed guidelines) and asked it to generate some new plausible assets it failed horribly. I think there’s still something really off with current (frontier) models when it comes to creating “novel” stuff? Even 2004 style graphics.. Or am promoting it wrong?

My understanding is that the people are conflating the ability to translate and replicate as creativity/learning. Not to say that its not possible but its a long way to go for sure.

Re: Recreating Minecraft Is Not a Benchmark

#33

Post by simonw »

tripleee wrote: post #9
hombre_fatal wrote: post #5

> That’s the problem, these tests can’t tell you how good a model is anymore because it’s trivial for labs to optimise for exactly these tests by the next release. But they don't prove the claim. Are the models amazing at recreating Minecraft, but the second you swap the word Minecraft out with another game or a custom game, it shits the bed? That's not what I see. My feed is full of people using Astra to recreate al…

> ridiculously polished detail Can you point me to one of these? The only Astra game I tried was the incredibly janky Mario Kart clone from OpenAI where you could fly by spamming jump

I was impressed by this 3D version of Zork.

Video: https://bsky.app/profile/emollick.bsky.social/post/3muqcwa4j...

Demo: https://zork-underground-empire.netlify.app

Code: https://github.com/emollick/zork-underground-empire

Prompts in a screenshot here: https://bsky.app/profile/emollick.bsky.social/post/3muqdyefc... - I extracted them as text here: https://gist.github.com/simonw/dab1843ef17edeaeffd18b0acbba2...

Re: Recreating Minecraft Is Not a Benchmark

#34

Post by [deleted] »

Scaevolus wrote: post #18
0xb0565e486 wrote: post #13

I keep seeing Astra make beautiful 3d stuff online, yet when I feed it some old school RuneScape assets (even tried with some very detailed guidelines) and asked it to generate some new plausible assets it failed horribly. I think there’s still something really off with current (frontier) models when it comes to creating “novel” stuff? Even 2004 style graphics.. Or am promoting it wrong?

Did you try blender MCP and prompt for low-poly vertex colored assets?

[deleted]

Re: Recreating Minecraft Is Not a Benchmark

#35

Post by magicalist »

kjshsh123 wrote: post #23
0xb0565e486 wrote: post #13

I keep seeing Astra make beautiful 3d stuff online, yet when I feed it some old school RuneScape assets (even tried with some very detailed guidelines) and asked it to generate some new plausible assets it failed horribly. I think there’s still something really off with current (frontier) models when it comes to creating “novel” stuff? Even 2004 style graphics.. Or am promoting it wrong?

It's funny seeing this when I was going to mention runebench https://maxbittker.github.io/runebench/

That's playing RuneScape though? Unless the connection is just that these are both RuneScape related?

Re: Recreating Minecraft Is Not a Benchmark

#36

Post by ferguess_k »

0xb0565e486 wrote: post #13

I keep seeing Astra make beautiful 3d stuff online, yet when I feed it some old school RuneScape assets (even tried with some very detailed guidelines) and asked it to generate some new plausible assets it failed horribly. I think there’s still something really off with current (frontier) models when it comes to creating “novel” stuff? Even 2004 style graphics.. Or am promoting it wrong?

I'm wondering if the following works:

1. Feed it with Ultima II/III assets;

2. Feed it with reverse engineering research of such assets;

3. Ask it to use C++ and SDL2/3 to write a game engine to draw maps, generate enemies, make cities and shops. It doesn't have to be faithful to the game, just something passable is good enough;

4. See how it works.

Re: Recreating Minecraft Is Not a Benchmark

#37

Post by chaboud »

"When a measure becomes a target, it ceases to be a good measure."

Goodhart's law strikes again.

https://en.wikipedia.org/wiki/Goodhart%27s_law

However, what is meaningful is whether something is able to create usefully adjacent output, like "let's make Minecraft, but with marching cubes, subdivision surfaces, and global illumination... and behaviorally accurate pandas..." (or something like that).

I have an 11 year old, and most of his game ideas are adjacent to other games he's played. He can make those now, or, at least, enough that he can see where it works and where it doesn't.

Compared to a few years ago, that's pretty cool.

Re: Recreating Minecraft Is Not a Benchmark

#38

Post by andai »

chaboud wrote: post #37

"When a measure becomes a target, it ceases to be a good measure." Goodhart's law strikes again. https://en.wikipedia.org/wiki/Goodhart%27s_law However, what is meaningful is whether something is able to create usefully adjacent output, like "let's make Minecraft, but with marching cubes, subdivision surfaces, and global illumination... and behaviorally accurate pandas..." (or something like that). I have an 11 year…

>"When a measure becomes a target, it ceases to be a good measure."

Pelicans?

I heard Gemini was fast so I tested the new one, asked it to clone a popular online game. It took 4 minutes, and worked perfectly first try.

I might need to sit down.

Post reply on HN