Live data from Hacker News

Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

github.com

51–60 of 62 posts

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#51

Earlier quoted context omitted.

> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). This'll be a niche for a long, long time. Games are generally carefully crafted to deliver a specific mechanical and/or narrative experience. A world populated by LLM/etc bots or content is one choice of what that experience might be, but it's not going to be a very satisfying one f…

for narrative/dialogue, yes, text generation is currently useless. censorship, slop, extreme positivity bias. even jailbroken Opus is shit. but audio generation we already have is pretty much good enough, and this is big. it's not AAA tier yet, sure, but still lightyears better half-assing it with mediocre voice actors. it is now an option to only use real voice actors for a few key characters, and even that won't be…

Same for dubbing.

Voicing a game in English is doable, voicing a game in n languages is only doable if you're the size of EA / Activision / Ubisoft.

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#52
post #7

With how much NVidia is developing AI-workload accelerating hardware, I expect this will cost maybe few dozen dollars and train in few hours within next few years. What I think will be interesting is when commodity hardware can run cheap inference from very capable, specialized models. Pretty sure it will spawn a new golden age of AI-powered desktop applications. For example, video game space has already been trying…

> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). This'll be a niche for a long, long time. Games are generally carefully crafted to deliver a specific mechanical and/or narrative experience. A world populated by LLM/etc bots or content is one choice of what that experience might be, but it's not going to be a very satisfying one f…

Perhaps Llama are good enough now to add flavour NPCs to the world, i.e. NPCs not on your critical path, but maybe on the town square where you can hear them talk about mundane life stuff.

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#53
post #29

Earlier quoted context omitted.

> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). To me this is a downside compared to the NPC generated by humans, since that’s the only reason I would like to read them.

Yeah. I have played a bunch of the roguelike Caves of Qud [1] and it has both hand-written text and procedurally generated text. The former is quite interesting and relevant to both gameplay and plot. The latter is mostly uninteresting and irrelevant, though it does work as "filler." This is similar to how procedurally-generated grass can give a more natural look to a hill than you'd get with tiles (which are incredi…

> I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through.

It's an interesting artistic and technical challenge to investigate, absolutely. I hope people work on it, and I'm sure they already are.

However, let me also offer a counterpoint with something much simpler: coffee.

It is possible to fully automate the process of making an espresso drink. You can buy cheap versions of these machines that will sit on your kitchen counter; the quality of the drinks so produced is not the highest, but I suspect it's entirely possible to build a machine that would actually make high-quality drinks, perhaps with even more precision than a human can.

Yet the opposite trend has prevailed over the last few decades. Not long ago, Americans drank drip coffee (hence the approximation thereof as the Americano). This is an easy process to make in large batches with minimal labor. It is almost fully automatic, without any "automation". And certainly this still exists all over the place. But we now also have coffee shops everywhere -- witness the explosion of Starbucks out of its home in Seattle -- where people, baristas, individually make espresso drinks for customers. This is immensely popular.

Why are tons of people now employed in this way, and why hasn't it been automated? Is it really just a matter of cost?

I don't think so.

I think the knowledge that another person took the trouble to do something for you is in fact part of the product. Consider the practice of drawing designs, like leaves or hearts, in the milk foam. This is entirely unnecessary, from a taste perspective. If you put a lid on it, as is commonly done, you will not even see it. But it carries a message -- that somebody gave enough of a damn to do it.

(We could then analyze how this phenomenon gets watered down and eventually destroys itself when it tries to turn itself into a mass-produced fast-food franchise staffed by underpaid/exploited proletarians, but that would take this post in another direction.)

My point is, I think something similar happens in video games. The very knowledge that another person was involved is important. You are receiving communication from this person. It does something to synchronize, partially, your mind with theirs. And this is a thing we appreciate. If there's no person on the other end, why should we care?

This general phenomenon of paradoxiciality can be a bad thing. It can "feel like" socializing, without producing the actual, real, thick social networks that give us rich lives. There can be an exploitative and even druglike dynamic. It isn't entirely a good class of phenomena.

But in small doses I think it's good, useful, important. I think it's a core part of what makes art valuable. And of course, it applies to games.

This is why, I believe, we will not fully automate the telling of stories, or the making of lattes. The person doing it is important.

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#54
post #48

Earlier quoted context omitted.

As a gamedev, what would you use currently instead for NPCs? And what would you say makes this/those solutions better than LLMs?

Dialogue systems vary by game and are generally custom. But they amount to hard coded if-else branches if you squint. The only one I ever wrote was a simple announcer for a sports-ish shooter. Your question is weird. Every video game ever made has shipped using not LLMs. Not a single commercial game has ever shipped with an LLM. So I’d say what makes classic NPC systems better is they’ve shipped tens of thousands of…

There are multiple games that use LLMs, available in Steam.

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#55
post #29

Earlier quoted context omitted.

Yeah. I have played a bunch of the roguelike Caves of Qud [1] and it has both hand-written text and procedurally generated text. The former is quite interesting and relevant to both gameplay and plot. The latter is mostly uninteresting and irrelevant, though it does work as "filler." This is similar to how procedurally-generated grass can give a more natural look to a hill than you'd get with tiles (which are incredi…

> I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through. It's an interesting artistic and technical challenge to investigate, absolutely. I hope people work on it, and I'm sure they already are. However, let me also offer a counterpoint with something much simpler: coffee. It is possible to fully automate the process of making an espresso dri…

I think the knowledge that another person took the trouble to do something for you is in fact part of the product.

It's an interesting thesis but I don't for a second believe Starbucks sells billions of coffees because people want the barista experience. Starbucks would absolutely automate their business with fancy machines if they could. The problem is that the drinks are so complicated and customizable that no one has built a machine capable of making them all.

Plus there are plenty of people who order Starbucks drinks through an app and never actually meet the barista who made them. If those drinks were made at some fully automated commissary and delivered by drone they'd be just as happy.

The very knowledge that another person was involved is important. You are receiving communication from this person.

That's important for some people, and some games, but not for everyone. I play roguelikes mainly for the challenge. The procedurally generators in these games can create bizarre and very challenging situations no human could ever come up with.

If there's no person on the other end, why should we care?

Because it's a puzzle for your mind to figure out. It's why people play solitaire games (with a deck of cards), random sudokus, tetris, etc.

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#56

Earlier quoted context omitted.

Dialogue systems vary by game and are generally custom. But they amount to hard coded if-else branches if you squint. The only one I ever wrote was a simple announcer for a sports-ish shooter. Your question is weird. Every video game ever made has shipped using not LLMs. Not a single commercial game has ever shipped with an LLM. So I’d say what makes classic NPC systems better is they’ve shipped tens of thousands of…

There are multiple games that use LLMs, available in Steam.

Name them and demonstrate they don’t suck.

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#57

Earlier quoted context omitted.

You don't even necessarily need to have them coming up with valid speech. Simply giving out random quests and rewards would keep people running on a loot treadmill for most open world multiplayer games.

To make this work, you need your LLM-based AI to outperform any other form of generating quests and rewards -- and that performance is measured on things like player enjoyment, game/story progression, exploitability, client system requirements or server operating costs, etc and most of those things are very hard to constrain or optimize for with an LLM right now. While the costs are hidden from end users and are goin…

You're trying to big brain something that isn't that complicated or special.

I never said that every interaction with an NPC must necessarily require AI to create a new quest and reward from scratch. Players aren't that savvy. You just generate a pool of quests and assign them to NPCs in the instance. Players who are addicted to loot treadmills will stay in the game and pay for boosts and other rewards as long as there is content to engage them. This more than pays for the AI service.

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#58

Earlier quoted context omitted.

There are multiple games that use LLMs, available in Steam.

Name them and demonstrate they don’t suck.

First you were talking about those games just existing, now you want me to prove to you that "they don't suck". That doesn't seem fair. Anyway, here you go:

https://store.steampowered.com/app/1519310

https://store.steampowered.com/app/1889620

Those are the two that I could easily remember, there are much more games (already in the 10s) that embed the LLM APIs for things like dialogue generation for some parts of the game, e.g. https://store.steampowered.com/app/2530950/

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#59
post #29

Earlier quoted context omitted.

Yeah. I have played a bunch of the roguelike Caves of Qud [1] and it has both hand-written text and procedurally generated text. The former is quite interesting and relevant to both gameplay and plot. The latter is mostly uninteresting and irrelevant, though it does work as "filler." This is similar to how procedurally-generated grass can give a more natural look to a hill than you'd get with tiles (which are incredi…

> I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through. It's an interesting artistic and technical challenge to investigate, absolutely. I hope people work on it, and I'm sure they already are. However, let me also offer a counterpoint with something much simpler: coffee. It is possible to fully automate the process of making an espresso dri…

> This general phenomenon of paradoxiciality

That was supposed to be "parasociality".

Re: Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c

#60
post #55

Earlier quoted context omitted.

> I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through. It's an interesting artistic and technical challenge to investigate, absolutely. I hope people work on it, and I'm sure they already are. However, let me also offer a counterpoint with something much simpler: coffee. It is possible to fully automate the process of making an espresso dri…

I think the knowledge that another person took the trouble to do something for you is in fact part of the product. It's an interesting thesis but I don't for a second believe Starbucks sells billions of coffees because people want the barista experience. Starbucks would absolutely automate their business with fancy machines if they could. The problem is that the drinks are so complicated and customizable that no one…

> Plus there are plenty of people who order Starbucks drinks through an app and never actually meet the barista who made them.

Yeah, that's a really strong counterargument.

Post reply on HN