Live data from Hacker News

AI Dungeon 2 costing over $10k/day to run on GCS/Colab

twitter.com

11–20 of 215 posts

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#11
post #4

Well that was obviously forseeable and dumb then. Use the user's resources or charge money.

The unexpected thing was that Google colab and GCS are separate such that transfer costs between them often end up as international external egress fees

Sorry, can you please try to explain that in another way? I don't understand what you mean.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#12

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

My understanding is that each user runs a separate Colab instance with its own hardware, it's not the kind of thing you could replicate easily with one server and a couple GPUs.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#13
I am trying to think some solutions but the crux is that users may be expecting to have a unique story tailored specifically to them. If that assumption is false, then we have some solutions:

- save top or similar stories and make them pre-determined to avoid calling the ai services

- decrease the amount of times the user can keep the story going: users can only give 3 times input instead of X

- charge people for the game

Probably there are more ideas out there.

One last idea: package the code and instruct users to run on their own machine(s) or have then to run on their own GCS account.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#16
post #4

Well that was obviously forseeable and dumb then. Use the user's resources or charge money.

The unexpected thing was that Google colab and GCS are separate such that transfer costs between them often end up as international external egress fees

I don't know anything about your architecture, but you can quickly drop in something like BunnyCDN as a caching proxy for about 4% of price of Google egress. Even if it's a Google->Google transfer, since ingress is free, Google->Caching Proxy->Google should be much better.

You could also consider a webhost like Hetzner, on which you can get a bunch of 1 gbps machines very cheap, though you have to manage them yourself.

You could try Cloudflare, but they may well consider it abuse and cut you off (though they like to pretend on HN that they don't do this).

Finally, I know that Google is handing out substantial credits for game developers and this could very well be up their alley.

If you can optimize the download size a bit and swap to a cheaper delivery method, it should become feasible to run on a hobbyist budget.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#17

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

My understanding is that each user runs a separate Colab instance with its own hardware, it's not the kind of thing you could replicate easily with one server and a couple GPUs.

I see, thanks. That's no longer the case now, is it? I'm presently downloading the torrent, and it seems like it will then run completely local?

Well, I guess that was the cost of getting it to go viral. The torrent seems quite active now.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#18

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

Colab is effectively JupyterLab/JupyterHub with Google's own add-ins (like integration with Google Drive). JupyterHub is a huge PITA to manage, and Colab also offers a degree of free compute and limited access to a dated, but still free K80 GPU.

From the tweet thread, it seems like there was some misunderstanding over where the files are being stored and being executed. This is a pretty common issue with Google Drive. I.e. if someone shares a file with me, and I copy it to a folder, it's just a pointer to the original file. Only after clicking "Add to My Drive" does it count against my storage allocation, and only then is it a distinct copy.

My guess is that the researchers expected each user to be able to run the game in their own personal free Colab environment, not be running it against the university's compute and storage budget.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#19

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

It doesn't seem to be as simple as downloading a binary.

We (those who read HN) could probably download the code and run it at home, but I think the authors want non-technical users to play. To do that, they need an accessible Python runtime, so they're hosting the game in a Colab notebook. The download in question is referring to downloading the weights of the neural net into the VM running the notebook.

If only redistributing Python apps wasn't so difficult.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#20

I am trying to think some solutions but the crux is that users may be expecting to have a unique story tailored specifically to them. If that assumption is false, then we have some solutions: - save top or similar stories and make them pre-determined to avoid calling the ai services - decrease the amount of times the user can keep the story going: users can only give 3 times input instead of X - charge people for the…

i think the game should be made runnable on a user's own machine too.

Make the data licensed AGPL if the author is afraid of people copying and making profit off it without including them!

Post reply on HN