I had a lot of fun playing the game. I think this game shows us how interesting AI technology will become in the 2020s. It’s open ended but somewhat incoherent right now, but I think we’ll figure out how to update this kind of technology to have an open ended yet internally consistent world by the end of the 2020s.
AI Dungeon 2 costing over $10k/day to run on GCS/Colab
51–60 of 215 posts
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#52Gcs egress is 0.12 per GB, so 3GB per user?
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#53Earlier quoted context omitted.
> Is there no way to have a single hosted instance rather than downloading again for each user? This might make the problem worse; then they'd have to do processing server side, rather than offloading it on the client. I dunno whether this would be more or less expensive than the initial download, but the torrent they put up seems cheaper either way.
Weren’t they already doing the processing server side? If it were client side then it wouldn’t be costing so much to run git clone every time, as the download would be from GitHub to the user’s computer. It would be free, in fact. My impression of the situation is that every user who tried to play would result in a new instance to spin up on Google’s cloud services and then begin downloading a fresh copy of the repo…
A client side request to GitHub would require them to serve up a relevant CORS header, but I do think you're right about me misunderstanding where execution is taking place. I'm unfamiliar with Jupyter Notebooks, and assumed "downloading" meant "to the client". I, too, am now confused about why this is set up like it is. Probably some constraint of Jupyter Notebooks that I'm unaware of.
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#54Why would a user need a 12gb gpu to run the game locally? The deep learning model is already trained, and I can’t imagine one needs a gpu to just evaluate the model.
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#55Why would a user need a 12gb gpu to run the game locally? The deep learning model is already trained, and I can’t imagine one needs a gpu to just evaluate the model.
It's a big model.
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#56Not blaming them, but still a nice example to illustrate the myth that 'the cloud' will let you operate without a sysadmin (someone that knows about setting up and managing infrastructure). Yes, there might not be servers you have to rack and electrical power to be provisioned, but the amount and granularity of the parameters and decisions on the virtual fabric to be decided on and continuously monitored for changes and opportunities that could have an enormous impact on performance and cost is orders of magnitude higher.
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#57Gcs egress is 0.12 per GB, so 3GB per user?
The model weighs about 6.2 GB, you can check it from install.sh [1] that currently downloads it via torrent (it was S3 days ago, I think).
[1] https://github.com/AIDungeon/AIDungeon/blob/master/install.s...
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#58Still not clear why a university research lab would keep this running 'for a few more days' and not pull the plug immediately until either having the costs under control or some sort of reasonal matching benefit in place.
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#59Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…
It doesn't seem to be as simple as downloading a binary. We (those who read HN) could probably download the code and run it at home, but I think the authors want non-technical users to play. To do that, they need an accessible Python runtime, so they're hosting the game in a Colab notebook. The download in question is referring to downloading the weights of the neural net into the VM running the notebook. If only red…
A lot of HN users couldn't run it and have a reasonable experience because they don't own computers with GPUs capable of generating a response in a timely manner.
Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab
#60Still not clear why a university research lab would keep this running 'for a few more days' and not pull the plug immediately until either having the costs under control or some sort of reasonal matching benefit in place.
Probably because they have to spend their grant money some way.