Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Dungeon 2 costing over $10k/day to run on GCS/Colab

twitter.com

Post reply on HN
Print view View original

21–30 of 215 posts

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#21

Post by ljm »

nickwalton00 wrote: post #6
55555 wrote: post #4

Well that was obviously forseeable and dumb then. Use the user's resources or charge money.

The unexpected thing was that Google colab and GCS are separate such that transfer costs between them often end up as international external egress fees

This is just GCS. For example, if you use their managed Kubernetes service, you will get a fresh load balancer for every service you expose to the internet. Not a shared load-balancer, a new one.

Unless you set up an alternative you'll get absolutely rinsed through the cost of the instance and then the egress charges on top.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#22

Post by aianus »

ljm wrote: post #21
nickwalton00 wrote: post #6

Earlier quoted context omitted.

The unexpected thing was that Google colab and GCS are separate such that transfer costs between them often end up as international external egress fees

This is just GCS. For example, if you use their managed Kubernetes service, you will get a fresh load balancer for every service you expose to the internet. Not a shared load-balancer, a new one. Unless you set up an alternative you'll get absolutely rinsed through the cost of the instance and then the egress charges on top.

All of the load balancers on GCP are shared. Maybe you meant to say you get a new, fresh IP address which is true but also not very expensive.

"Cloud Load Balancing is a fully distributed, software-defined, managed service for all your traffic. It is not an instance- or device-based solution, so you won’t be locked into physical load balancing infrastructure or face the HA, scale, and management challenges inherent in instance-based LBs."

https://cloud.google.com/load-balancing/

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#24

Post by Wowfunhappy »

cbarrick wrote: post #19
klingonopera wrote: post #7

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

It doesn't seem to be as simple as downloading a binary. We (those who read HN) could probably download the code and run it at home, but I think the authors want non-technical users to play. To do that, they need an accessible Python runtime, so they're hosting the game in a Colab notebook. The download in question is referring to downloading the weights of the neural net into the VM running the notebook. If only red…

Could something like PyInstaller[1] work?

[1]https://www.pyinstaller.org

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#25

Post by sneak »

chii wrote: post #20
designium wrote: post #13

I am trying to think some solutions but the crux is that users may be expecting to have a unique story tailored specifically to them. If that assumption is false, then we have some solutions: - save top or similar stories and make them pre-determined to avoid calling the ai services - decrease the amount of times the user can keep the story going: users can only give 3 times input instead of X - charge people for the…

i think the game should be made runnable on a user's own machine too. Make the data licensed AGPL if the author is afraid of people copying and making profit off it without including them!

AGPL does not prevent someone from copying data and making a profit off it and not including them.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#26

Post by gtm1260 »

designium wrote: post #13

I am trying to think some solutions but the crux is that users may be expecting to have a unique story tailored specifically to them. If that assumption is false, then we have some solutions: - save top or similar stories and make them pre-determined to avoid calling the ai services - decrease the amount of times the user can keep the story going: users can only give 3 times input instead of X - charge people for the…

Part of the reason this idea was pretty viral/successful is because anyone could try it out right away, without the usual nonsense that comes with running open source code locally.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#27

Post by chongli »

blotter_paper wrote: post #5
baroffoos wrote: post #2

Insane but also expected. When I tried it out when it was posted here and saw it took multiple minutes to warm up I knew it was probably expensive. >And it's currently costing 30-40 cents per download. Is there no way to have a single hosted instance rather than downloading again for each user?

> Is there no way to have a single hosted instance rather than downloading again for each user? This might make the problem worse; then they'd have to do processing server side, rather than offloading it on the client. I dunno whether this would be more or less expensive than the initial download, but the torrent they put up seems cheaper either way.

Weren’t they already doing the processing server side? If it were client side then it wouldn’t be costing so much to run git clone every time, as the download would be from GitHub to the user’s computer. It would be free, in fact.

My impression of the situation is that every user who tried to play would result in a new instance to spin up on Google’s cloud services and then begin downloading a fresh copy of the repo from GitHub. This is what cost so much in bandwidth.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#28

Post by andrekorol »

cbarrick wrote: post #19
klingonopera wrote: post #7

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

It doesn't seem to be as simple as downloading a binary. We (those who read HN) could probably download the code and run it at home, but I think the authors want non-technical users to play. To do that, they need an accessible Python runtime, so they're hosting the game in a Colab notebook. The download in question is referring to downloading the weights of the neural net into the VM running the notebook. If only red…

The game's GitHub page[1] states that you would need a "beefy" GPU ~12 GB and CUDA to play the game locally.

I think that's why the author was serving the game through Colab since the majority of users probably don't have a 12GB GPU.

[1]https://github.com/AIDungeon/AIDungeon/

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#29

Post by tedunangst »

To play backseat problem solver... You have free ingress? So start up a few $5 DO droplets and serve files from there. That gives you 1TB transfer per month.

I haven't tried this, but my understanding is that's per droplet. So when drop A is about exhausted, start B and switch over the traffic. Then shut down A. Then start C and shut down B, etc. Unlimited transfer? (Until your account gets banned, anyway.)

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#30

Post by brianwawok »

tedunangst wrote: post #29

To play backseat problem solver... You have free ingress? So start up a few $5 DO droplets and serve files from there. That gives you 1TB transfer per month. I haven't tried this, but my understanding is that's per droplet. So when drop A is about exhausted, start B and switch over the traffic. Then shut down A. Then start C and shut down B, etc. Unlimited transfer? (Until your account gets banned, anyway.)

You can buy bandwidth from a simple CDN for cheap enough to not need to do this dance.
Post reply on HN