Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Dungeon 2 costing over $10k/day to run on GCS/Colab

twitter.com

Post reply on HN
Print view View original

161–170 of 215 posts

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#161

Post by [deleted] »

koffiezet wrote: post #99
klingonopera wrote: post #7

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

As an "SRE engineer", I don't agree. Certainly in an university/R&D environment, you cannot compete with what 'the cloud' is able to offer within the reach of a few mouse clicks. It would be absolutely moronic to set up an entire infra yourself with GPU's, ... specifically tailored to for 1 research project to then throw it away after a few months, or see the scope change every 2 weeks. One of the reasons such AI tec…

[deleted]

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#162

Post by cobookman »

Shouldn’t this be free egress as traffic is straight to a google product/ colab server or is this being downloaded in the browser?

Also cost was before cdn was enabled. This kind of traffic generally costs fractions of a cent per GiB after signing a contract with negotiation.

“”” Egress to Google products (such as YouTube, Maps, Drive), whether from a VM in Google Cloud with an external IP address or an internal IP address No charge “””

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#163

Post by darkcha0s »

[deleted] wrote: post #158
TomBombadildoze wrote: post #155

Someone, probably: People who don't know how their services work are doomed to spend lots of money on them. From the GCS pricing page: Network egress within Google Cloud applies when you move or copy data from one bucket in Cloud Storage to another or when another Google Cloud service accesses data in your bucket. Within the same location (for example, US-EAST1 to US-EAST1 or EU to EU) -- Free From the original tweet…

[deleted]

Yeah, but your cloud sucks.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#164

Post by logfromblammo »

strenholme wrote: post #82
strenholme wrote: post #51

I had a lot of fun playing the game. I think this game shows us how interesting AI technology will become in the 2020s. It’s open ended but somewhat incoherent right now, but I think we’ll figure out how to update this kind of technology to have an open ended yet internally consistent world by the end of the 2020s.

As it turns out, it’s actually possible to bona fide win the game. In my case, I was dating two girls, one was uncomfortable with the other girl, and broke up with me, so I asked the remaining girl if she would marry me. At this point, she said yes, we rode off in to the sunset and the game proclaimed “CONGRATS YOU WIN” then it saves the game for me. I guess I could load the game and deal with domestic squabbles, hav…

I got a win-game blurb for "> RESPECT WOMEN" in a game that had become really NSFW, really quick, and offensively so, from completely innocuous commands.

Then another, not surprisingly, for "> WIN GAME".

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#165

Post by hombre_fatal »

im3w1l wrote: post #157

You are Faif, a ranger living in the kingdom of Larion. You have a hunting bow and a quiver of arrows. You spot the deer and are ready to finish your hunt when suddenly you hear a voice calling out to you from behind a tree. > nosaving Saving turned off. > shoot the deer You quickly turn around and see a young man with a long beard standing there. He is dressed in rags and his face is covered by a large bandage. His…

It's hit or miss. You'll have better results being more specific in your commands because it gives the model more to work off of and let's you crystalize elements of the scene through reaffirmation.

"Fire an arrow at the deer" or "Check the pockets of the corpse".

I'm often surprised at how well the game can understand me and that it has anything to say in reply at all.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#167

Post by jrockway »

jstanley wrote: post #134
_gfrc wrote: post #124

Earlier quoted context omitted.

No, the computation is not running on the client in the browser. This is the traffic to transfer the model from GCS to Google Colab. This is what makes the price so surprising - you are copying data from one Google Service to another, but it's billed as egress.

But if you were hosting it yourself you wouldn't transfer 6G of data around per user. You'd be a bit more intelligent about it.

You'd save on transferring the bytes around, but now you would have to self-host jupyter and the GPUs it uses. That is going to be even more expensive than IP transit because now you have to have 60,000 12GB GPUs in your datacenter.

Like I said before, this is one of those things that wouldn't exist without the Cloud. If you run things on your user's computers, you have to send them a lot of bits. If you run things on your own computers, you're spared that bandwidth, but now have to have enough "computers" to satisfy your users. It's simply something that's not super cheap to run these days.

I will admit that it is surprising that Google Google traffic is billed at the normal egress rates, but the reasoning does make sense -- a 30Gbps flow is nothing to sneeze at. That is using some tangible resources.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#168

Post by cobookman »

skohan wrote: post #108
skissane wrote: post #68

Earlier quoted context omitted.

One hears stories about researchers with unspent grant money scrambling madly to find something, anything, to spend it on before the grant expires. Use it or lose it. My Dad told me how back in the 1970s he worked at a government-funded research lab. One time they called up a laboratory glassware supplier and said "We'd like to order $10,000 worth of glassware". The supplier asked "Sure, what specifically would you l…

It sounds like there is a missing incentive to go under budget. There's something wrong with the system if people feel like they just have to burn money.

I like how google did employee travel. They calculated a cap = 2/3 * historical employee data + 1/3 * current cost of business flight.

Obviously for common routes such as Sf to la, nyc to sf...etc they hard coded values.

for every $1 under the cap you received 50cent to use on a future travel plan over the calculated cap.

You could fly economy on an international flight saving money, then bank 50% of savings to buy a first class ticket some other time.

This created a negative feedback loop keeping travel costs in check.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#169

Post by buboard »

klingonopera wrote: post #7

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

> What am I missing?

They overbudgeted the "infrastructure" section of their grants and have money to burn; and nobody's interested in setting up another datacenter.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#170

Post by [deleted] »

andrekorol wrote: post #28
cbarrick wrote: post #19

Earlier quoted context omitted.

It doesn't seem to be as simple as downloading a binary. We (those who read HN) could probably download the code and run it at home, but I think the authors want non-technical users to play. To do that, they need an accessible Python runtime, so they're hosting the game in a Colab notebook. The download in question is referring to downloading the weights of the neural net into the VM running the notebook. If only red…

The game's GitHub page[1] states that you would need a "beefy" GPU ~12 GB and CUDA to play the game locally. I think that's why the author was serving the game through Colab since the majority of users probably don't have a 12GB GPU. [1] https://github.com/AIDungeon/AIDungeon/

[deleted]
Post reply on HN