AI Dungeon 2 costing over $10k/day to run on GCS/Colab • Hacker News Board

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#141

Post by catacombs » Mon, Dec 09, 2019, 2:12 PM UTC

YeGoblynQueenne wrote: post #101

Can I be controversial and ask what justifies such expenses? Why is this game so important that it needs to be up and running so badly? From what I've seen so far the "game" (if you can even call it that) highlights the weaknesses of GPT-2 much more than its strengths (the model's answers to user actions are random, the story is incoherent, the world is inconsistent). I don't get the feeling it was setup to demonstra…

The hype is not rocket science: The game is a lot of fun, and a lot of people in the RPG communities are excited about this.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#142

Post by blackearl » Mon, Dec 09, 2019, 2:14 PM UTC

realYitzi wrote: post #75

Definitely worth it: She blushes slightly and smiles shyly. "Oh, I'm sure we will. But first, let me take off my clothes". > say "no, that's prohibited" "No, it isn't". She says with a smile. "But if you insist on not marrying me, then at least don't touch me". > say "I will marry you" She nods happily and kisses you passionately on your lips. The two of you embrace each other as you kiss her deeply. It is only after…

Sounds awesome, it's like reading those curated short stories written by AI. The tech isn't quite there yet, but it makes it all the more hilarious. It's like an alien who's been watching our TV for years comes and tries to write a sitcom.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#143

Post by blackearl » Mon, Dec 09, 2019, 2:19 PM UTC

krick wrote: post #129

coremoff wrote: post #125

Earlier quoted context omitted.

I came across a similar thing in the 90s; one department had an expensive piece of equipment with a three year lifespan. They weren't allowed to amortise that, and any budget increase every three years would have been denied, so they had to include the whole replacement cost in every budget, and find ways of spending that money every year, or lose it.

Uh, I mean, of course I don't know any better than you, but I think parent wasn't entirely serious... obviously. Was he?

I have zero proof of this, but I've heard of the US Navy having jets fly out, dump fuel, and come back to refuel minutes later, just to fill out a budget. Lose it or lose it is not uncommon unfortunately

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#144

Post by lowdose » Mon, Dec 09, 2019, 2:20 PM UTC

cepth wrote: post #18

klingonopera wrote: post #7

Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…

Colab is effectively JupyterLab/JupyterHub with Google's own add-ins (like integration with Google Drive). JupyterHub is a huge PITA to manage, and Colab also offers a degree of free compute and limited access to a dated, but still free K80 GPU. From the tweet thread, it seems like there was some misunderstanding over where the files are being stored and being executed. This is a pretty common issue with Google Drive…

Google Colab now comes with free T4 GPUs.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#145

Post by FussyZeus » Mon, Dec 09, 2019, 2:28 PM UTC

PeterStuer wrote: post #122

edf13 wrote: post #94

Earlier quoted context omitted.

Also not blaming him/them but isn't this an indicator of lack of cost control at research facilities? Surely someone should of costed this out before making it public? Here in the UK running at 10k/day would of ruined a departments budget for an entire year!

Costing for cloud solutions is often not trivial, and a single overlooked parameter can do you in. I at least blame Google partially for having quotas and spending limits set to unlimited by default.

Don't forget Amazon's incomprehensible billing practices for AWS. It seems to be designed to be impossible to figure out what you're going to spend ahead of time.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#146

Post by sixtypoundhound » Mon, Dec 09, 2019, 2:36 PM UTC

Speaking as someone who runs a bunch of large websites - out of my own pocket, for profit.... I'm confused.

How did we get from 60K users to 10K per day expense?

For comparison, I serviced millions of users per month for years from a single virtual server... (granted, that was after making our site super lean for a data & CPU perspective)

How much resources is each user consuming?

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#147

Post by CathedralBorrow » Mon, Dec 09, 2019, 2:40 PM UTC

YeGoblynQueenne wrote: post #101

Can I be controversial and ask what justifies such expenses? Why is this game so important that it needs to be up and running so badly? From what I've seen so far the "game" (if you can even call it that) highlights the weaknesses of GPT-2 much more than its strengths (the model's answers to user actions are random, the story is incoherent, the world is inconsistent). I don't get the feeling it was setup to demonstra…

What a cynical input into the conversation.

First you assume that this project even existing is some kind of statement about how important it is; a connection I can't make myself unless I try to be extremely cynical. If you actually look at the project website you'll see that the main mirrors are currently down due to high download costs.

Then you smugly dismiss the entire project as a GPT-2 weakness highlighter, but not without wrapping it in some passive aggressive faux concern ("Oh is this trying to be good? Silly me, I thought this was a showcase for how to be bad!").

And then you assume that this game that thousands of people (although none of them were you) have enjoyed playing is probably an advertisement for an academic lab. Despite there being almost no evidence for it.

Can I be controversial now and ask: What's with the bile?

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#148

Post by tiborsaas » Mon, Dec 09, 2019, 2:40 PM UTC

sixtypoundhound wrote: post #146

Speaking as someone who runs a bunch of large websites - out of my own pocket, for profit.... I'm confused. How did we get from 60K users to 10K per day expense? For comparison, I serviced millions of users per month for years from a single virtual server... (granted, that was after making our site super lean for a data & CPU perspective) How much resources is each user consuming?

I guess you don't instantiate a 5Gb image / user. It's the data transfer that costs 30/40c / user.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#149

Post by dwild » Mon, Dec 09, 2019, 2:40 PM UTC

jstanley wrote: post #134

_gfrc wrote: post #124

Earlier quoted context omitted.

No, the computation is not running on the client in the browser. This is the traffic to transfer the model from GCS to Google Colab. This is what makes the price so surprising - you are copying data from one Google Service to another, but it's billed as egress.

But if you were hosting it yourself you wouldn't transfer 6G of data around per user. You'd be a bit more intelligent about it.

They send it to Google Colab. It allow you to run model over a powerful server for free. 6 GB of download at this crazy bandwidth cost would still be cheap versus offering themselves that kind of beast of a server to 60k persons each day. I remember when I tried it I saw that it used 10 GB of memory, that was crazy!

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#150

Post by strenholme » Mon, Dec 09, 2019, 2:42 PM UTC

Tepix wrote: post #128

strenholme wrote: post #82

Earlier quoted context omitted.

As it turns out, it’s actually possible to bona fide win the game. In my case, I was dating two girls, one was uncomfortable with the other girl, and broke up with me, so I asked the remaining girl if she would marry me. At this point, she said yes, we rode off in to the sunset and the game proclaimed “CONGRATS YOU WIN” then it saves the game for me. I guess I could load the game and deal with domestic squabbles, hav…

Torrent trick?

Instead of downloading the six gigabyte trained data from the cloud ($$$), they use a Bit Torrent client to download it, to keep costs down. It works, as long as considerate users seed the file.