AI Dungeon 2 costing over $10k/day to run on GCS/Colab • Hacker News Board

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#211

Post by YeGoblynQueenne » Mon, Dec 09, 2019, 9:49 PM UTC

CathedralBorrow wrote: post #210

YeGoblynQueenne wrote: post #205

Earlier quoted context omitted.

Also, can I please ask you to tone down the god-moding if you want to have a disagreement? "passive aggressive faux concern", "bile", what's all that about? Why do you think you have an insight to my state of mind and emotions after reading a comment I made on the internet? This is just frustrating. Is that how you want the internet to be, really? A bunch of people assuming each other are either assholes or idiots, a…

Agreed. That was much more aggressive than it should have been. I don't think I have an insight into your state of mind, and I'm sorry for using that to land some stupid burn. The point would have come across just as well without my bile. You're right, that's not what I want the Internet to be. In that spirit, I would very politely suggest that you read your original comment and consider if it represents what you wan…

Well, thank you, that's very welcome and I feel bad for making you apologise. I'm glad you see things this way!

So, I'm looking at my comment again and I think I should have omitted the following two sentences:

>> I don't get the feeling it was setup to demonstrate _weakness_ though. I think it was meant to demonstrate strngths.

>> I suppose it's just advertising for the BYU Perception, Control and Cognition Lab, but it sounds awfuly expensive for advertising for an academic group.

The first one does sound as if I'm taking a swipe at the research team. This was not my intention but it came out all wrong.

The second one makes assumptions about the motivation of the BYU research team, and I should have kept those to myself.

My original question, what justifies the high cost of the product, I feel is valid so I wouldn't change it. But, clearly, I took it farther than I should have. I apologise that I didn't think my comment through enough so as to avoid having it come across as an attack on the BYU team and I'm sorry it upset you.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#212

Post by YeGoblynQueenne » Mon, Dec 09, 2019, 10:37 PM UTC

catacombs wrote: post #141

YeGoblynQueenne wrote: post #101

Can I be controversial and ask what justifies such expenses? Why is this game so important that it needs to be up and running so badly? From what I've seen so far the "game" (if you can even call it that) highlights the weaknesses of GPT-2 much more than its strengths (the model's answers to user actions are random, the story is incoherent, the world is inconsistent). I don't get the feeling it was setup to demonstra…

The hype is not rocket science: The game is a lot of fun, and a lot of people in the RPG communities are excited about this.

I agree that it's entertaining and I see the appeal to role-players. It appeals to me, too (well, I'm a gamer). What I don't understand is how come someone's paying $10k a day to keep it running if it's not really returning anything. $10k is not a small sum. It sounds like a heck of a lot to pay to keep a game running just because people find it fun.

Basically, I don't think I've heard of anything like this before. Usually when people put something on the internet for free either it's very cheap for them, or they have a way to recoup the costs, e.g. by serving ads or asking for donations etc. To just throw money at something that doesn't return anything is very uncommon.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#213

Post by aianus » Mon, Dec 09, 2019, 11:18 PM UTC

pm90 wrote: post #45

aianus wrote: post #22

Earlier quoted context omitted.

All of the load balancers on GCP are shared. Maybe you meant to say you get a new, fresh IP address which is true but also not very expensive. "Cloud Load Balancing is a fully distributed, software-defined, managed service for all your traffic. It is not an instance- or device-based solution, so you won’t be locked into physical load balancing infrastructure or face the HA, scale, and management challenges inherent i…

Nothing in the paragraph indicates it’s shared. Also: it might be shared in the implementation but you will still be billed for every single https LB that you use (or an NLB if you’re doing tcp load balancing). Every unique kubernetes ingress resource WILL spin up a NEW, uniquely billed Https LB. Every unique kubernetes service with specific annotations will spin up a NEW, unique LB (internal or external). The author…

You are not billed by the "load balancer", you are billed by the "forwarding rule" which makes it very obvious that the infrastructure is shared and that you will have additional costs with every K8S ingress.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#214

Post by dodobirdlord » Tue, Dec 10, 2019, 8:31 AM UTC

krick wrote: post #126

dodobirdlord wrote: post #79

Earlier quoted context omitted.

Google Colab gives each user a dedicated Nvidia Tesla K80 GPU for 12 hours for free, which is super cool and presumably why the project is on Colab. But as each user spins up their own Colab instance it pulls down the 6GB of GPT-2 model weights, incurring 30-40 cents of data egress charges against the GCP Storage Bucket that the data is stored in. 60k users yesterday * 6GB each -> 360TB of data egress! Normally, a sc…

But you could set up your own file-serving node and then data transfer to end-user Colab instances would be free, right? ("Free" — costing only as much your CDN costs for 360TB/day, which is still quite pricey, I think, but not as much as $10K/day. I.e. Google wouldn't charge you for data transfer here.)

Yup. I think I saw yesterday that they were looking to move the model to BitTorrent.

Re: AI Dungeon 2 costing over $10k/day to run on GCS/Colab

#215

Post by jadbox » Sat, Dec 21, 2019, 6:11 AM UTC

databrecht wrote: post #196

jadbox wrote: post #183

Earlier quoted context omitted.

So be more constructive, I'd like to learn more about IBM Cloud as I have no experience with it. Is there a Lambda-like service, pay-by-use 'serverless' database hosting, and support for direct client-to-DB connections with RLS/ACLs?

Are you looking for something inside IBM cloud? If not, FaunaDB might be interesting? It's a serverless database, pay as you go (with free tier), security layer for client to DB connections, strongly consistent, scalable and distributed (multiple regions). Disclaimer: I work for them.

I looked it up, and I do like it. Thanks for the reference. But, there's one big thing missing and that is the DB needs to be open sourced. Do that and I'll move my projects to your hosted service and promote it everywhere I possibly can.

But, it needs to be open source. Otherwise I'd rather stick with AWS Cognito + Dynamo/AppSync, because primarily that I can depend that it'll be around in five years. Even if I believe FaunaDB cloud will around, I know I won't be able to convince my other stakeholders. But, if it's open source, I can always argue that if FaunaDB-as-a-service disapears, I can host it myself (as a last resort).