91–100 of 215 posts
Well that was obviously forseeable and dumb then. Use the user's resources or charge money.
I am trying to think some solutions but the crux is that users may be expecting to have a unique story tailored specifically to them. If that assumption is false, then we have some solutions: - save top or similar stories and make them pre-determined to avoid calling the ai services - decrease the amount of times the user can keep the story going: users can only give 3 times input instead of X - charge people for the…
Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…
If agree in theory, but I'd like to disagree with some assumptions. AI devs don't necessarily have experience with efficient network infrastructure. (And ops people with AI development) You can't just "buy servers for the same price". Where are you going to put them? How are you going to power them? How is the bandwidth for them provisioned? At some scale these are non-trivial questions - you can't just buy a rack, s…
Not blaming them, but still a nice example to illustrate the myth that 'the cloud' will let you operate without a sysadmin (someone that knows about setting up and managing infrastructure). Yes, there might not be servers you have to rack and electrical power to be provisioned, but the amount and granularity of the parameters and decisions on the virtual fabric to be decided on and continuously monitored for changes…
Here in the UK running at 10k/day would of ruined a departments budget for an entire year!
Earlier quoted context omitted.
It's a 6GB download. Mid-high end GPUs are 8GB. So what makes it need 12GB? How specifically is that memory used up by various types of decompression, intermediate calculations, etc.
I believe that the 1.5B model weights take up the 6GB themselves. Presumably f32 weight values? Since they are all needed for an evaluation pass they will eat up 6GB on the GPU off the bat. Not too surprising that everything else can't fit in 2GB, since that's going to have to fit the entire model architecture and all intermediary values.
Will this model work with half precision weights? Is it very awkward to use "brain" 16 bit floats?
I realise Cloudflare is a dirty word around here but you could pay $20/mo on their pro plan and serve this file, along with any other static assets, through their CDN. I do this for assets for my own games hobby site. Granted, not getting tens of thousands of downloads a day, but there's nothing in their Ts&Cs to indicate to me that it wouldn't work even if I were.
I realise Cloudflare is a dirty word around here but you could pay $20/mo on their pro plan and serve this file, along with any other static assets, through their CDN. I do this for assets for my own games hobby site. Granted, not getting tens of thousands of downloads a day, but there's nothing in their Ts&Cs to indicate to me that it wouldn't work even if I were.
Why is Cloudflare a dirty word around here? I'm afraid I'm out of the loop
Not blaming them, but still a nice example to illustrate the myth that 'the cloud' will let you operate without a sysadmin (someone that knows about setting up and managing infrastructure). Yes, there might not be servers you have to rack and electrical power to be provisioned, but the amount and granularity of the parameters and decisions on the virtual fabric to be decided on and continuously monitored for changes…
It was already going away, and cloud was not the reason. But with the whole cloud = ditch sysadmins, and the weird way that 'devops' was interpreted to mean a similar thing, a whole lot of baby got thrown out alongside the bathwater.
Virtualization, the maturity of modern networking, compute and storage systems, and the sheer capacity of each atomic piece of hardware means that kind of stuff had already become a once-every-couple-of-years kinda task for most sysadmins, except the few still stuck in some kind of backwards anachronistic environment.
Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept? With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capab…
One of the reasons such AI tech is able to move this fast because the cloud has become a commodity. People start to expect this kind of flexibility. It's impossible to self-host every possible thing some guy might want to try out.
Once you have a stable application - sure you might want to invest in own hosting for this, but even then it's an expensive up-front investment for something that might not go anywhere. I know a few places where they have a bunch of maxed-out nVidia DGX systems catching dust for exactly this reason.
> You are Jesus, a knight living in the kingdom of Larion. You have a steel longsword and a wooden shield. You are on a quest to defeat the evil dragon of Larion. You've heard he lives up at the north of the kingdom. You set on the path to defeat him and walk into a dark forest. As you enter the forest you see a huge black dragon flying above it's lair. The beast looks very angry as you approach.
+ suck his d-ck
> Your screams are silenced when you feel the head of the dragon's ---- entering your mouth. He begins thrusting inside of you and soon you're filled with his ---. You collapse onto the ground and die
Well. That went really more wild than I could have ever expected.