201–210 of 215 posts
Earlier quoted context omitted.
[deleted]
So be more constructive, I'd like to learn more about IBM Cloud as I have no experience with it. Is there a Lambda-like service, pay-by-use 'serverless' database hosting, and support for direct client-to-DB connections with RLS/ACLs?
Earlier quoted context omitted.
[deleted]
I checked a while ago and IBM Cloud costed the same price down to the dollar. IBM bills by the network interface, hundreds of dollars per months to get a dual bonded NIC for private and public transfer. If you use the bandwidth close to capacity all day long for he whole month, it ends up the same cost as you would pay on Amazon for transferring that amount of GB.
So to catch up, AID2 is leveraging a cute offering by Google Colab which will provide any Google account a few hours of free access to a fairly powerful GPU. To use Colab under this arrangement, each person playing the game has to load the model into their personal VM, which is being billed by Google as 6GB of data transfer from GCP into Colab. The obvious questions; 1) Was there a place they could have hosted the mo…
So a server can be completely stateless, i.e. it receives text so far (say, 5 KB), applies GPT-2 generate and returns.
The problem is that GPT-2 generate seems to be very computationally intensive. As I understand, it actually does number crunching with all these 6 GB of data, so it takes 5-10 seconds even on GPU (K80, at least, is that slow).
Is GPU capable of running multiple GPT-2 generate in parallel? No idea.
Assuming that a high-end GPU would be able to produce response in 2 seconds, you can only run maybe 10 concurrent sessions per server if you want fast response time.
Earlier quoted context omitted.
At 10k per day it is absolutely worth it to hire someone to take care of a bunch of cheap servers.
Servers with GPU's are not going to be cheap :)
10k/day makes sense for a short project, but if this is going to be an on-going thing -- like 2-3 years -- then absolutely buy that hardware and have an internal team run it.
Earlier quoted context omitted.
What a cynical input into the conversation. First you assume that this project even existing is some kind of statement about how important it is; a connection I can't make myself unless I try to be extremely cynical. If you actually look at the project website you'll see that the main mirrors are currently down due to high download costs. Then you smugly dismiss the entire project as a GPT-2 weakness highlighter, but…
>> First you assume that this project even existing is some kind of statement about how important it is; a connection I can't make myself unless I try to be extremely cynical. That would be cynical, but I did not make this assumption. I questioned the justification of the high cost to maintain the project, not the existence of the project per se. I did not doubt that people enjoyed the project but, again, I don't und…
Cloud bandwidth costs are a rip off. For this particular situation though, make sure to replicate your files across multiple buckets, one in each GCP zone, otherwise you incur the cross zone transfer costs. We do something similar to serve julia downloads, because it turns out that most downloads are from people running on the cloud (so we basically replicate our binaries to every cloud provider and then to every reg…
Can you describe a little more how you direct downloads to an in-zone copy?
Cloud bandwidth costs are a rip off. For this particular situation though, make sure to replicate your files across multiple buckets, one in each GCP zone, otherwise you incur the cross zone transfer costs. We do something similar to serve julia downloads, because it turns out that most downloads are from people running on the cloud (so we basically replicate our binaries to every cloud provider and then to every reg…
This is good advice for only a particularly narrow use case. If your clients are likely to be on-prem in the same cloud as you, I think most cloud providers will charge you local data transfer fees (e.g. nothing) rather than egress fees, even if the request is made over the public IP address. If your users aren’t mostly in the cloud with you, than this strategy doesn’t help at all.
Sure, but in the situation in the linked tweet 100% of the users where in the same cloud, just in the wrong region. For julia, the distribution is much more mixed, which is why we have the fastly fallback, which works as a traditional CDN. Still caching locally in each of the clouds is useful as people often download a fresh tarball of nightly julia when they CI their packages, so the load from the clouds is quite high.
Not blaming them, but still a nice example to illustrate the myth that 'the cloud' will let you operate without a sysadmin (someone that knows about setting up and managing infrastructure). Yes, there might not be servers you have to rack and electrical power to be provisioned, but the amount and granularity of the parameters and decisions on the virtual fabric to be decided on and continuously monitored for changes…
Also not blaming him/them but isn't this an indicator of lack of cost control at research facilities? Surely someone should of costed this out before making it public? Here in the UK running at 10k/day would of ruined a departments budget for an entire year!
great, "it would have been good except we were too popular" do you then cut off your popular product while it's gaining traction, and thus almost guarantee you kill it in its infancy?
Earlier quoted context omitted.
>> First you assume that this project even existing is some kind of statement about how important it is; a connection I can't make myself unless I try to be extremely cynical. That would be cynical, but I did not make this assumption. I questioned the justification of the high cost to maintain the project, not the existence of the project per se. I did not doubt that people enjoyed the project but, again, I don't und…
Also, can I please ask you to tone down the god-moding if you want to have a disagreement? "passive aggressive faux concern", "bile", what's all that about? Why do you think you have an insight to my state of mind and emotions after reading a comment I made on the internet? This is just frustrating. Is that how you want the internet to be, really? A bunch of people assuming each other are either assholes or idiots, a…
In that spirit, I would very politely suggest that you read your original comment and consider if it represents what you want the Internet to be.