Training a ChatGPT-beating model for much less than $85,000is entirely feasible. At CentML, we're actively working on model training and inference optimization without affecting accuracy, which can help reduce costs and make such ambitious projects realistic. By maximizing (>90%) GPU and platform hardware utilization, we aim to bring down the expenses associated with large-scale models, making them more accessible for various applications. Additionally, our solutions also have a positive environmental impact, addressing the excess CO2 concerns. If you're interested in learning more about how we are doing it, please reach out via our website: https://centml.ai
Could you train a ChatGPT-beating model for $85k and run it in a browser?
111–120 of 175 posts
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#112I remember watching one of the final episodes of Connections 3: With James Burke, and he casually said we'd have personal assistants that we could talk to (in our PDAs). That was 1997 and I knew enough about computers to think he was being overly optimistic about the speed of progress. Not in our lifetimes. Guess I was wrong!
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#113Earlier quoted context omitted.
The ones that can't use the GPT4 for whatever reason. Maybe you are a company and you don't want to send OpenAI your prompts. Or a person who has very private prompts and feel sketchy about sending them over. Or maybe you are an individual who has a use case that's too edgy for OpenAI or a silicon valley corporate image. When Replika shut down people trying to have virtual boyfriend/girlfriends on their platform, the…
Those are seriously niche use cases. They exist but can they fund gpt5 level development?
Most corporations/governments would prefer to keep their AI conversations private. Definitely mainstream desire, not niche.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#114> My friends at Replicate told me that a simple rule of thumb for A100 cloud costs is $1/hour. AWS charges $32/hr for an 8xA100s (p4d.24xlarge) which comes out to $4/hour/gpu. Yes you can get lower pricing with a 3 year reservation but thats not what this question is asking. You also need 256 nodes to be colocated on the same fabric -- which AWS will do for you but only if you reserve for years.
model-depending, you can train on lesser (cheaper) GPUs but system-level optimizations are needed. Which is what we provide at centml.ai
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#115I wonder why anyone would want to run it in a browser, other than to show it could be done? It's not like the extra latency would matter, since these things are slow.
Running it on a server you control makes more sense. You can pick appropriate hardware for running the AI. Then access it from any browser you like, including from your phone, and switch devices whenever you like. It won't use up all the CPU/GPU on a portable device and run down your battery.
If you want to run the server at home, maybe use something like Tailscale?
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#116Earlier quoted context omitted.
it's hard to tell from what you're saying: you're planning on putting an ML infrastructure training server on a regular 15A circuit, not in a data center or machine room? And power is paid for by somebody else? My thinking about pricing doesn't include that option because I wouldn't just hook a server like that up to a regular outlet in an office and use it for production work. If that works for you- you can happily…
Nvidia happily sells what you're describing. They call it "DGX Station A100", it has 4 80GB A100 and retails for 80k. Not sure i believe their claimed noise level of Of course that's still a very small system when talking LLM training, the only reason why i would not put that in a regular office is it's extreme price. Do you really want something worth 80k in a form factor that could be casually carried through the d…
If you live near an inexpensive datacenter, you can park it there. Throw in a storage machine or two (TrueNAS MINI R looks like a credible low-effort option). If your workload is to run a year long computation on it and otherwise mostly ignore it, then your operational costs will be quite low.
Most people who rent cloud servers are not doing this type of workload.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#117Earlier quoted context omitted.
Those are seriously niche use cases. They exist but can they fund gpt5 level development?
Most corporations/governments would prefer to keep their AI conversations private. Definitely mainstream desire, not niche.
Who does your government and corporate email? In the UK it's all either Gmail (for government) and Outlook (NHS). For compliance reasons they simply want data center certification and location restrictions.
If you think a small corp is going to get a big gov contract outside of a nepo-state you're in for a shock.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#118Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#119Earlier quoted context omitted.
10yrs from now models will be in the OS. Maybe even in silicon. No downloads required.
Not in mine. I don't even want redhat's bullshit in there. I'm not installing some black box into my OS that was programmed with motives that can't be extracted from the model at rest.
iOS already has this to a degree, for a couple of years.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#120It seems the quality goes up & cost goes down significantly with Colossal AI's recent push: https://medium.com/@yangyou_berkeley/colossalchat-an-open-so... Their writeup makes it sounds like, net, 2X+ over Alpaca, and that's an early run The browser side is interesting too. Browser JS VMs have a memory cap of 1GB, so that may ultimately be the bottleneck here...
Interesting, since I looked last year, Chrome has started raising the caps internally on buffer allocation to potentially 16GB: https://chromium.googlesource.com/chromium/src/+/2bf3e35d7a4...
Last time I tried on a few engines, it was just 1-2GB for typed arrays, which are essentially the backing structure for this kind of work. Be interesting to try again..
For our product, we actually want to dump 10GB+ on to the WebGL side, which may or may not get mirrored on the CPU side. Not sure if additional limits there on the software side. And after that, consumer devices often have another 10GB+ CPU RAM free, which we'd also like to use for our more limited non-GPU stuff :)