Ask HN: What ML platform are you using?

I am very much a beginner in the space of machine learning and have been overwhelmed by the choices available. Eventually I do want to simply want to build my own rig and just train models on that, but I don't have that kind of money right now, nor is it easy to find GPUs even if could afford them.

So I am basically stuck to cloud solutions for now, which is why I want to hear personal experiences of HN folks who have used any of the available ML platforms. Their benefits, short comings, which are more beginner friendly, cost effective, etc

I am also not opposed to configuring environments myself rather than using managed solutions (such as Gradient) if it is more cost effective to do so, or affords better reliability // better than average resource availability... because I read some complaints that Colab has poor GPU availability since shared among subscribers, and that the more you use it the less time is allocated to you... not sure how big of a problem it actually is though.

I am very motivated to delve into this space (it's been on my mind a while) and I want to do it right, which is why I am asking for personal experiences on this forum given that there is a very healthy mix of technology hobbyists as well as professionals on HN, of which the opinion of both is equally valuable to me for different reasons.

Also please feel free to include any unsolicited advice such as learning resources, anecdotes, etc,

Thanks for reading until the end.

vast.ai has pretty low prices and gives you remote ssh into a GPU instance that you then have root on (albeit containerized).

Having a local GPU is effectively a requirement for doing "development" work (e.g. getting an architecture and/or codebase to the point where you would even be able to start training). Unfortunately, getting your own GPU is just absurdly expensive these days and probably not worth it. In the meantime, colab/kaggle/paperspace can be _okay_ as dev environments. Unfortunately, renting compute on vast.ai all day just to do occasional dev work gets expensive pretty quickly.

For something in-between vast.ai and AWS, datacrunch.io has slightly higher prices, with remote SSH into a server and a few more "niceties" that you get with a traditional cloud such as CPU instances and the ability to use those to pre-load data onto disk.

If and when you are able to get a GPU - just make sure to get nvidia as they have a stranglehold over the industry. The RTX cards are great - I've been doing tons of multimodal work on an RTX 2070 I bought pre-pandemic for around 350$. It only has 8 GiB of vram but is actually quite similar to a server-style V100 otherwise. I assume it probably costs like 2000$ these days.

If you're interested in the realm of running inference/training on giant models (say GPT-J 20B), you may find yourself in lack of VRAM. Using libraries like deepspeed, you can split the work across multiple GPU's. I highly recommend investing time to learning multi-GPU libraries or framework-provided features like pytorch's distributed data parallel as the size of models becomes a limiting factor very quickly in the case of transformers. A sibling comment mentions that you will need institutional support for training such models. This may be true, unfortunately. All I will say is that if you are even mildly competent, the demand for that type of work is increasing a lot lately.

Oh and yes, there is a new-ish site called replicate that I have been using to allow people to run inference on models that I've trained https://replicate.com/ without needing to be a coder. A lot of people use colab for this but that platform is annoying to support in practice.