Live data from Hacker News

The State of Machine Learning Frameworks

thegradient.pub

191–200 of 201 posts

Re: The State of Machine Learning Frameworks

#191
post #153

Earlier quoted context omitted.

More anecdata: we consistently outperform lightgbm, xgboost, random forests, linear models, etc. using neural networks even on smaller datasets. This applies whether we implemented the other algorithms ourselves or simply compared to someone else’s results with them. In my experience it really comes down to how many “tricks” you know for each algorithm and how well can you apply and combine these “tricks”. The differ…

This was a really interesting and insightful comment, thanks for sharing. I think the conclusion I shared in my sibling comment was probably a little too broad. I particularly like this: > In my experience it really comes down to how many “tricks” you know for each algorithm and how well can you apply and combine these “tricks”. The difference is that neural networks have many more of these tricks and a broader cover…

I think your conclusions are accurate. For many problems LightGBM or xgboost can often yield decent results in short amounts of time and for many problems that’s sufficient. A lot of the work we do is about pushing the results as far as we can take them and the business case justifies the extra time it can take to get there. For those types of problems, today, we would probably choose a neural network because then we have a lot more knobs as you mentioned.

Just like the rest of ML, whether neural networks are the right choice still depends on the problem at hand and the team implementing the solution. It definitely impacts where the performance / time curves intersect. If we just need something decent fast, or we’re working with another team that doesn’t have the same background, we tend to focus on approaches with fewer moving pieces. If we need the best possible performance, have a qualified team to get there, and have the time to iterate on development then the curves would favor neural networks.

Re: The State of Machine Learning Frameworks

#192

We are considering to move to PyTorch, we really dislike how the Tensorflow 1.x -> 2.0 transition is handled. For years a lot of stuff has been added to tf.contrib , some things were only in tf.contrib and now that it's dropped in TF a lot of project (including ours) have to do quite large rewrites. Since the last few 1.x iterations, Tensorflow has been complaining that the older RNN layers are deprecated and that we…

It's been pretty good with PyTorch. The API has been fairly stable and I've adopted code developed from 0.4.0 to 1.0.0+ with barely the need for any tweaks. Granted, it's a younger project so for now things are stable but maybe 3 years from now they may have some giant API refresh. But I find their API quite nice for the most part so I don't see them needing to switch everything up periodically.

Wait, pytorch is torch at its core, right? That is almost 10 years old and the last rewrite was version 7.

Re: The State of Machine Learning Frameworks

#193
post #170

Earlier quoted context omitted.

What challenges are you worried about with transferring PyTorch to production? It’s been wonderful to work with, but I haven’t put a PyTorch model in high volume production yet, so I’m curious too.

1) My laziness to look for what’s available to do this. 2) Core belief to NOT use any product backed by FB. But anyway, at this point I got so many things already running over TF + Keras that I don’t see any use case of reverting back the entire code base written over 3-4 years to other platform just because new grads from university are using some library more over other. I got everything I need, so why to suffer un…

Yeah, I guess if TF works for you, stick with it. I started learning with Theano+Keras, then with TF, and finally PyTorch, and was much happier when I switched to PyTorch, FWIW. I think it’s worth trying if you haven’t.

Re: The State of Machine Learning Frameworks

#194

I work at a small company as an engineer and recently was asked to do a project that would require some neural net magic. I had some experience with keras/tensorflow so that was my first choice. Despite the absolute nightmare of getting it installed and running on a gpu, I managed it and had a fantastic model. It was doing so well that the company wanted to expand the project and build out a multi-gpu rig as part of…

Could you provide a link or resources how to run pytorch models from c++?

https://pytorch.org/cppdocs/ This is a brief description of the API. It worked fine for us even for complex models.

Re: The State of Machine Learning Frameworks

#195
I thought the article was a good read and compared the two frameworks with only small hints of personal bias, but one point about industry changing to use pytorch because of researchers already knowing it seems like wishful thinking. Unless PyTorch addresses its mobile and serving issues it is simply not a great choice for many production situations. This article actually influenced me to stick with TF instead of learning PyTorch due to my industry needs.

Additionally I think tensorflow opt in by default for eager execution is fine maybe good even. Many models are relatively simple and I doubt the gains for rewriting them to utilize the execution graph will be worth it when with the keras frontend you can just dump the h5py model and run it from there which many companies already do.

Rewriting will only be an issue for sufficiently complex models and at that point I imagine competent ML professionals will have baked the time for that into the estimate of the engineering costs.

Re: The State of Machine Learning Frameworks

#196

I work at a small company as an engineer and recently was asked to do a project that would require some neural net magic. I had some experience with keras/tensorflow so that was my first choice. Despite the absolute nightmare of getting it installed and running on a gpu, I managed it and had a fantastic model. It was doing so well that the company wanted to expand the project and build out a multi-gpu rig as part of…

Regarding TensorFlow, you could use the docker images and run everything in containers instead. They tend to work pretty fine out of the box.

Re: The State of Machine Learning Frameworks

#197
post #167

I work at a small company as an engineer and recently was asked to do a project that would require some neural net magic. I had some experience with keras/tensorflow so that was my first choice. Despite the absolute nightmare of getting it installed and running on a gpu, I managed it and had a fantastic model. It was doing so well that the company wanted to expand the project and build out a multi-gpu rig as part of…

What was the bug?

Late reply, but it was a bug using XLA GPUs to add concurrency to the training process. Maybe someone figured it out or fixed it, but I've moved on already.

Re: The State of Machine Learning Frameworks

#198

I work at a small company as an engineer and recently was asked to do a project that would require some neural net magic. I had some experience with keras/tensorflow so that was my first choice. Despite the absolute nightmare of getting it installed and running on a gpu, I managed it and had a fantastic model. It was doing so well that the company wanted to expand the project and build out a multi-gpu rig as part of…

Complexity of installing TensorFlow, even with the inclusion of custom compilation and hacking Bazel (to make it work under CUDA version that it doesn't officially support) is low , compared to releasing a model that works in production. Because of that, it doesn't make much sense to judge a "differential programming language" like TensorFlow or PyTorch by the ease of installation. It'd be like saying "I prefer C# ov…

I did say I had a fantastic model with Tensorflow. I gave up after a while because I didn't have time to hack on that stuff. I wouldn't mind learning to and trying it out, but the nature of the small company meant I needed to find a solution sooner. Now I have comparable results with Pytorch and it's easier to work with. That's a win/win in my book.

Re: The State of Machine Learning Frameworks

#199

I work at a small company as an engineer and recently was asked to do a project that would require some neural net magic. I had some experience with keras/tensorflow so that was my first choice. Despite the absolute nightmare of getting it installed and running on a gpu, I managed it and had a fantastic model. It was doing so well that the company wanted to expand the project and build out a multi-gpu rig as part of…

Regarding TensorFlow, you could use the docker images and run everything in containers instead. They tend to work pretty fine out of the box.

I looked into this but hadn't gotten to the point of using them when I made the switch. I may go back and try it out to see how it goes. Thanks for the encouragement.

Re: The State of Machine Learning Frameworks

#200
post #167

Earlier quoted context omitted.

What was the bug?

Late reply, but it was a bug using XLA GPUs to add concurrency to the training process. Maybe someone figured it out or fixed it, but I've moved on already.

You shouldn’t need XLA for multi GPU training. Have you tried training without it?
Post reply on HN