Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The State of Machine Learning Frameworks

thegradient.pub

Post reply on HN
Print view View original

171–180 of 201 posts

Re: The State of Machine Learning Frameworks

#171

Post by Stasis5001 »

jimfleming wrote: post #150
oli5679 wrote: post #126

I have worked as data scientist on a lot of finance domain problems - forecasting default, fraud, conversion probability ect. Lightgbm library has consistently performed well. I've been interested in how many colleagues instantly jump to neural nets when in my experience this often doesn't beat lightgbm on medium sized datasets not related to text/images.

More anecdata: we consistently outperform lightgbm, xgboost, random forests, linear models, etc. using neural networks even on smaller datasets. This applies whether we implemented the other algorithms ourselves or simply compared to someone else’s results with them. In my experience it really comes down to how many “tricks” you know for each algorithm and how well can you apply and combine these “tricks”. The differ…

Any good references on the art part, or is it just an intuition you develop over time? In my experience, all the ML education will teach you a ton of theory and basics but none of the practical details you're referring to.

Re: The State of Machine Learning Frameworks

#172

Post by rytill »

mlthoughts2018 wrote: post #69
machinelearning wrote: post #63

Earlier quoted context omitted.

Are you saying tensorflow doesn't have more code to check in or that even bad engineers don't pretend that checking in more code looks productive?

Not even bad engineers pretend like more code == more productive. Emphatically so.

Bad engineers have pretended like more code is more productive. Certain programmer-types will grasp at any straws to maintain their ego.

Re: The State of Machine Learning Frameworks

#173

Post by oli5679 »

jimfleming wrote: post #150
oli5679 wrote: post #126

I have worked as data scientist on a lot of finance domain problems - forecasting default, fraud, conversion probability ect. Lightgbm library has consistently performed well. I've been interested in how many colleagues instantly jump to neural nets when in my experience this often doesn't beat lightgbm on medium sized datasets not related to text/images.

More anecdata: we consistently outperform lightgbm, xgboost, random forests, linear models, etc. using neural networks even on smaller datasets. This applies whether we implemented the other algorithms ourselves or simply compared to someone else’s results with them. In my experience it really comes down to how many “tricks” you know for each algorithm and how well can you apply and combine these “tricks”. The differ…

Do you have any advice on how to increase performance of NN? Id be interested to see some examples of NN doing better than lgbm benchmark on medium size tabular data. What black magic is needed to achieve this? Would be super valuable to my job:)

This tuning approach gets good results for Lightgbm. I'd recommend using TimeSeriesSplit.

https://www.kaggle.com/nanomathias/bayesian-optimization-of-...

I've seen colleagues do something like this, or random search over NN architecture (NUM layers, nodes per layer, learning rate, dropout rate), always falling short of results this archives, despite far longer time to code up an tune model.

Re: The State of Machine Learning Frameworks

#174

Post by nmca »

mlevental wrote: post #118
nmca wrote: post #110

Jax, for those that haven't heard of it, is the thing y'all want.

why would you use Jax over pytorch? even if it has technical merits it lacks an ecosystem of readily available models to study and tweak.

At some point you stop caring about being able to import a set of imagnet pretrained weights and start caring about extreme flexibility. Think about implement ting, say "Scene Representation Networks" https://arxiv.org/abs/1906.01618 in each of the three frameworks. Tf is a pig, pytorch is slow, and Jax is going to crush the problem.

The lack of say, keras.applications is a shame, but it won't last, and if you have a GPU or 8 the power of optimized (p/v)map definitely makes up for it.

Re: The State of Machine Learning Frameworks

#176

Post by dchichkov »

rtkaratekid wrote: post #154

I work at a small company as an engineer and recently was asked to do a project that would require some neural net magic. I had some experience with keras/tensorflow so that was my first choice. Despite the absolute nightmare of getting it installed and running on a gpu, I managed it and had a fantastic model. It was doing so well that the company wanted to expand the project and build out a multi-gpu rig as part of…

Complexity of installing TensorFlow, even with the inclusion of custom compilation and hacking Bazel (to make it work under CUDA version that it doesn't officially support) is low, compared to releasing a model that works in production.

Because of that, it doesn't make much sense to judge a "differential programming language" like TensorFlow or PyTorch by the ease of installation. It'd be like saying "I prefer C# over C++" because it is easier to install.

Re: The State of Machine Learning Frameworks

#177

Post by chillee »

nmca wrote: post #174
mlevental wrote: post #118

Earlier quoted context omitted.

why would you use Jax over pytorch? even if it has technical merits it lacks an ecosystem of readily available models to study and tweak.

At some point you stop caring about being able to import a set of imagnet pretrained weights and start caring about extreme flexibility. Think about implement ting, say "Scene Representation Networks" https://arxiv.org/abs/1906.01618 in each of the three frameworks. Tf is a pig, pytorch is slow, and Jax is going to crush the problem. The lack of say, keras.applications is a shame, but it won't last, and if you have a…

I mean, the authors implemented it in pytorch: https://github.com/vsitzmann/scene-representation-networks

Do you have any particular evidence that PyTorch is slow here?

Re: The State of Machine Learning Frameworks

#178

Post by disgruntledphd2 »

lalaland1125 wrote: post #108
kuu wrote: post #101

Earlier quoted context omitted.

Uhm I would think that the tf.keras API is a clone of Keras (I mean, Keras was made by a Google Engineer!!)... which came before Pytorch

tf.keras contains a deprecated API that was the original keras as well as a new API that basically is a clone of Pytorch. You can tell the difference because only the new API works nicely with eager mode.

And the torch API has been around for a looooonnng time, as Pytorch is a re-implementation of Torch, which was written in Lua: http://torch.ch/

Re: The State of Machine Learning Frameworks

#179

Post by hobofromabroad »

oli5679 wrote: post #126

I have worked as data scientist on a lot of finance domain problems - forecasting default, fraud, conversion probability ect. Lightgbm library has consistently performed well. I've been interested in how many colleagues instantly jump to neural nets when in my experience this often doesn't beat lightgbm on medium sized datasets not related to text/images.

The problem with these models is that you have to be careful that your are not modeling something that incorporates a trend.

And if you have any kind of seasonality you a dataset with a large enough timeframe. (At least more than a year.)

Nonetheless, LightGBM and xgboost are also commonly used in the insurance sector.

They are still somewhat problematic for conversion rates in a highly dynamic market though.

Re: The State of Machine Learning Frameworks

#180

Post by fabiomaia »

klintcho wrote: post #135
UncleOxidant wrote: post #62

Earlier quoted context omitted.

Last year I was tasked with looking into a NAS (Neural Architectures Search) paper and analyzing the algorithm. The paper came with a TensorFLow implementation. Trying to read that TF code was quite difficult. I searched around and found a PyTorch implementation - much easier to read and understand, and it ran about 50% faster as well (the latter was a bit surprising). I tend to think that TensorFlow lends itself to…

What made me fall in love with PyTorch was also that the "neural network training process" is defined almost as it is in theory, in code in PyTorch - loop through epochs - loop through each batch - run a forward pass for the batch ( model(batch) ) - calculate the loss for the batch ( criteria(y, yprim) - compute the gradients/backprop ( loss.backward() ) - update the weights (optimizer.step()) This really enforced ev…

Yes! Exactly.
Post reply on HN