Slightly irrelevant but curious question about the Analytics for 7day (34K), 14day and 30day active users. I'm running a similar site so, could it be that, a lot of users reading documentation are using ad/tracking blockers so that active users count appear higher than it actually is in GA. Documentation users tend to read quite high pages per session. If I'm right then they should see less page views per user than expected.
Introducing Keras 2
41–50 of 70 posts
Re: Introducing Keras 2
#42Is it better to learn Keras instead of tflearn? Copying a comment I made in another thread where one response recommended Keras: I currently have a small pet project where I think some simple ML would be cool but I don't know where to start. Basically my use case is that I have a bunch of 64x64 images (16 colors) which I manually label as "good", "neutral" or "bad". I want to input this dataset and train the network…
Personally, I find Keras better than tflearn.
But you need to know the fundamentals on TF and NNs (RNNs,LSTMs, etc...). Keras makes it easier to build on those concepts with less programming. I've found TFLearn to be slightly complicated. Both Keras and tflearn make it simpler to deal with TF.
Creating a good train-test dataset is general ML problem. Keras doesn't solve that and isn't meant to do that.
However, Keras (and tflearn too) makes it easy to throw a statistically bad dataset to an NN, add multiple layers and then let TF take over and derive a inefficient model in a few hours. The amazing part is that the inefficient NN (driven by TF) might still return a slightly acceptable accuracy. This is awesome because you may be an amateur and yet have some okay results to start with. Later you can improvise the dataset to improve the accuracy.
In general, throwing NNs at everything isn't good. They result in hard-to-decompile blackbox models. If NNs give you good classification, you could also try the same with other classifiers. You could also start looking into scikit-learn algos and see if those could be used in your case.
Re: Introducing Keras 2
#43Is it better to learn Keras instead of tflearn? Copying a comment I made in another thread where one response recommended Keras: I currently have a small pet project where I think some simple ML would be cool but I don't know where to start. Basically my use case is that I have a bunch of 64x64 images (16 colors) which I manually label as "good", "neutral" or "bad". I want to input this dataset and train the network…
If you don't know how to set up a dataset, it's probably too early for you to worry about performance and efficiency.
If you haven't, already, I'd suggest to learn some general machine learning, including how to use logistic regression, random forests and SVMs.
Keras is certainly capable of what you want to do, at least from your description.
One way is to interpret the colors as grayscale images, that would be the fastest option. If however the 16 colors are actually from a palette, it may be better to convert the image to three channels, r/g/b. And if the 16 colors are 16 entirely different things, like 0 - Water, 1 - sand, 2 - earth and so on, you could even turn one 16 color image into 16 images with two colors (1 bit), and get a better model.
Again, getting into machine learning or deep learning is not as easy as reading the Keras documentation. You need to understand the basics first.
Re: Introducing Keras 2
#44Earlier quoted context omitted.
I'd definitely watch the first few episodes of Ng's stuff, up to and including logistic regression (unless you know all of that already, in which case: read papers and do practice projects for yourself--or compete in kaggle if you don't have any application ideas) The most common way to apply machine learning is supervised classification. The basic formula is: we learn a model (set of weights) to approximately map da…
It surely does make sense to me, but I seriously think (maybe hope, even?) that "hacking-driven" approach here is significantly overvalued. Because of sociological reasons. After all, all this is mathematical problems, and while I'm aware that NNs are pretty much unexplored space, there surely must exist some quite significant amount of knowledge at level below the NNs that can be actually systematically learned . Al…
The problem with looking for a theoretical as to why one method should be chosen over another is that you run into the "No Free Lunch theorem"[1]:
any two optimization algorithms are equivalent when their performance is averaged across all possible problems
Once you accept that, then you start looking at practical considerations.
Having said that, if you do want to do the math then you might like the course from Oxford/Nando DeFreitas (now at DeepMind/Oxford)[2]
[1] https://en.wikipedia.org/wiki/No_free_lunch_theorem
[2] https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPo..., https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearni...
Re: Introducing Keras 2
#45Earlier quoted context omitted.
So how, for instance, do I even decide, that Random Forest is not enough for this task and I want to build some specific kind of neural net? The problem here is that it's really hard to give generic advice. As an analogy this is like asking "how do I know if Rails is enough for this task". The answer is usually "yes", but the specifics matter a lot. So in this specific case (and I realize you aren't looking for speci…
"Try 'em all" is not just an answer, but the only answer. The No Free Lunch Theorem says that averaged across all possible problems, no single classifier is the best; in fact, they're all equivalent. However, you probably don't care about all possible problems, but a specific one. Over the last decade or so, we've discovered that deep learning works really well on certain classes of problems, particularly those that…
Right - this is good advice.
To paraphrase the learnings of thousands of data scientists on years of Kaggle competitions:
A quick and dirty model for a baseline: Random Forest
Structured data: Use a boosted tree algorithm (specifically the XGBoost implementation of gradient boosting), ensembled with maybe Extra Trees, Random Forests and MLPs
Some kind of time component on large datasets: FTL regression, XGB
Binary data (images or sound): Deep neural nets
Text: Try LSTMs, but this will often be beaten by manual feature engineering and Word2Vec derived features put into XGB.
Re: Introducing Keras 2
#46Copying my rare product endorsement from the previous submission: Keras is so good that it is effectively cheating in machine learning, where even Tensorflow tutorials can be replaced with a single line of code. (which is important for iteration; Keras layers are effectively Lego blocks). A simple read of the Keras examples ( https://github.com/fchollet/keras/tree/master/examples ) and documentation ( https://keras.i…
Yes
Trying to do something simple in TF is a pain, on the code there are some conflicting examples and code snippets that "train" a network just to print a loss number on the screen but actually do nothing besides that
Keras is easy to use and better if you're running CPU only
Re: Introducing Keras 2
#47[deleted]
Re: Introducing Keras 2
#481. Still no support for multiple losses. Models like VAEs cannot be idiomatically implemented. The second loss has to be 'hacked' in. Notice how in the official example for VAE, the kl_loss is computed using variables which are NOT available via the loss function ( https://github.com/fchollet/keras/blob/master/examples/varia... ) 2. It's still an input->output paradigm, rather than a {input, output}->loss paradigm wh…
See the release notes: https://github.com/fchollet/keras/wiki/Keras-2.0-release-not... - You can use a Keras model to compute some tensor(s), turn that into a loss, and manually add that loss to the model via `add_loss` (it just needs to only depend on the model's inputs). - Not all of your model outputs have to have a loss associated with them. So you can do both {input, output}->loss and input->output in your workf…
I will look into updating the VAE example, as I've ported the example to the keras 2.0 API recently. There is currently no documentation on add_loss as far as I can see, so I will have to try a few things.
Re: Introducing Keras 2
#49I'm only starting with all that machine-learning, NN stuff and as many others I want to ask for some guidance/resources/learning material. What I feel especially lacking is something very broad and generic, some overview of existing techniques (but not as naïve as Ng's ML course, I assume). There exist a lot of estimators and classifiers, there exist a lot of techniques and tricks to train models, there exist a lot o…
So how, for instance, do I even decide, that Random Forest is not enough for this task and I want to build some specific kind of neural net? The problem here is that it's really hard to give generic advice. As an analogy this is like asking "how do I know if Rails is enough for this task". The answer is usually "yes", but the specifics matter a lot. So in this specific case (and I realize you aren't looking for speci…
Another less-recognised point is that in industry, you also need to ask "how can I maintain this?" and "what can go wrong with my algorithm?".
In one use case, a "blip" in your algorithm might mean showing the wrong kind of advertisement to a user. Not great, but ultimately no big deal. In another, it might mean automatically buying billions of dollars' worth of pumpkin futures (cf. Knight capital).
In the latter case you need a much greater penalty on model complexity, and much more emphasis on interpretability.
Re: Introducing Keras 2
#50Earlier quoted context omitted.
So how, for instance, do I even decide, that Random Forest is not enough for this task and I want to build some specific kind of neural net? The problem here is that it's really hard to give generic advice. As an analogy this is like asking "how do I know if Rails is enough for this task". The answer is usually "yes", but the specifics matter a lot. So in this specific case (and I realize you aren't looking for speci…
Another less-recognised point is that in industry, you also need to ask "how can I maintain this?" and "what can go wrong with my algorithm?". In one use case, a "blip" in your algorithm might mean showing the wrong kind of advertisement to a user. Not great, but ultimately no big deal. In another, it might mean automatically buying billions of dollars' worth of pumpkin futures (cf. Knight capital). In the latter cas…
While I agree with your point (and often use this in interview questions) that wasn't what caused the Knight Capital problem.
That was bad software engineering and deployment practices, and had nothing to do with interprability of the model (actually it had little to do with the model at all.) They repurposed a feature toggle, then misdeployed the code: http://pythonsweetness.tumblr.com/post/64740079543/how-to-lo...
I understand that this was an example, but I'm sure someone will misread it as what happened in that case.