Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Keras 2

blog.keras.io

Post reply on HN
Print view View original

51–60 of 70 posts

Re: Introducing Keras 2

#51

Post by kriro »

krick wrote: post #22

I'm only starting with all that machine-learning, NN stuff and as many others I want to ask for some guidance/resources/learning material. What I feel especially lacking is something very broad and generic, some overview of existing techniques (but not as naïve as Ng's ML course, I assume). There exist a lot of estimators and classifiers, there exist a lot of techniques and tricks to train models, there exist a lot o…

Even though you specifically say you are willing to go full blown PhD and are interested in digging deep on algorithms etc. I strongly recommend working through "Practical Deep Learning for Coders" course at fast.ai It's free :)

It gives you an excellent feel for what is possible and they are very focused on solving interesting and practical problems right away. They explicitly try to take the "requires a math PhD" out of deep learning. Once you're through with the course you have a very solid practical overview and understanding and can solve tons of real world problems (it's almost a startup idea generator tbh.) and once you're at that stage it becomes tons easier to dive deep into specific algorithms and optimizations.

tl;dr: Take the course (they also walk you through setting up a AWS GPU server so no fancy hardware required) and you'll be able to solve real world problems with state of the art algorithms.

Re: Introducing Keras 2

#52

Post by bpicolo »

jph00 wrote: post #13
bpicolo wrote: post #6

Earlier quoted context omitted.

I'm definitely going to give this a shot, thanks for the link. Approaching ML at a higher level is exactly what I need to develop a better interest in it. I realize that underpinnings are important, but waiting 30 minutes for mnist on to process on my localhost is just unbearably boring.

If you try the course, be sure to make use of the forums for it too: http://forums.fast.ai . As you'll see, they're extremely active and helpful for all deep learning students (and all practitioners in general). Disclaimer: I teach the course. Although it is free and ad-free... :)

Will do. I'm in a good position to start (no shortage of python/aws experience), so the only fighting will be with the deep learning bits, hah.

Side note: Great job on making AWS setup/teardown straightforward. 90c/h is not terrible, but not cheap to accidentally leave on!

Re: Introducing Keras 2

#53

Post by soVeryTired »

nl wrote: post #50
soVeryTired wrote: post #49

Earlier quoted context omitted.

Another less-recognised point is that in industry, you also need to ask "how can I maintain this?" and "what can go wrong with my algorithm?". In one use case, a "blip" in your algorithm might mean showing the wrong kind of advertisement to a user. Not great, but ultimately no big deal. In another, it might mean automatically buying billions of dollars' worth of pumpkin futures (cf. Knight capital). In the latter cas…

While I agree with your point (and often use this in interview questions) that wasn't what caused the Knight Capital problem. That was bad software engineering and deployment practices, and had nothing to do with interprability of the model (actually it had little to do with the model at all.) They repurposed a feature toggle, then misdeployed the code: http://pythonsweetness.tumblr.com/post/64740079543/how-to-lo...…

Yep - meant it as an example of a general catastrophic software glitch rather than a ML algorithm gone haywire.

Re: Introducing Keras 2

#54

Post by minimaxir »

nl wrote: post #45
mattkrause wrote: post #35

Earlier quoted context omitted.

"Try 'em all" is not just an answer, but the only answer. The No Free Lunch Theorem says that averaged across all possible problems, no single classifier is the best; in fact, they're all equivalent. However, you probably don't care about all possible problems, but a specific one. Over the last decade or so, we've discovered that deep learning works really well on certain classes of problems, particularly those that…

Right - this is good advice. To paraphrase the learnings of thousands of data scientists on years of Kaggle competitions: A quick and dirty model for a baseline: Random Forest Structured data: Use a boosted tree algorithm (specifically the XGBoost implementation of gradient boosting), ensembled with maybe Extra Trees, Random Forests and MLPs Some kind of time component on large datasets: FTL regression, XGB Binary da…

LightGBM (https://github.com/Microsoft/LightGBM) is shaping up to beat XGBoost; it has mostly API parity and it won in benchmarks before a v2 with a new algorithm.

Re: Introducing Keras 2

#56

Post by srrr »

diminish wrote: post #41

Slightly irrelevant but curious question about the Analytics for 7day (34K), 14day and 30day active users. I'm running a similar site so, could it be that, a lot of users reading documentation are using ad/tracking blockers so that active users count appear higher than it actually is in GA. Documentation users tend to read quite high pages per session. If I'm right then they should see less page views per user than e…

For most websites I am familiar with, 20-30% of users use adblockers or other privacy plugins. On content with a target audience of developers I have sometimes seen 60%. Most of these users are not recorded in google analytics, so the real number of unique users is higher than reportet by analytics.

Since no data from these users is sent at all (no user data and no pageview data) page views per user is not directly influenced because your are missing users and pageviews in your reporting. It could be influenced because users with an adblocker behave differently then users without. To analyse this you would have to look at your server generated web logs.

The given data in the image is often not used to find out what the total amount of unique users on your webpage is. It is used for computing engagement: Monthly active users vs. daily active users. In this example we only have 7-day active users and no daily active users, but it basically is like: 34738/107942=0.32. At a value of 1 (the maximum) you have a high engagement. In simple terms: Each user would come back every week for this month. 0.32 is quite low. Around 0.25 would be the lower bound because we have 4 weeks in a month.

Re: Introducing Keras 2

#57

Post by nl »

minimaxir wrote: post #54
nl wrote: post #45

Earlier quoted context omitted.

Right - this is good advice. To paraphrase the learnings of thousands of data scientists on years of Kaggle competitions: A quick and dirty model for a baseline: Random Forest Structured data: Use a boosted tree algorithm (specifically the XGBoost implementation of gradient boosting), ensembled with maybe Extra Trees, Random Forests and MLPs Some kind of time component on large datasets: FTL regression, XGB Binary da…

LightGBM ( https://github.com/Microsoft/LightGBM ) is shaping up to beat XGBoost; it has mostly API parity and it won in benchmarks before a v2 with a new algorithm.

I tried LightGBM for a Kaggle. I couldn't get anywhere near XGB.

I was using the LambdaRank stuff. Given the boasting the LightGBM team had done I had assumed it would be close to XGB out-of-the-box for a ranking problem (since XGB only does pairwise ranking). It was far enough away that I had to ask if I was misinterpreting the output[1].

That was 6 months ago now, so maybe it has improved. I know they made big claims.

[1] https://github.com/Microsoft/LightGBM/issues/37

Re: Introducing Keras 2

#58

Post by minimaxir »

nl wrote: post #57
minimaxir wrote: post #54

Earlier quoted context omitted.

LightGBM ( https://github.com/Microsoft/LightGBM ) is shaping up to beat XGBoost; it has mostly API parity and it won in benchmarks before a v2 with a new algorithm.

I tried LightGBM for a Kaggle. I couldn't get anywhere near XGB. I was using the LambdaRank stuff. Given the boasting the LightGBM team had done I had assumed it would be close to XGB out-of-the-box for a ranking problem (since XGB only does pairwise ranking). It was far enough away that I had to ask if I was misinterpreting the output[1]. That was 6 months ago now, so maybe it has improved. I know they made big clai…

Development was rapid when I was working on a blog post in January using the tool. Things have likely improved if you want to give it another shot.

Re: Introducing Keras 2

#59

Post by jph00 »

apaszke wrote: post #38
jph00 wrote: post #14

Earlier quoted context omitted.

I'm intrigued!... The kernel calling overhead and lack of any GPU while/scan/map/etc for Pytorch seems like a limitation, but I guess on 2nd thoughts you can still do all the keras fit/predict stuff and auto-connecting up the layers.

These ops are just not needed in PyTorch. while is just a Python while loop. Scan is a for loop, map is a list comprehension that applies modules. No need for anything fancy.

Sure - but on pytorch they suffer the kernel launch overhead each time through the loop, whereas on tensorflow and theano they do not. Which really impacts the kinds of algorithms that work well on each platform. Does that seem like a reasonable assessment to you?

Re: Introducing Keras 2

#60

Post by jph00 »

bpicolo wrote: post #52
jph00 wrote: post #13

Earlier quoted context omitted.

If you try the course, be sure to make use of the forums for it too: http://forums.fast.ai . As you'll see, they're extremely active and helpful for all deep learning students (and all practitioners in general). Disclaimer: I teach the course. Although it is free and ad-free... :)

Will do. I'm in a good position to start (no shortage of python/aws experience), so the only fighting will be with the deep learning bits, hah. Side note: Great job on making AWS setup/teardown straightforward. 90c/h is not terrible, but not cheap to accidentally leave on!

Regarding AWS, one participant has created a system that lets you use spot instances for the course. It's published on the forum. Great way to save $$$ (400% or more...)
Post reply on HN