Live data from Hacker News

Launching our Data Science and Big Data Track

blog.udacity.com

31–39 of 39 posts

Re: Launching our Data Science and Big Data Track

#32
So looking through this 'track', I see one course which seems like it might be more central to the discipline, "Intro to Data Science"[0]. Has anybody had a chance to compare this one against Bill Howe's "Introduction to Data Science"[1] on Coursera?

[0] https://www.udacity.com/course/ud359 [1] https://www.coursera.org/course/datasci

Re: Launching our Data Science and Big Data Track

#33
post #12

99% of people looking for information about big data and 99% of people looking to do data science, don't have nowhere near big data, and don't need to be taught hadoop. Those people are instead often lacking fundamental knowledge and are looking for a trick technological solution instead of reviewing their basics. A "data science" track should hence be 90% about algorithms, data structures, linear algebra and compute…

your list of stuff to learn is rather heavy on implementation - wouldn't it make more sense to use a poor implementation of an algorithm in hadoop that uses 10 machines than to squeeze every ounce of performance using GPU's, cache locality etc. in a single machine, at great expense in programmer time?

I think that one nice thing about the idea of "big data" is being about to parallelize the problem and just throw more cores at it.

But on the other hand, I do think that when people think of "big data" they have in some magic solution that doesn't really exist. At the end of the day big data is just statistics.

Re: Launching our Data Science and Big Data Track

#34
post #20

Earlier quoted context omitted.

Aren't there already plenty of courses and books out there about algorithms, data structures, RDBMS, etc? I have a pretty good background in a lot of that (can always learn more of course), but I don't know anything about Hadoop and MapReduce (which is conveniently not mentioned in your critique, probably because it does fall under your list of acceptable topics), so I find this course interesting. I find the claim o…

What really annoys me is this particular blog post, not just the existence of the course, for example this thing: “What is Big Data?” They will teach you fundamental principles of Hadoop, MapReduce, and how to make sense of big data. Developers will learn skills that provide fundamental building blocks towards deriving maximum value from the world's data. Technologists and business managers will gain the knowledge to…

A lot of universities are definitely trying to push more and more of these sorts of courses though; check out, for example, cs229r at Harvard: http://people.seas.harvard.edu/~minilek/cs229r/index.html , along with some other course examples at the bottom of that page. Do you think things are changing for the better in this manner?

Re: Launching our Data Science and Big Data Track

#35
post #12

99% of people looking for information about big data and 99% of people looking to do data science, don't have nowhere near big data, and don't need to be taught hadoop. Those people are instead often lacking fundamental knowledge and are looking for a trick technological solution instead of reviewing their basics. A "data science" track should hence be 90% about algorithms, data structures, linear algebra and compute…

your list of stuff to learn is rather heavy on implementation - wouldn't it make more sense to use a poor implementation of an algorithm in hadoop that uses 10 machines than to squeeze every ounce of performance using GPU's, cache locality etc. in a single machine, at great expense in programmer time? I think that one nice thing about the idea of "big data" is being about to parallelize the problem and just throw mor…

This is a fair counterpoint, but if your lack of fundamentals caused you to write an O(n^k); k>1 algorithm, you're not going to be able to pay your way to the solution with more cores if you truly have "big data". Even constant multipliers of a poor O(n) algorithm will cost you serious bucks if your default optimization strategy is "buy more computers" rather than a few afternoons of quiet thought.

Re: Launching our Data Science and Big Data Track

#36
post #21

Earlier quoted context omitted.

100% agree except that I think courses like this are great for people who want to bluff their way through a job interview to get one of those $150K-$250K big data jobs that are the rage right now (watching Andrew Ng's machine learning lectures beforehand as well would be the pro move in my book). In my experience so far, most of these positions appear to be Java programming gigs where, sadly, issues like SIMD, cache…

It's faster than python and R, dogg.

You can do extreme performance coding for Java, Javascript, R, Python, Haskell or the language of your choice as long as the number-crunching is done by calling low-level heavily optimized code. For example, PyCUDA:

http://mathema.tician.de/software/pycuda/

And this is where I get told by data scientists that they don't wish to support such code. And IMO that's fine for piddling around and experimentation. But for production, on thousands to hundreds of thousands of servers, running 24/7, at companies with billions and billions of dollars in the bank, that's leaving way too many transistors and electrons on the table for me to stomach. In contrast, here's what you can achieve when you do pay attention to these things:

http://istc-bigdata.org/index.php/mapd-a-way-to-map-big-data...

Re: Launching our Data Science and Big Data Track

#37

Earlier quoted context omitted.

Helping Walmart is, at best, morally suspect.

Reducing retail waste is better for everyone. Though some of the stuff they do is questionable, like how carefully items in the store are placed to maximize the amount of unnecessary crap they sell to impulsive people.

How does 'exploitative labor practices' float your boat.

Re: Launching our Data Science and Big Data Track

#38
post #36

Earlier quoted context omitted.

It's faster than python and R, dogg.

You can do extreme performance coding for Java, Javascript, R, Python, Haskell or the language of your choice as long as the number-crunching is done by calling low-level heavily optimized code. For example, PyCUDA: http://mathema.tician.de/software/pycuda/ And this is where I get told by data scientists that they don't wish to support such code. And IMO that's fine for piddling around and experimentation. But for pr…

You're in a different part of the problem space.

Re: Launching our Data Science and Big Data Track

#39

Earlier quoted context omitted.

Reducing retail waste is better for everyone. Though some of the stuff they do is questionable, like how carefully items in the store are placed to maximize the amount of unnecessary crap they sell to impulsive people.

How does 'exploitative labor practices' float your boat.

Well hopefully those jobs will be automated soon enough as machine vision and robotics rapidly pick up pace.
Post reply on HN