Am I the only one who was expecting to learn about data science and instead I got some moralising?
An Interview with an Anonymous Data Scientist (2016)
91–100 of 108 posts
Re: An Interview with an Anonymous Data Scientist (2016)
#92I feel so many data scientists are bullshit. I had the worse interviews, like someone telling me about how ARIMA is so good and why would I even use a LSTM network. Even worse is they cite some bullshit consulting article with skewed data to prove their point.
Re: An Interview with an Anonymous Data Scientist (2016)
#93some interviews ask me the stupidest questions "how large is your dataset?" , "have you ever worked with 100GB of data". fucking morons
Re: An Interview with an Anonymous Data Scientist (2016)
#94I've been seeing nothing but negative, dismissive comments about data science on HN lately, which is really disappointing. There's definitely a lot of hype right now about DL, but almost all of my job does not deal with Big Data or Deep Learning, 'just' machine learning + stats + calc + scripting + data cleaning + deploying models. I think most people don't have big data (Amazon has an x1 with 4 TB of RAM, after all!…
I think their hype got even you a little bit. That is revealed by the word "even" in the phrase: 'people calling CNNs or even non-deep-learning models "AI"'...
What I mean by this is - I don't see how anyone could reasonably call a Random Forest "AI" with a straight face, whereas someone could (wrongly, but understandably) call a CNN / RNN / etc. AI if only because it has the word "neural" in it.
There's two groups:
- People who are overly enthusiastic about neural nets
- People who are cynically calling every ML algorithm "AI", up to and including linear regression
and I'm more annoyed at the last one.
Re: An Interview with an Anonymous Data Scientist (2016)
#95I've been seeing nothing but negative, dismissive comments about data science on HN lately, which is really disappointing. There's definitely a lot of hype right now about DL, but almost all of my job does not deal with Big Data or Deep Learning, 'just' machine learning + stats + calc + scripting + data cleaning + deploying models. I think most people don't have big data (Amazon has an x1 with 4 TB of RAM, after all!…
I've been seeing nothing but negative, dismissive comments about data science on HN lately, which is really disappointing. There's definitely a lot of hype right now about DL, but almost all of my job does not deal with Big Data or Deep Learning, 'just' machine learning + stats + calc + scripting + data cleaning + deploying models. But, all those things people did in the '90's or even earlier. It was called "data war…
> But, all those things people did in the '90's or even earlier. It was called "data warehousing" or "decision support" back then.
I would say data warehousing is more concerned with things like OLAP, Star Schema, ETL, etc. than what people are calling 'data science' right now. The same thing with 'decision support', since data warehousing grew out of decision support systems. The most overlap here is with 'data mining' algorithms like association rules clustering.
> The fundamental techniques - linear regression, logistic regression, k-mean clustering - go back even earlier, to the OR community post-WW2.
Here I think you've got a stronger argument. OR has a long, proud history of using applied math for business objectives. But again, I would say most of OR deals with different problems and different techniques - it's more about prescriptive analytics, constrained optimization, linear programming, simulations, etc. than the type of predictive modeling in most data science.
I see data science as a separate field even though it's stitched together from a bunch of others. It's certainly not entirely new, and certainly overhyped in some annoyingly-breathless news reports. I could say the same thing about CS - was it entirely "new" when it started as a discipline? Isn't CS "just" applied math?
Re: An Interview with an Anonymous Data Scientist (2016)
#96Earlier quoted context omitted.
I think the contrast is between statisticians and physicists PhDs compiling GPU support... even some CS PhDs have a hard time with that... this is less important as time goes on since the engineers figure it out and make it readily available.
When I installed Theano, it was just `pip install theano`, and editing a couple of lines in a config file. Are other GPU libs (tensorflow, caffe, etc.) really that much more difficult?
pip install tensorflow-gpu
is all I do, once the dependancies are setup.
Re: An Interview with an Anonymous Data Scientist (2016)
#97Can anyone comment on his point about Spark's ML libs? I note that was from last year (about 2015 code), not sure what level of beta they were at, but yeah, I use it for batch processing, but have never used the ML aspects, so just curious. > And even up to last year, there’s just massive bugs in the machine learning libraries that come bundled with Spark. It’s so bizarre, because you go to Caltrain, and there’s a gi…
Getting better obviously, but the feet-on-the-ground experience for MLlib is still far from pleasant: hard to configure, hard to manage, hard to scale, hard to debug. By way of anecdote, Spark's MLlib used to contain an implementation of word2vec that failed when used on more than 2 billion words (some arcane integer overflow). So much for scale! As for performance, in 2016, the break-even point where a Spark cluster…
lol, that PDF is referencing Spark 1.3 from March 2015 and to say that you need 12 modern Spark machines to break-even with one machine running a non-distributed ML framework is ridiculously wrong. And he wan Spark on EMR, which was pretty unoptimized back then.
Re: An Interview with an Anonymous Data Scientist (2016)
#98I'm not native English speaker and I find this sentence from the article weird: > Because the frightening thing is that even if you remove those specific variables, if the signal is there, you're going to find correlates with it all the time, and you either need to have a regulator that says, “You can use these variables, you can't use these variables,” or, > I don't know, we need to change the law. As a data scienti…
Race is an incredibly sensitive topic in America. The best analogy I can come up with for the author's statement is this:
You're looking to pick the fastest runners out of a group of people. You run an optimization algorithm to pick out the fastest in that group. Nothing about this optimization accounts for the fact that 1/3 of the people in the group have been being shot in the foot with a gun prior to your optimization. The data will show that they are poor runners without addressing the crime previously committed. In fact, many people would consider it a second act of crime.
Re: An Interview with an Anonymous Data Scientist (2016)
#99DL is hyped as a big thing, but why are multiple layers on a NN a breakthrough? The only breakthrough is hardware, but I don't see that hyped.
More layers allowed us to explore exponentially more network architectures. And if you look at a lot of advances in deep learning, particularly in convnets, the architecture is actually key - as important or more than the weights themselves. I guess another thing is that more layers give a disproportionate increase in performance. Some of it is hardware but there have definitely been advances in the theory; people aren't getting these new results from 10, 20 yr old networks that have been made larger.
Re: An Interview with an Anonymous Data Scientist (2016)
#100Good interview, there are a bunch of bits I feel like I ought to be Quoting For Truth but then I'd end up with a pretty bloated reply. > I want to emphasize that historically, from the very first moment somebody thought of computers, there has been a notion of: “Oh, can the computer talk to me, can it learn to love?” And somebody, some yahoo, will be like, “Oh absolutely!” And then a bunch of people will put money in…
For some devils advocacy... I remember hearing from some old salty in the oil business that geologists are the wrong people to ask about peak oil. They always understimated future discoveries. The ones that tended to get it right were finaciers and investors. The idea is that geologists have their noses down in the details of practical, useful knowledge that they have or can get. Financiers don't really know anything…
This reminds me of the inspiration for the name of Taleb's "green lumber fallacy". From Wikipedia:
The term green lumber refers to a story by authors Jim Paul and Brendan Moynihan in their book What I Learned Losing A Million Dollars, where a trader made a fortune trading lumber he thought was literally "green" rather than fresh cut.[26] "This gets at the idea that a supposed understanding of an investment rationale, a narrative or a theoretical model is unhelpful in practical trading."[27]
The protagonist makes a big discovery. He remarks that a fellow named Joe Siegel, one of the most successful traders in a commodity called "green lumber," actually thought that it was lumber painted green (rather than freshly cut lumber, called green because it had not been dried). And he made it his profession to trade the stuff! Meanwhile the narrator was into grand intellectual theories and narratives of what caused the price of commodities to move, and went bust. It is not just that the successful expert on lumber was ignorant of central matters like the designation "green." He also knew things about lumber that nonexperts think are unimportant. People we call ignorant might not be ignorant. The fact is that predicting the order flow in lumber and the usual narrative had little to do with the details one would assume from the outside are important. People who do things in the field are not subjected to a set exam; they are selected in the most nonnarrative manner—nice arguments don’t make much difference.[25]