Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Interview with an Anonymous Data Scientist (2016)

logicmag.io

Post reply on HN
Print view View original

71–80 of 108 posts

Re: An Interview with an Anonymous Data Scientist (2016)

#71

Post by azag0 »

aoki wrote: post #46
brucephillips wrote: post #27

Earlier quoted context omitted.

> If you ask folks in nonlinear optimization, they'll tell you that DL is not possible. I sincerely doubt anyone who knows more than one sentence about deep learning would say that, since deep learning doesn't claim to optimize.

i suspect that what he's referring to is that he's heuristically minimizing a somewhat arbitrary (loss) function in a million-ish dimensions using the simple variants of gradient descent that work under these conditions. it sounds far too WIBNI to produce good results reliably (in practice, let alone in theory). the landscape has so many stationary points at which to get stuck; why would you ever get good results? th…

I think this recent paper [1] sheds quite a bit of light on this.

[1] https://arxiv.org/abs/1703.00810v3

Re: An Interview with an Anonymous Data Scientist (2016)

#72

Post by Radim »

EdwardDiego wrote: post #22

Can anyone comment on his point about Spark's ML libs? I note that was from last year (about 2015 code), not sure what level of beta they were at, but yeah, I use it for batch processing, but have never used the ML aspects, so just curious. > And even up to last year, there’s just massive bugs in the machine learning libraries that come bundled with Spark. It’s so bizarre, because you go to Caltrain, and there’s a gi…

Getting better obviously, but the feet-on-the-ground experience for MLlib is still far from pleasant: hard to configure, hard to manage, hard to scale, hard to debug.

By way of anecdote, Spark's MLlib used to contain an implementation of word2vec that failed when used on more than 2 billion words (some arcane integer overflow). So much for scale!

As for performance, in 2016, the break-even point where a Spark cluster started being competitive with a single-machine implementation was around 12 Spark machines (a bit of a hindrance to rapid iterative development, which is the corner stone of R&D): https://radimrehurek.com/florence15.pdf

Re: An Interview with an Anonymous Data Scientist (2016)

#74

Post by jnbiche »

gaius wrote: post #70
perturbation wrote: post #62

I've been seeing nothing but negative, dismissive comments about data science on HN lately, which is really disappointing. There's definitely a lot of hype right now about DL, but almost all of my job does not deal with Big Data or Deep Learning, 'just' machine learning + stats + calc + scripting + data cleaning + deploying models. I think most people don't have big data (Amazon has an x1 with 4 TB of RAM, after all!…

I've been seeing nothing but negative, dismissive comments about data science on HN lately, which is really disappointing. There's definitely a lot of hype right now about DL, but almost all of my job does not deal with Big Data or Deep Learning, 'just' machine learning + stats + calc + scripting + data cleaning + deploying models. But, all those things people did in the '90's or even earlier. It was called "data war…

> seems noone is a programmer anymore either, they're all "senior certified enterprise solution architects"

To be fair, few of the "senior architects" I've worked with in big companies knew how to program very well.

Re: An Interview with an Anonymous Data Scientist (2016)

#75

Post by sundarurfriend »

It's an interesting read, though not very enlightening in terms of new information. It's same old pre-existing arguments put in a more informal, more directly honest package.

As another person who's seen robots fall over again and again and has a scope for the difficulty of the problem, I'd say there's also the risk of the day to day failures making us lose sight of the forest for the trees, with availability bias working against us.

Also,

> the Y Combinator autistic Stanford guy thing

> the Aspy worldview

It's a bit worrying that use of these terms has turned into a kind of slur, to lump a kind of imagined stunted-worldview with a medical diagnosis. Not particularly pissed that this guy used these, more worried about what it indicates - that these have become so common as to infiltrate friendly informal conversations from seemingly intelligent people.

Re: An Interview with an Anonymous Data Scientist (2016)

#76

Post by eksemplar »

comstock wrote: post #7

Any bets on when the current deep learning bubble is going to burst? It’s shocking to me how much technical people buy into this, how “this time it’s different” and AI isn’t “over-promising and substantially under-delivering” this time. Really odd to watch it come round again, when the reality is we’re more likely to see some near incremental progresses, partly fueled by more compute and algorithmic advances. Partly…

It won't. It's going to change the way we do public sector. Not so much because the tech is revolutionary but because the upper echelons of society are sold on it and are actually putting it to use.

Technically you could do a lot of the decision making it'll be doing with human made models and a lot of data, but the machine is cheaper and it's backed by consulting agencies.

RPA was the first indication. It's basically screenscraping and small bots, stuff that's been around for a long time, I mean, it's basically what people use to bot in video games. Yet it's become a multimillion dollar industry over the course of a few years because it caught the right drift.

Like RPA, machine learning isn't just hype. It actually does some things with data really well, and when you couple that with the fact that ministers want this tech, well, that's all you need.

Re: An Interview with an Anonymous Data Scientist (2016)

#77

Post by eksemplar »

comstock wrote: post #32
brucephillips wrote: post #28

Earlier quoted context omitted.

The big tech companies are demonstrably using deep learning to solve previously unsolvable problems. It's a significant advance. What's yet to be seen is if startups can profit from this advance, since it depends on massive data and compute.

AlphaGo is interesting. But what big new problems have been solved? (rather than incrementally improved).

This isn't related to alpha go as such, but we can now predict which citizens are going to need help raising their children by having a machine cross reference their case history with public records.

It's not legal yet, but it will be, because it will potentially save lives (and money).

Re: An Interview with an Anonymous Data Scientist (2016)

#78

Post by mfukar »

comstock wrote: post #32
brucephillips wrote: post #28

Earlier quoted context omitted.

The big tech companies are demonstrably using deep learning to solve previously unsolvable problems. It's a significant advance. What's yet to be seen is if startups can profit from this advance, since it depends on massive data and compute.

AlphaGo is interesting. But what big new problems have been solved? (rather than incrementally improved).

Well, "big new" sounds like a destructive qualification. Why can't it be "big" and "old"? https://arxiv.org/pdf/1712.01208.pdf

Re: An Interview with an Anonymous Data Scientist (2016)

#79

Post by likelynew »

carlsborg wrote: post #18

It was 2016 and he said "I’ve noticed on AWS prices was that a few months ago, the spot prices on their GPU compute instances were $26 an hour for a four-GP machine, and $6.50 an hour for a one-GP machine. That’s the first time I’ve seen a computer that has human wages.." Minimum wage (or thereabouts $7.20) now gets you a whopping p2.8xlarge (8 GPU, 32 vcpus, 488GB RAM), and the single GPU machine p2.xlarge is now $0…

g3.xlarge is many times faster and spot prices are like 0.5$ per hour.

Re: An Interview with an Anonymous Data Scientist (2016)

#80

Post by lou1306 »

wolfgke wrote: post #11
Terr_ wrote: post #4

Good interview, there are a bunch of bits I feel like I ought to be Quoting For Truth but then I'd end up with a pretty bloated reply. > I want to emphasize that historically, from the very first moment somebody thought of computers, there has been a notion of: “Oh, can the computer talk to me, can it learn to love?” And somebody, some yahoo, will be like, “Oh absolutely!” And then a bunch of people will put money in…

> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. Luckily math has developed methods such as error-detecting/error-correcting codes…

And yet the Schiaparelli lander crashed because the machine couldn't give the right answer to a question that was wrong.

All these solutions are good for a noisy input, but have no use when the input is incorrect (ie. doesn't match reality).

Post reply on HN