Live data from Hacker News

A week-long programming retreat

facebook.com

71–80 of 158 posts

Re: A week-long programming retreat

#71
post #69

Earlier quoted context omitted.

New regularization techniques to stop these massive NNs from overfitting is also one of the main factors for the current deep learning explosion.

Which regularisation techniques have you got in mind?

i'd say dropout, batch norm

Re: A week-long programming retreat

#73

Pretty awesome! If I ever had to say the one thing that differentiates successful people from unsuccessful people it wouldn't be intelligence, or even perseverance, or passion. It'd be focus. With focus, you can be amazingly successful in so many types of occupations. (That being said, passion / perseverance / intelligence can often lead to focus)

> (That being said, passion / perseverance / intelligence can often lead to focus)

Arent those more of a prerequisites for focus? Focus is merely a side effect not something you can aim for.

Re: A week-long programming retreat

#74

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

There was a revolution, when they started using backpropogation to optimize the gradient search. It's also why I don't agree with calling them "neural" anything because there is no proof brains learn using backpropogation. I feel like current direction threw away all neurobiology and focus too much on the mathematical.

Re: A week-long programming retreat

#75

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

There was a revolution, when they started using backpropogation to optimize the gradient search. It's also why I don't agree with calling them "neural" anything because there is no proof brains learn using backpropogation. I feel like current direction threw away all neurobiology and focus too much on the mathematical.

Backpropagation was first applied to Neural Networks over 30 years ago

[Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature]

Re: A week-long programming retreat

#76
post #56

Earlier quoted context omitted.

I 100% agree that hackers should be judged by his/her hacking. On the other hand, I found that many people consider me wrong when I judge a hacker by his/her hacking when he/she is accused of sexual abuse. But apparently they are okay with someone accused of IP crime.

Sexual abuse and IP crime are very different in terms of their stigma, and rightly so, in my opinion, so I don't see what's surprising about your final paragraph.

Surprising part is that even on HN people disagree with "hackers should be judged by his/her hacking".

Re: A week-long programming retreat

#77
post #42

I find it peculiar about people's continued good feelings towards Carmack. During the Zenimax/Oculus case: He claimed that he never wiped his hard drive: An independent court expert found that most of his hard drive was wiped after Carmack heard about the lawsuit. So Carmack lied in his affidavit. He claimed that no source code from Zenimax ever got transferred over to Oculus. Then later admitted that the emails he h…

He's always been one of my programming heroes. I don't know what he did or didn't do in this case. But, if he did lie, that would bring him down a notch in my mind. However, if there's anyone who could quickly rewrite a system like that, it's definitely Carmack.

Anyway, I will probably always read or listen to anything he has to say on the subject of programming.

Re: A week-long programming retreat

#78
post #64
post #51

Earlier quoted context omitted.

Yeah, Carmack has a history of unethical behavior. From the book "Masters of Doom": > Late one night Carmack and his friends snuck up to a nearby school where they knew there were Apple II machines. Carmack had read about how a thermite paste could be used to melt through glass, but he needed some kind of adhesive material, like Vaseline. He mixed the concoction and applied it to the window, dissolving the glass so t…

Carmack: Breaks into a building at age 14, steals a bit of code today. "Oh, he has a history of unethical behavior, we shouldn't look up to him." Bill Gates: Steals a bulldozer to race with his buddies at age 19(?), is responsible for Microsoft's corporate culture and history of predatory behavior. Hackernews loves him and can't get enough of Microsoft. Boy, they're a fair sight better than Google, eh lads? I pity th…

Bill Gates has more money and prestige. This is HN where the predominate idea is to start a new business and become the new Bill Gates.

Re: A week-long programming retreat

#79
post #75

Earlier quoted context omitted.

There was a revolution, when they started using backpropogation to optimize the gradient search. It's also why I don't agree with calling them "neural" anything because there is no proof brains learn using backpropogation. I feel like current direction threw away all neurobiology and focus too much on the mathematical.

Backpropagation was first applied to Neural Networks over 30 years ago [Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature]

Not to deep nets iirc, which was one of the advancements.

Re: A week-long programming retreat

#80
post #56
post #52

Earlier quoted context omitted.

How does this comment critique the posted write up? People fawn over John because he shows a great spirit of the hacker. A hacker should be judged by his/her hacking.

I 100% agree that hackers should be judged by his/her hacking. On the other hand, I found that many people consider me wrong when I judge a hacker by his/her hacking when he/she is accused of sexual abuse. But apparently they are okay with someone accused of IP crime.

You're not actually equating sex crimes and IP theft are you?
Post reply on HN