Earlier quoted context omitted.
New regularization techniques to stop these massive NNs from overfitting is also one of the main factors for the current deep learning explosion.
Which regularisation techniques have you got in mind?
A week-long programming retreat
71–80 of 158 posts
Re: A week-long programming retreat
#72Re: A week-long programming retreat
#73Pretty awesome! If I ever had to say the one thing that differentiates successful people from unsuccessful people it wouldn't be intelligence, or even perseverance, or passion. It'd be focus. With focus, you can be amazingly successful in so many types of occupations. (That being said, passion / perseverance / intelligence can often lead to focus)
Arent those more of a prerequisites for focus? Focus is merely a side effect not something you can aim for.
Re: A week-long programming retreat
#74Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…
Re: A week-long programming retreat
#75Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…
There was a revolution, when they started using backpropogation to optimize the gradient search. It's also why I don't agree with calling them "neural" anything because there is no proof brains learn using backpropogation. I feel like current direction threw away all neurobiology and focus too much on the mathematical.
[Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature]
Re: A week-long programming retreat
#76Earlier quoted context omitted.
I 100% agree that hackers should be judged by his/her hacking. On the other hand, I found that many people consider me wrong when I judge a hacker by his/her hacking when he/she is accused of sexual abuse. But apparently they are okay with someone accused of IP crime.
Sexual abuse and IP crime are very different in terms of their stigma, and rightly so, in my opinion, so I don't see what's surprising about your final paragraph.
Re: A week-long programming retreat
#77I find it peculiar about people's continued good feelings towards Carmack. During the Zenimax/Oculus case: He claimed that he never wiped his hard drive: An independent court expert found that most of his hard drive was wiped after Carmack heard about the lawsuit. So Carmack lied in his affidavit. He claimed that no source code from Zenimax ever got transferred over to Oculus. Then later admitted that the emails he h…
Anyway, I will probably always read or listen to anything he has to say on the subject of programming.
Re: A week-long programming retreat
#78Earlier quoted context omitted.
Yeah, Carmack has a history of unethical behavior. From the book "Masters of Doom": > Late one night Carmack and his friends snuck up to a nearby school where they knew there were Apple II machines. Carmack had read about how a thermite paste could be used to melt through glass, but he needed some kind of adhesive material, like Vaseline. He mixed the concoction and applied it to the window, dissolving the glass so t…
Carmack: Breaks into a building at age 14, steals a bit of code today. "Oh, he has a history of unethical behavior, we shouldn't look up to him." Bill Gates: Steals a bulldozer to race with his buddies at age 19(?), is responsible for Microsoft's corporate culture and history of predatory behavior. Hackernews loves him and can't get enough of Microsoft. Boy, they're a fair sight better than Google, eh lads? I pity th…
Re: A week-long programming retreat
#79Earlier quoted context omitted.
There was a revolution, when they started using backpropogation to optimize the gradient search. It's also why I don't agree with calling them "neural" anything because there is no proof brains learn using backpropogation. I feel like current direction threw away all neurobiology and focus too much on the mathematical.
Backpropagation was first applied to Neural Networks over 30 years ago [Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature]
Re: A week-long programming retreat
#80Earlier quoted context omitted.
How does this comment critique the posted write up? People fawn over John because he shows a great spirit of the hacker. A hacker should be judged by his/her hacking.
I 100% agree that hackers should be judged by his/her hacking. On the other hand, I found that many people consider me wrong when I judge a hacker by his/her hacking when he/she is accused of sexual abuse. But apparently they are okay with someone accused of IP crime.