Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A week-long programming retreat

facebook.com

Post reply on HN
Print view View original

121–130 of 158 posts

Re: A week-long programming retreat

#121

Post by balls187 »

panic wrote: post #51
swang wrote: post #42

I find it peculiar about people's continued good feelings towards Carmack. During the Zenimax/Oculus case: He claimed that he never wiped his hard drive: An independent court expert found that most of his hard drive was wiped after Carmack heard about the lawsuit. So Carmack lied in his affidavit. He claimed that no source code from Zenimax ever got transferred over to Oculus. Then later admitted that the emails he h…

Yeah, Carmack has a history of unethical behavior. From the book "Masters of Doom": > Late one night Carmack and his friends snuck up to a nearby school where they knew there were Apple II machines. Carmack had read about how a thermite paste could be used to melt through glass, but he needed some kind of adhesive material, like Vaseline. He mixed the concoction and applied it to the window, dissolving the glass so t…

Doing something stupid at Age 14 constitutes a history of unethical behavior?

Re: A week-long programming retreat

#122

Post by zombieprocesses »

plg wrote: post #105

The idea of programming something from "scratch" (whatever your definition, and programming language) is the best way to really understand something new. Reading about it, hearing someone speak about it is one thing ... but opening up a blank .c file and adopting a "ok, let's get on with this" approach is something much different. It takes time though and one has to combat the "how come you're reinventing the wheel"…

Absolutely. There is no substitute for walking the walk.

Also, if it is intimidating to do a whole project from scratch, break it down into parts and tackle those first.

Small successes in the beginning can be the boost you need to finish the project.

Re: A week-long programming retreat

#123

Post by jorgemf »

tostitos1979 wrote: post #116
rhacker wrote: post #103

It's actually super refreshing learning that even programming masters like Carmack are just now learning NNs and watching Youtube Stanford classes like the rest of us. These are actual people, not gods :) Everybody poops!

Yeah .. but his first impulse was to write backprop from scratch. I saw the lectures, been dabbling with NN for years, and I never thought to do it. I always thought the Stanford people made you do it on assn 1 to pay your dues or something. I continue to think of Carmack as the Master hacker.

> his first impulse was to write backprop from scratch

backprop is a very simple algorithm, nothing to fear there. The problems are to calculate the derivates if you want to be flexible building your model. But for feedforward networks with sigmoid activation, the equations to update the weights are a joke.

Re: A week-long programming retreat

#124

Post by BenoitEssiambre »

hotmilk wrote: post #104
BenoitEssiambre wrote: post #88

"I initially got backprop wrong both times, comparison with numerical differentiation was critical! It is interesting that things still train even when various parts are pretty wrong — as long as the sign is right most of the time, progress is often made." That is the bane of doing probabilistic code. Errors show up not as clear cut wrong values or crashes but as subtle biases. You are always wondering, even when it…

Reproducing known results goes a long way.

That is only possible when you are not experimenting with new algorithms.

Re: A week-long programming retreat

#125

Post by neiled »

jorgemf wrote: post #123
tostitos1979 wrote: post #116

Earlier quoted context omitted.

Yeah .. but his first impulse was to write backprop from scratch. I saw the lectures, been dabbling with NN for years, and I never thought to do it. I always thought the Stanford people made you do it on assn 1 to pay your dues or something. I continue to think of Carmack as the Master hacker.

> his first impulse was to write backprop from scratch backprop is a very simple algorithm, nothing to fear there. The problems are to calculate the derivates if you want to be flexible building your model. But for feedforward networks with sigmoid activation, the equations to update the weights are a joke.

I'm not sure we agree on the definition of very simple. It maybe very simple if you already know it...

Re: A week-long programming retreat

#126

Post by The_suffocated »

chillee wrote: post #112
The_suffocated wrote: post #32

Earlier quoted context omitted.

I don't think the revolution was about hardware improvement. I did some neural network research (and published a few papers) in the 1990s and switched to other research disciplines afterwards. So, I'm not really familiar with the recent developments. But to my knowledge, there was indeed a revolution in neural network research. It was about how to train a DEEP neural network. Traditionally, neural networks were train…

The things that have most improved training neural networks since you left were: 1. Smart (Xavier/He) initialization 2. ReLU activations 3. Batch normalization 4. Residual connections The GPUs and dataset size were definitely very important though.

Thanks for the info. I may want to keep up with the recent advances some day.

Re: A week-long programming retreat

#127

Post by mxschumacher »

munificent wrote: post #109
deepaksurti wrote: post #8

>> I’m not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. This is a lesson for me and probably many others. Don't get hung up on tools, ship!!!

I think the subtext of your comment here is something like, "Look how productive he is without mastering the real tools we Unix hackers use! He can get by with second rate Windows stuff!" Maybe I'm reading you wrong. But if I am right, it's good to take a look outside of the Unix bubble. Visual Studio is literally the world's most sophisticated developer tool. More human hours of engineering have been poured into it…

does it beat intellj's IDEs?

Re: A week-long programming retreat

#128

Post by Amezarak »

jorgemf wrote: post #94
United857 wrote: post #4

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

> not any sort of fundamental algorithmic paradigm shift I think I cannot agree with this. There has been a lot of improvements to the algorithms to solve problems and the pace has speed up thanks to GPUs. You just cannot make a neural network from 15 years ago bigger and think it is going to work with modern GPUs, it is not going to work at all. Moreover, new techniques have appeared to solve other type of problems.…

Do you have any suggestions on reading material?

Re: A week-long programming retreat

#129

Post by mcguire »

munificent wrote: post #109
deepaksurti wrote: post #8

>> I’m not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. This is a lesson for me and probably many others. Don't get hung up on tools, ship!!!

I think the subtext of your comment here is something like, "Look how productive he is without mastering the real tools we Unix hackers use! He can get by with second rate Windows stuff!" Maybe I'm reading you wrong. But if I am right, it's good to take a look outside of the Unix bubble. Visual Studio is literally the world's most sophisticated developer tool. More human hours of engineering have been poured into it…

I laughed when I read the tools he was using, fvwm and vi. Back in 1993 or so, I wrote a hundred thousand lines of filesystem performance benchmarks using those exact tools. They're surprisingly productive.

Re: A week-long programming retreat

#130

Post by twic »

dopeboy wrote: post #40

The recurse center (formerly known as Hackerschool) is offering one week mini retreats (their typical program is months long). I applied and didn't get in but after reading this, I may try again. https://www.recurse.com/blog/127-a-new-way-to-join-the-rc-co...

Reminds me of /dev/fort:

https://devfort.com/

Post reply on HN