A week-long programming retreat • Hacker News Board

Re: A week-long programming retreat

#91

Post by AboutTheWhisles » Mon, Mar 05, 2018, 3:28 PM UTC

swang wrote: post #42

I find it peculiar about people's continued good feelings towards Carmack. During the Zenimax/Oculus case: He claimed that he never wiped his hard drive: An independent court expert found that most of his hard drive was wiped after Carmack heard about the lawsuit. So Carmack lied in his affidavit. He claimed that no source code from Zenimax ever got transferred over to Oculus. Then later admitted that the emails he h…

> He claimed that no source code from Zenimax ever got transferred over to Oculus. Then later admitted that the emails he had taken form his Zenimax laptop on his last day there did contain source code.

There is a big difference between wrapping up a source control tree and having fragments in emails.

Also let's not pretend that the idea that you can remember something and re-type it is fine, but if you copy and paste it from an email, that's terrible. The line is pretty blurry.

Re: A week-long programming retreat

#92

Post by nemo1618 » Mon, Mar 05, 2018, 3:36 PM UTC

yitchelle wrote: post #59

qychtkd wrote: post #22

Master of Doom contains an anecdote on this aspect of John Carmack's life. On page 252, it mentioned how he had sequestered himself in a "small, anonymous hotel room somewhere in Florida" as he researched Trinity. He had a dolch portable computer with pentium II and full length PCI slots, while subsisting only on pizza and Diet Coke. That bit for some reason made a big impression on me when I read it on the bus ride…

Agreed, I guess that the trick to selecting a location is to be in a location the the inside the hotel/cabin room is much more attractive that on the outside. I had a colleague that told me that his most productive period was when he was stuck at the hospital for a couple of weeks but was able to do some coding.

There was a post on HN years ago advocating coding on a cruise ship -- it's like a nice hotel, but the internet is so crappy that you'll only use it when you really need it (e.g. finding docs, syncing git), which is great for productivity!

Re: A week-long programming retreat

#93

Post by pedrosorio » Mon, Mar 05, 2018, 3:41 PM UTC

nightski wrote: post #79

eliben wrote: post #75

Earlier quoted context omitted.

Backpropagation was first applied to Neural Networks over 30 years ago [Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature]

Not to deep nets iirc, which was one of the advancements.

The GP implies that was not tried before: "There was a revolution, when they started using backpropogation to optimize the gradient search".

"Back-propagation allowed researchers to train supervised deep artificial neural networks from scratch, initially with little success. Hochreiter's diploma thesis of 1991[1][2] formally identified the reason for this failure in the "vanishing gradient problem", which not only affects many-layered feedforward networks,[3] but also recurrent networks."

https://en.m.wikipedia.org/wiki/Vanishing_gradient_problem

Re: A week-long programming retreat

#94

Post by jorgemf » Mon, Mar 05, 2018, 4:06 PM UTC

United857 wrote: post #4

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

> not any sort of fundamental algorithmic paradigm shift

I think I cannot agree with this. There has been a lot of improvements to the algorithms to solve problems and the pace has speed up thanks to GPUs. You just cannot make a neural network from 15 years ago bigger and think it is going to work with modern GPUs, it is not going to work at all. Moreover, new techniques have appeared to solve other type of problems.

I am talking about things like batch normalization, RELUs, LSTMs or GANs. Yes, neural networks still use gradient descent, but there are people working on other algorithms now and they seem to work but they are just less efficient.

> This is similar to how the integrated circuit enabled the personal computing "revolution" but down at the transistor level, it's still using the same principles of digital logic since the 1940s.

This claim has exactly the same problem as before. You can also say evolution has done nothing because the same principles that are in people they were there with the dinosaurs and even with the first cells. We are just a lot more cells than before.

Re: A week-long programming retreat

#95

Post by bringtheaction » Mon, Mar 05, 2018, 4:13 PM UTC

forgotmypw wrote: post #62

No-JS link: https://m.facebook.com/permalink.php?story_fbid=211040872252... John Carmack After a several year gap, I finally took another week-long programming retreat, where I could work in hermit mode, away from the normal press of work. My wife has been generously offering it to me the last few years, but I’m generally bad at taking vacations from work. As a change of pace from my current Oculus work, I wanted to…

Better non-JS link: http://archive.is/MvKHy. This link is of a snapshot of the DOM of rendered page. The one you linked has a not-so-comfortable layout due to being made for old phones with small screens.

Re: A week-long programming retreat

#96

Post by Jare » Mon, Mar 05, 2018, 4:26 PM UTC

thaumasiotes wrote: post #48

swang wrote: post #42

I find it peculiar about people's continued good feelings towards Carmack. During the Zenimax/Oculus case: He claimed that he never wiped his hard drive: An independent court expert found that most of his hard drive was wiped after Carmack heard about the lawsuit. So Carmack lied in his affidavit. He claimed that no source code from Zenimax ever got transferred over to Oculus. Then later admitted that the emails he h…

> So why do people still fawn over him when it seems like his ethics are dubious? > Why does he get a free pass on this? Because many people draw a distinction between something being "unethical" vs being "illegal". What exactly do you think is the problem with his ethics? How serious do you think it is?

It's often a mistake to infer someone's ethics from their apparent behaviour within the context of a legal battle. The consequences from even very minor details slipped or misinterpreted can be huge.

Re: A week-long programming retreat

#97

Post by habitue » Mon, Mar 05, 2018, 4:33 PM UTC

chooseaname wrote: post #81

michaelmcmillan wrote: post #53

Earlier quoted context omitted.

IQ tests seem to suggest otherwise.

Are these even legitimate? If a person can paint a masterpiece but can barely balance their checkbook, IQ tests won't show their mastery in painting, only their mediocrity in math.

The argument you're making has often been made as "there are different kinds of genius". The research seems to have debunked this. Rather than seeing multiple ways to be a genius, we see some general intelligence ability that helps you no matter what you're trying to do.

In the case of our painter, if we're talking about Michaelangelo or something, he'd likely come out with an excellent IQ.

As an aside, IQ tests don't try to make you do math problems and the like, it's all pattern matching kinds of questions where you learn all of the necessary context in the test itself. Which makes sense right, it's trying to measure your ability to learn and generalize, not what you happen to know before you take the test.

Re: A week-long programming retreat

#98

Post by kridsdale1 » Mon, Mar 05, 2018, 4:40 PM UTC

gonehome wrote: post #27

doomlaser wrote: post #7

I like his insight that a lot of base neural network implementation code is conceptually simple in the same way as writing a raytracer.

I think the point about NN and ray tracing being simple systems that allow for complex outcomes is something that seems to be a deeper truth about the universe. Stephen Wolfram and Max Tegmark both talk and write about this - it also shows up in old cellular automata like Conway’s game of life. It’s pretty cool that so much complexity can come from a few small rules or equations.

You could say that physics is a relatively simple program that allows for outcomes as complex as the universe.

Re: A week-long programming retreat

#99

Post by bsenftner » Mon, Mar 05, 2018, 4:43 PM UTC

psyc wrote: post #34

pjmlp wrote: post #23

Earlier quoted context omitted.

That attitude is quite prevalent on the games industry, hence anyone that would spend some time among AAA devs would learn that they don't care one second about stuff like portable 3D APIs and OSes, as people in forums like HN think they do. What matters is making a cool game with whatever tech is available and ship.

Game engine programmer here. I came up on DOS and Windows, and for a long time that's what I was comfortable with, and everything else I presumed to be alien. Once I'd worked on things that were both cross-platform and low-level, the idea that different hardware platforms and OSes were different went out the window somewhat. It was literally enlightening. In the old days, we had to be mercenaries. SNES is popular now…

There really is only a few post-game programmer job that will use the skills you've developed creating games: science and new technology development. That is where you'll find developers and dev shops that respect coding without frameworks (because the platform is so new there are none) or without the popular, latest dev tool toys. Working in such a shop is often supremely satisfying because your job is to make that new platform conform to modern development standards. A job an experienced game developer does on their own anyway.

Re: A week-long programming retreat

#100

Post by ryandrake » Mon, Mar 05, 2018, 4:48 PM UTC

tinderliker wrote: post #14

>On some level, I suspect that Deep Learning being so trendy tweaked a little bit of contrarian in me, and I still have a little bit of a reflexive bias against “throw everything at the NN and let it sort it out!” I am the same kind of person. But when John Carmack approaches this with scepticism and concludes that it is indeed not over-hyped, I guess its worth learning after all! CS231N, here I come.

I admit to having the same grouchy thoughts about machine learning and AI: When is this fad going to blow over and we can all get back to writing deterministic programs instead of collecting data and training models, or whatever it is this new breed is doing? Might be time to rethink and revive my curiosity.