Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A week-long programming retreat

facebook.com

Post reply on HN
Print view View original

131–140 of 158 posts

Re: A week-long programming retreat

#131

Post by tweek273 »

plg wrote: post #105

The idea of programming something from "scratch" (whatever your definition, and programming language) is the best way to really understand something new. Reading about it, hearing someone speak about it is one thing ... but opening up a blank .c file and adopting a "ok, let's get on with this" approach is something much different. It takes time though and one has to combat the "how come you're reinventing the wheel"…

This is the biggest challenge I face. As I attempt to teach myself computer science and programming, the toughest aspect is to work through SICP. I am always tempted to take the path of least resistance and follow a tutorial to build a tangible program that will impress. Must remember that this is a journey and to build brick by brick, even if that means gathering the ingredients for the clay, then mixing, then laying the bricks!

Re: A week-long programming retreat

#132

Post by Jach »

leblancfg wrote: post #82
michaelmcmillan wrote: post #53

Earlier quoted context omitted.

IQ tests seem to suggest otherwise.

To be fair, I've never heard of a focus test.

However you're defining focus I'm sure correlates pretty well with the personality trait conscientiousness. (IQ is the best single-variable predictor of nice things like life success, for various definitions of success, IQ + Conscientiousness predicts even better...)

Re: A week-long programming retreat

#133

Post by davidcuddeback »

Amezarak wrote: post #128
jorgemf wrote: post #94

Earlier quoted context omitted.

> not any sort of fundamental algorithmic paradigm shift I think I cannot agree with this. There has been a lot of improvements to the algorithms to solve problems and the pace has speed up thanks to GPUs. You just cannot make a neural network from 15 years ago bigger and think it is going to work with modern GPUs, it is not going to work at all. Moreover, new techniques have appeared to solve other type of problems.…

Do you have any suggestions on reading material?

Assuming you're asking specifically about the history of DL/ML, I can recommend this (4-part) blog series: http://www.andreykurenkov.com/writing/ai/a-brief-history-of-.... It includes references for the relevant publications that identify problems (e.g., exploding gradients) and their solutions.

Re: A week-long programming retreat

#134

Post by Jach »

swang wrote: post #42

I find it peculiar about people's continued good feelings towards Carmack. During the Zenimax/Oculus case: He claimed that he never wiped his hard drive: An independent court expert found that most of his hard drive was wiped after Carmack heard about the lawsuit. So Carmack lied in his affidavit. He claimed that no source code from Zenimax ever got transferred over to Oculus. Then later admitted that the emails he h…

Is this the same court expert who allegedly used unreadable code printouts (that nevertheless look different at a distance) to establish non-literal code copying and whose expert testimony is still under court seal instead of in the public record? It's hard to draw any inferences from that trial as an outsider. You'd do better to focus on the stories in Masters of Doom that show dubious ethics (like 'borrowing' work resources to work on games you later sell to get money and be free from the job where you were borrowing resources). Of course to some of us those ethics aren't really that dubious, it's the law that has dubious ethics.

Re: A week-long programming retreat

#135

Post by lazyasciiart »

sanxiyn wrote: post #57
edanm wrote: post #49

Earlier quoted context omitted.

I don't know anything, at all, about this case or much about him except the standard. But is there any point in bringing it up now? I mean, whether or not he did something illegal/unethical, does this now mean that literally every single time he writes something, someone feels the need to chime in with "but remember that bad thing he did once?". I'm not saying this isn't a legit conversation or that you're unethical…

I agree with you, but I also think there is some kind of double standard going on, say, compared with the case of Jacob Appelbaum, etc.

It's not a 'double standard' to treat different situations differently. If you personally want to consider sexual assault and IP theft as the same situation, then you can, but IMO (and I believe in the average worldview) sexual assault is worse.

Re: A week-long programming retreat

#136

Post by davidcuddeback »

chillee wrote: post #113
Humdeee wrote: post #108

Earlier quoted context omitted.

I am not in the AI space at all, but I am under the impression that python is the most used language for it. If workload is becoming an issue, wouldn't the low hanging fruit be a more performance driven software language? With how fast things are evolving, developing something like an ASIC ($$$) for this might be outdated before it even hits release, no?

The heavy lifting done in neural networks is all offloaded to C++ code and the GPU. Very little of the computation time is spent in python. ASICs will definitely be very helpful, and there's currently a bit of a rush to develop them. Google's TPUs might be one of the first efforts, but several other companies and startups are looking to have offerings too.

> Google's TPUs might be one of the first efforts, but several other companies and startups are looking to have offerings too.

NVidia's Volta series includes tensor cores as well [1]. So far, I think they've only released the datacenter version, which is available on EC2 p3 instances [2].

[1] https://www.nvidia.com/en-us/data-center/volta-gpu-architect...

[2] https://aws.amazon.com/ec2/instance-types/p3/#Product_Detail...

Re: A week-long programming retreat

#137

Post by IamNotAtWork »

Iv wrote: post #15
United857 wrote: post #4

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

In computer vision at least, deep learning has been a revolution. More than half of what I knew in the field became obsolete almost overnight (it took about a year or two I would say) and a lot of tasks received an immediate boost in term of performances. Yes, neural networks have been here for a while, gradually improving, but they were simply non-existent in many fields where they are now the favored solution. Ther…

Can you elaborate on what in CV became obsolete overnight? I took a survey course in CV but I haven't kept up. You still do facial detection, object recognization, camera calibration, image stitching the same way in 2012? Or has it changed because the processing has gotten faster and the results are near real-time?

Re: A week-long programming retreat

#138

Post by psyc »

bsenftner wrote: post #99
psyc wrote: post #34

Earlier quoted context omitted.

Game engine programmer here. I came up on DOS and Windows, and for a long time that's what I was comfortable with, and everything else I presumed to be alien. Once I'd worked on things that were both cross-platform and low-level, the idea that different hardware platforms and OSes were different went out the window somewhat. It was literally enlightening. In the old days, we had to be mercenaries. SNES is popular now…

There really is only a few post-game programmer job that will use the skills you've developed creating games: science and new technology development. That is where you'll find developers and dev shops that respect coding without frameworks (because the platform is so new there are none) or without the popular, latest dev tool toys. Working in such a shop is often supremely satisfying because your job is to make that…

Great advice! I did front and back end web dev at Microsoft for a few years. One of many nice things about working there was I, personally, didn’t get typecast too much. Though, ironically, after my web stint, some game teams didn’t want me anymore :( But I eventually did find a role in HoloLens.

Re: A week-long programming retreat

#139

Post by test6554 »

jorgemf wrote: post #123
tostitos1979 wrote: post #116

Earlier quoted context omitted.

Yeah .. but his first impulse was to write backprop from scratch. I saw the lectures, been dabbling with NN for years, and I never thought to do it. I always thought the Stanford people made you do it on assn 1 to pay your dues or something. I continue to think of Carmack as the Master hacker.

> his first impulse was to write backprop from scratch backprop is a very simple algorithm, nothing to fear there. The problems are to calculate the derivates if you want to be flexible building your model. But for feedforward networks with sigmoid activation, the equations to update the weights are a joke.

It's been a while, but I remember backprop starting at the end of the neural net, and working backwards. Each weight that contributed to a wrong answer had its weight value weakened or even reversed by some small factor. And each weight that contributed to a correct answer had its weight value strengthened by some small factor.

So it's probably as simple as

newWeight = oldWeight +/- (stepValue * someFactor)

Re: A week-long programming retreat

#140

Post by _mhr_ »

colmvp wrote: post #68
arnioxux wrote: post #19

I second his recommendation of CS231N: http://cs231n.stanford.edu/ You can probably go through the whole thing including assignments in under a week full-time.

Personally, I found it took me much longer. I watched Karpathy's lectures, took notes and stewed upon the ideas, and read a bunch of other materials such as blog posts and research papers to try and truly comprehend some of the concepts mentioned in the course. I found myself knowing how to create CNN's, but the why of the entire process still feels under-developed. But I'll admit it could be because my understanding…

What did you read/do to develop your calc and linear algebra skills? I feel like I know calc and linear algebra fairly well on paper, but I'm unsure how to translate that to the computer.
Post reply on HN