Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A week-long programming retreat

facebook.com

Post reply on HN
Print view View original

151–158 of 158 posts

Re: A week-long programming retreat

#151

Post by gwern »

BenoitEssiambre wrote: post #88

"I initially got backprop wrong both times, comparison with numerical differentiation was critical! It is interesting that things still train even when various parts are pretty wrong — as long as the sign is right most of the time, progress is often made." That is the bane of doing probabilistic code. Errors show up not as clear cut wrong values or crashes but as subtle biases. You are always wondering, even when it…

There might be something deeper there. I am thinking of the line of research, associated with Bengio, about biologically-plausible backprop - it turns out that you can backprop random weights and backprop will still work! Which is important because it's not too plausible that the brain is calculating exact derivatives and communicating them around to each neuron individually to update them, but it can send back error more easily.

Re: A week-long programming retreat

#152

Post by Iv »

Cacti wrote: post #118
Iv wrote: post #15

Earlier quoted context omitted.

In computer vision at least, deep learning has been a revolution. More than half of what I knew in the field became obsolete almost overnight (it took about a year or two I would say) and a lot of tasks received an immediate boost in term of performances. Yes, neural networks have been here for a while, gradually improving, but they were simply non-existent in many fields where they are now the favored solution. Ther…

It is indeed pretty amazing in CV. In a field nearly as old as CS itself, I'd say at least 75% of the existing techniques were made obsolete in a span of only a few years. You could have started your PhD in 2006, made professor in 2012, and nearly everything you had learned would have been _completely_ different.

I got my diploma in 2003. Actually I got lucky that I found a client who needed a computer vision specialist to add fancy features to their DL framework so I could train myself in this new direction

Re: A week-long programming retreat

#153

Post by Iv »

IamNotAtWork wrote: post #137
Iv wrote: post #15

Earlier quoted context omitted.

In computer vision at least, deep learning has been a revolution. More than half of what I knew in the field became obsolete almost overnight (it took about a year or two I would say) and a lot of tasks received an immediate boost in term of performances. Yes, neural networks have been here for a while, gradually improving, but they were simply non-existent in many fields where they are now the favored solution. Ther…

Can you elaborate on what in CV became obsolete overnight? I took a survey course in CV but I haven't kept up. You still do facial detection, object recognization, camera calibration, image stitching the same way in 2012? Or has it changed because the processing has gotten faster and the results are near real-time?

We used to make features detectors manually

E.g. https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature...

These were at the root of many detectors. They still are for some applications but for most of them, a few layers of CNN manage to train far better and very counter-intuitive detectors.

Facial detection/recognition was based on features, this is not my specialty, I don't know if DL got better there too as their features were pretty advanced but if they are not there yet I am sure it is just a matter of time.

I can see image stitching benefiting from a deep-learned filter pass too.

Camera calibration is pretty much a solved problem by now, I don't think DL adds a lot to it.

Like I said, not everything became obsolete, but around 50% of the field was taken over my DL algorithms where, before that, hand-crafted algorithms had usually vastly superior performances.

Re: A week-long programming retreat

#154

Post by mcphage »

sanxiyn wrote: post #56
z3phyr wrote: post #52

Earlier quoted context omitted.

How does this comment critique the posted write up? People fawn over John because he shows a great spirit of the hacker. A hacker should be judged by his/her hacking.

I 100% agree that hackers should be judged by his/her hacking. On the other hand, I found that many people consider me wrong when I judge a hacker by his/her hacking when he/she is accused of sexual abuse. But apparently they are okay with someone accused of IP crime.

> I found that many people consider me wrong when I judge a hacker by his/her hacking when he/she is accused of sexual abuse. But apparently they are okay with someone accused of IP crime.

Yup! Why do you suppose that might be?

Re: A week-long programming retreat

#155

Post by wittjeff »

nemo1618 wrote: post #92
yitchelle wrote: post #59

Earlier quoted context omitted.

Agreed, I guess that the trick to selecting a location is to be in a location the the inside the hotel/cabin room is much more attractive that on the outside. I had a colleague that told me that his most productive period was when he was stuck at the hospital for a couple of weeks but was able to do some coding.

There was a post on HN years ago advocating coding on a cruise ship -- it's like a nice hotel, but the internet is so crappy that you'll only use it when you really need it (e.g. finding docs, syncing git), which is great for productivity!

Post may have been by Tynan (http://tynan.com); he's big on working while on transatlantic cruises. Following his advice, I have done the same thing several times.

Here are some tips: - Find the best cruises most easily on cruisesheet.com (disclosure: it's Tynan's project)

- Royal Caribbean has the best internet at sea through O3b. In the Caribbean or Mediterranean it's about 70ms latency. In the middle of the Atlantic it's about 220ms latency. So Skype may work but the delays are annoying. Plenty of bandwidth now though. You may have only "one 9" service on average though. So scheduled conference calls are always a gamble.

- Repositioning cruises (many ships move from the Mediterranean to the Caribbean and back seasonally) in April/May and October/November are the cheapest cruises you'll ever find. 5-8 days at sea means plenty of time to get some work done, as well as goof off a bit during the evenings. For the second week (Europe or Caribbean), I find it easy enough to get an hour or two of email catch-up in on port days after getting back from a shore excursion, but it's easier to just say port day = vacation, sea day = work and great food.

A 2-week repositioning cruise may cost as little as $600 per person including taxes and fees. Add another $200pp for gratuities, a few $hundred for shore excursions, and a few $hundred for airfare to get back home. Depending on where you go, you may get the benefits of a 2-week vacation for the price of one on land.

Similarly, if you want to do a one-week offsite with your startup (particularly if you're all remote most of the time anyway), this is probably cheaper than flying your crew to any big city.

The seminars-at-sea model was also well proven out by geekcruises.com, now renamed insightcruises.com

My wife and I have been renting an off-season beach house near Boston for 8 months out of the year for less than the price of a basement studio in Cambridge, then traveling during the summer. This means our repositioning cruises are further discounted by the fact that we aren't paying rent or mortgage on an empty house. It's hard to beat if you both have good schedule flexibility.

Re: A week-long programming retreat

#157

Post by st1ck »

tweek273 wrote: post #131
plg wrote: post #105

The idea of programming something from "scratch" (whatever your definition, and programming language) is the best way to really understand something new. Reading about it, hearing someone speak about it is one thing ... but opening up a blank .c file and adopting a "ok, let's get on with this" approach is something much different. It takes time though and one has to combat the "how come you're reinventing the wheel"…

This is the biggest challenge I face. As I attempt to teach myself computer science and programming, the toughest aspect is to work through SICP. I am always tempted to take the path of least resistance and follow a tutorial to build a tangible program that will impress. Must remember that this is a journey and to build brick by brick, even if that means gathering the ingredients for the clay, then mixing, then layin…

I'm not really sure it's the most effective learning paradigm. It must be easier to just copy-paste trivial programs and then try to modify them, gradually increasing complexity level, until you get certain level of understanding.

Re: A week-long programming retreat

#158

Post by JacobiX »

Iv wrote: post #153
IamNotAtWork wrote: post #137

Earlier quoted context omitted.

Can you elaborate on what in CV became obsolete overnight? I took a survey course in CV but I haven't kept up. You still do facial detection, object recognization, camera calibration, image stitching the same way in 2012? Or has it changed because the processing has gotten faster and the results are near real-time?

We used to make features detectors manually E.g. https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature... These were at the root of many detectors. They still are for some applications but for most of them, a few layers of CNN manage to train far better and very counter-intuitive detectors. Facial detection/recognition was based on features, this is not my specialty, I don't know if DL got better there too as…

Just to confirm for the facial recognition/ detection, modern DNN algorithms outperform the 'classic' methods that took decades of continuous improvement ...
Post reply on HN