A week-long programming retreat • Hacker News Board

>On some level, I suspect that Deep Learning being so trendy tweaked a little bit of contrarian in me, and I still have a little bit of a reflexive bias against “throw everything at the NN and let it sort it out!”

I am the same kind of person. But when John Carmack approaches this with scepticism and concludes that it is indeed not over-hyped, I guess its worth learning after all!

CS231N, here I come.

Re: A week-long programming retreat

#15

Post by Iv » Mon, Mar 05, 2018, 6:49 AM UTC

United857 wrote: post #4

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

In computer vision at least, deep learning has been a revolution. More than half of what I knew in the field became obsolete almost overnight (it took about a year or two I would say) and a lot of tasks received an immediate boost in term of performances.

Yes, neural networks have been here for a while, gradually improving, but they were simply non-existent in many fields where they are now the favored solution.

There WAS a big fundamental paradigm shift in algorithmic. Many people argue that it should not be called "neural networks" but rather "differentiable functions networks". DL is not your dad's neural network, even if it looks superficially similar.

The shift is that now, if you can express your problem in terms of minimization of a continuous function, there is a new whole zoo of generic algorithms that are likely to perform well and that may benefit from throwing more CPU resources.

Sure it uses transistors in the end, but revolutions do not necessarily mean a shift in hardware technology. And, by the way, if we one day switch from transistors to things like opto-thingies, if it brings a measely 10x boost on performances, it won't be on par with the DL revolution we are witnessing.

Re: A week-long programming retreat

#16

Post by pm » Mon, Mar 05, 2018, 6:50 AM UTC

deepaksurti wrote: post #8

>> I’m not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. This is a lesson for me and probably many others. Don't get hung up on tools, ship!!!

I find this interesting since I used to get so excited as a teenager when one of the id peeps updated their .plan file.

Re: A week-long programming retreat

#17

Post by Wevah » Mon, Mar 05, 2018, 6:50 AM UTC

brynet wrote: post #6

Cieplak wrote: post #5

Interesting to hear that C++ isn’t that well supported on OpenBSD. The story is quite the opposite with FreeBSD, where it’s really easy to use either clang or gcc. I usually spin up new jails to keep different installations sandboxed. CLion works quite nicely with most window managers on FreeBSD, but I rarely boot Xwindows these days and usually prefer to work with emacs inside tmux from the console.

C++ support is pretty good with base Clang 5, OpenBSD is using the libc++ standard library and libc++abi (FreeBSD uses PathScale's libcxxrt). The situation with debuggers is another story, the base version of gdb is outdated (GPLv2 version). The ports/packages version (egdb) has better support for things like C++, threads, but may lack other features. There is currently no support for the LLVM debugger, sadly.

I wondered why he didn’t use LLDB; thanks.

Re: A week-long programming retreat

#18

Post by jakecrouch » Mon, Mar 05, 2018, 6:51 AM UTC

United857 wrote: post #4

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

The best way to advance AI is probably to make the hardware faster, especially now that Moore's Law is in danger of going away. The people doing AI research generally seem to be fumbling around in the dark, but you can at least be certain that better hardware would make things easier.

Re: A week-long programming retreat

#19

Post by arnioxux » Mon, Mar 05, 2018, 6:55 AM UTC

I second his recommendation of CS231N: http://cs231n.stanford.edu/

You can probably go through the whole thing including assignments in under a week full-time.

Re: A week-long programming retreat

#20

Post by mdale » Mon, Mar 05, 2018, 7:08 AM UTC

Iv wrote: post #15

United857 wrote: post #4

Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…

In computer vision at least, deep learning has been a revolution. More than half of what I knew in the field became obsolete almost overnight (it took about a year or two I would say) and a lot of tasks received an immediate boost in term of performances. Yes, neural networks have been here for a while, gradually improving, but they were simply non-existent in many fields where they are now the favored solution. Ther…

Yea .. if 10x compute made all the difference we would see elastic cloud compute doing things 10x as well ;) As always these new application work in tandem of hardware and software advances; seems odd to point to one or the other.