31–40 of 158 posts
Hands up for everyone who expected John to announce the first self-aware AI at the end.
Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…
Traditionally, neural networks were trained by back propagations, but many implementations had only one or two hidden layers because training a neural network with many layers (it wasn't called "deep" NN back then) was not only hard, but often led to poorer results. Hochreiter identified the reason in his 1991 thesis: it is because of the vanishing gradient problem. Now the culprit was identified but the solution had yet to be found.
My impression is that there weren't any breakthroughs until several years later. Since I'd left that field, I don't know what exactly these breakthroughs were. Apparently, the invention of LTSM networks, CNNs and the replacement of sigmoids by ReLUs were some important contributions. But anyway, the revolution was more about algorithmic improvement than the use of GPUs.
Compatible with Firefox Reader View: https://pastebin.com/raw/hZ32q3Rv
>> I’m not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. This is a lesson for me and probably many others. Don't get hung up on tools, ship!!!
That attitude is quite prevalent on the games industry, hence anyone that would spend some time among AAA devs would learn that they don't care one second about stuff like portable 3D APIs and OSes, as people in forums like HN think they do. What matters is making a cool game with whatever tech is available and ship.
This perspective makes it supremely frustrating anytime I try to get a 'normal' programming job, and run into the 'but do you have .NET4, ASP.NET Core, and JS6??' mentality.
>> I’m not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. This is a lesson for me and probably many others. Don't get hung up on tools, ship!!!
He took a week off to focus on a problem set, and ended up spending a number of hours working around the limitations of his setup instead.
I mean, I don't want to sound judgmental. Perhaps that was part of his plan. It just stuck out to me as seeming orthogonal to his expressed goals.
The man is a genius but why openBSD? FreeBSD would have been my choice.
Earlier quoted context omitted.
> The question becomes; if its not C++, then what is the first class citizen on OpenBSD. Being an UNIX derivative the answer is quite easy, C.
Sure, but is there a popular, modern C compiler out there that doesn't compile C++ as well? Clang, GCC, Intel, MSVC... all of them compile C++ as well as C. I don't think it's possible to have first-hand support for C and not C++ any more unless you ship your own compiler. It sounds like the issue here is that OpenBSD is shipping older versions of LLVM/Clang and GCC.
I was just speaking from language point of view, without regard to specific implementations.
Earlier quoted context omitted.
That attitude is quite prevalent on the games industry, hence anyone that would spend some time among AAA devs would learn that they don't care one second about stuff like portable 3D APIs and OSes, as people in forums like HN think they do. What matters is making a cool game with whatever tech is available and ship.
Game engine programmer here. I came up on DOS and Windows, and for a long time that's what I was comfortable with, and everything else I presumed to be alien. Once I'd worked on things that were both cross-platform and low-level, the idea that different hardware platforms and OSes were different went out the window somewhat. It was literally enlightening. In the old days, we had to be mercenaries. SNES is popular now…
John has been experimenting with a lot of stuff -- Racket, Haskell, Computer Vision and now Neural Networks. I guess there is no professional intent, but the spirit of hacking lives on.
https://www.recurse.com/blog/127-a-new-way-to-join-the-rc-co...