Good writeup -- and one of the main reminders for me is this: People throw around words like "revolution" for the current deep-learning push. But it's worth remembering that the fundamental concepts of neural networks have been around for decades. The current explosion is due to breakthroughs in scalability and implementation through GPUs and the like, not any sort of fundamental algorithmic paradigm shift. This is s…
New regularization techniques to stop these massive NNs from overfitting is also one of the main factors for the current deep learning explosion.
Master of Doom contains an anecdote on this aspect of John Carmack's life. On page 252, it mentioned how he had sequestered himself in a "small, anonymous hotel room somewhere in Florida" as he researched Trinity. He had a dolch portable computer with pentium II and full length PCI slots, while subsisting only on pizza and Diet Coke.
That bit for some reason made a big impression on me when I read it on the bus ride to school. To be able to let yourself go and research and code what you truly believe in or curious or excited about (w/ room service and not have to clean up after yourself haha) seemed incredible.
I wonder if John still sticks to Florida, or if he goes to different places each year; in a city or just a hotel off of a highway or near an airport. My favorites have been Hyatt - Place Amsterdam Airport, - Regency Charles De Gaulle, and - Lake Tahoe. Something about sterile rooms, room service, a hotel near, but not too close to beautiful and historical landmarks just center you and allow you to think.
>> I’m not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. This is a lesson for me and probably many others. Don't get hung up on tools, ship!!!
That attitude is quite prevalent on the games industry, hence anyone that would spend some time among AAA devs would learn that they don't care one second about stuff like portable 3D APIs and OSes, as people in forums like HN think they do.
What matters is making a cool game with whatever tech is available and ship.
Interesting to hear that C++ isn’t that well supported on OpenBSD. The story is quite the opposite with FreeBSD, where it’s really easy to use either clang or gcc. I usually spin up new jails to keep different installations sandboxed. CLion works quite nicely with most window managers on FreeBSD, but I rarely boot Xwindows these days and usually prefer to work with emacs inside tmux from the console.
My gut feeling is(was) that C++ is a first class citizen on OpenBSD. Its interesting having my beliefs challenged like this from someone so influential. The question becomes; if its not C++, then what is the first class citizen on OpenBSD. And if it is C++ then how do we improve support? Just going to ports seems like a poor answer.
> The question becomes; if its not C++, then what is the first class citizen on OpenBSD.
Being an UNIX derivative the answer is quite easy, C.
I find it very admirable that he can sit down for a week and just focus on one main subject. Personally, I get derailed all the time when I don't have a very well defined goal in mind.
It looks a little something like this: I'll be reading a manpage and notice another manpage referenced at the bottom. So I obviously keep crawling this suggestions tree until I bump into a utility whose purpose is unclear. So then I'll go searching online to try and figure out what kinds of problems or use-cases it's meant to help with.
Pretty awesome! If I ever had to say the one thing that differentiates successful people from unsuccessful people it wouldn't be intelligence, or even perseverance, or passion. It'd be focus. With focus, you can be amazingly successful in so many types of occupations.
(That being said, passion / perseverance / intelligence can often lead to focus)
I like his insight that a lot of base neural network implementation code is conceptually simple in the same way as writing a raytracer.
I think the point about NN and ray tracing being simple systems that allow for complex outcomes is something that seems to be a deeper truth about the universe. Stephen Wolfram and Max Tegmark both talk and write about this - it also shows up in old cellular automata like Conway’s game of life.
It’s pretty cool that so much complexity can come from a few small rules or equations.
My gut feeling is(was) that C++ is a first class citizen on OpenBSD. Its interesting having my beliefs challenged like this from someone so influential. The question becomes; if its not C++, then what is the first class citizen on OpenBSD. And if it is C++ then how do we improve support? Just going to ports seems like a poor answer.
> The question becomes; if its not C++, then what is the first class citizen on OpenBSD. Being an UNIX derivative the answer is quite easy, C.
Sure, but is there a popular, modern C compiler out there that doesn't compile C++ as well? Clang, GCC, Intel, MSVC... all of them compile C++ as well as C. I don't think it's possible to have first-hand support for C and not C++ any more unless you ship your own compiler. It sounds like the issue here is that OpenBSD is shipping older versions of LLVM/Clang and GCC.
Interesting to hear that C++ isn’t that well supported on OpenBSD. The story is quite the opposite with FreeBSD, where it’s really easy to use either clang or gcc. I usually spin up new jails to keep different installations sandboxed. CLion works quite nicely with most window managers on FreeBSD, but I rarely boot Xwindows these days and usually prefer to work with emacs inside tmux from the console.
My gut feeling is(was) that C++ is a first class citizen on OpenBSD. Its interesting having my beliefs challenged like this from someone so influential. The question becomes; if its not C++, then what is the first class citizen on OpenBSD. And if it is C++ then how do we improve support? Just going to ports seems like a poor answer.
It's possible given his technical depth he was using more obscure C++ constructs which aren't as well supported.