I guess the real issue here is the "work-life balance is stellar". Humans just cant stand being in a comfortable position for a long time. People struggling in their jobs/life will say:"what are you talking about??? I would kill for this opportunity!", but the fact is that we know that growth only happens when there is change and when you are being challenged. This will affect not only your work, but also your relationships. Absolutely any job gets boring after some time, even if you are a Nascar racer. People get depressed when they have "everything". My formula to deal with this is to reinvent yorself many times, and look at your work/life from different angles each time. Understanding that you will be in the exact same position after some time makes you realize that eventually you will have to deal with it.
Ask HN: Has anyone here worked on the Windows kernel?
151–160 of 174 posts
Re: Ask HN: Has anyone here worked on the Windows kernel?
#152Earlier quoted context omitted.
Can you no longer use windbg to attach to a running kernel on a remote box, and debug the remote box from your development machine? I believe that's how kernel debugging is generally done.
(I’m a Linux kernel dev) Tools like windbg probably work great until you are debugging the code that runs when windbg tries to take over. Or you’re debugging something sensitive to interrupts and it’s literally impossible to keep up with a tool like windbg. Or you a debugging something that overwrites windbg in memory because you have a corrupt pointer. Or you triple-fault the machine and it reboots with so much prej…
Having worked on both windows and Linux kernel development, I can easily say that Windows has superior debugging facilities. It's pretty rare to make the debugger fall over. Working in the kernel is certainly not without its own challenges, but I feel like Linux makes it harder than it needs to be.
Re: Ask HN: Has anyone here worked on the Windows kernel?
#153I think I can count the number of kernel changes I've submitted on one hand, but I work on core virtualization that involves a lot of pretending to be hardware and (these days) a lot of poking directly at hardware registers. I would say James Mickens sums things up nicely in "The Night Watch[0]." For example, you mention debugging with logs and metrics -- this snippet came to mind: “Yeah, that sounds bad. Have you ch…
As an aside: James Mickens is a treasure. If anybody reading this hasn't read any of his articles or watched any of his talks before stop now and do it. You won't be sorry. https://mickens.seas.harvard.edu/wisdom-james-mickens
I did as you suggested. I was not sorry. Thank you!
Re: Ask HN: Has anyone here worked on the Windows kernel?
#154If MS or any FAANG wants to hire you for a kernel dev position, go for it; they clearly see a lot of potential in you that maybe you don't see yourself. I want to point out a couple of things that you should be prepared for: > 2. Debugging exclusively via metrics and logs, since I can't just attach a debugger to a running server. You often can't do this in kernel/OS development work either. Serial printf logging is o…
> > 3. Designing systems in general. Some people love the challenge of distributed transactions, eventual consistency and all that jazz, but it just rubs my brain the wrong way. I'm not interested at all in that problem space [1].
>I'm not sure I understand this issue. I mean, kernels have subsystems for doing stuff; you might need to design one someday? But it won't be dull-as-dishwater web technology stacks, it'll be you writing data structures directly in C/C++ or Rust if MS goes there.
Ah, I meant designing distributed systems. Like, figuring out the difference services, storage requirements, databases, then planning out the infrastructure. It's really not my cup of tea. I do enjoy designing systems/subsystems/components as long as they're within a single host :)
Re: Ask HN: Has anyone here worked on the Windows kernel?
#155Haven't worked on NT kernel, did implement a device driver for it once. Spent first 4 years of my career doing Linux kernel development. In general, go for it. The level of understanding you obtain by working on the kernel about how things -really- work will make you a better engineer even if it turns out not to be for you. Things I would caution you about based on the downsides of your current job: 1. Kernels make k…
Those are fair points. I should've expanded a bit on my list though, since a lot of my points could be interpreted in a way that I didn't mean.
1. Yeah, I have no doubt about that. My peeve with k8s is not its inherent complexity, but just how much worse it made our day-to-day work. When we ran things on managed services, everything was a lot more understandable and straightforward. k8s is piles and piles of hard-to-discover yamls and I (and other folks in the team) feel that it made our infra a lot harder to understand and change.
2. This was my most unclear point. What turns me off about my current experience debugging distributed systems is having to trudge through millions of log messages coming in constantly. I just can't say I like the ELK stack, despite its popularity.
3. Fair point, though I'm aware of the distributed nature of things even within a single host. I do expect it to be different though. Will I still have to account for "the response was lost, you don't know what happened" situations? My understanding is that being close to the metal I can at least trust that the wires are all still there and working, and if they aren't, well, I can say the machine (or something within it) is broken and needs to be replaced. Do you have to do things like sagas or deal with eventual consistency at the kernel level? Hardware guarantees seem a lot stronger than what you get in networked systems.
4. Yay!
5. The difference here is that even a distributed system that's built correctly will still page someone one day. There'll be more traffic than it was built for, or some network dependency will be down (like, not the things you accounted for, but something fundamental like a DNS server or whatever), etc. My expectation (and past experience) is that in code that's intended to run on a single host, I'll either get it right and it'll just work as intended, or there'll be bugs that I have to fix, and once I do things will just work. What I find extremely frustrating about services is that it seems that no matter how much effort you put into quality, there will still be times when it just goes down and people have to put out a fire.
Definitely going for it after all the responses here!
Re: Ask HN: Has anyone here worked on the Windows kernel?
#156If you're miserable in your job now then switching seems like there's lots of upside and only a little downside (it could potentially be worse than current). The main thing I'd consider in this is the work/life balance. Will it allow the same stellar level of that? People vary in their priorities. For some it's interesting/fun problems to solve, for some it's impact, but for me it's work/life balance, presuming of co…
Is work/life balance a US thing? In my country, for example, the employer can't easily make you work for longer than 8 hours per day, and even then there are big big compensations like extra payment with time off, restrictions on how much overtime per year is allowed and so on. So when people speak of a bad life/work balance, how does that happen in the first place?
Re: Ask HN: Has anyone here worked on the Windows kernel?
#157I work at Apple on the kernel team. Congratulations on the job! Getting into kernel development is very difficult. You said you were looking for things that you might not like about kernel development, so here’s a few: - Distributed systems. Everything is distributed these days with NUMA, cache coherency, PCI transactions, etc. You’ll have to know how to use the right kind of atomic memory ordering, debug lockless al…
> - Distributed systems. Everything is distributed these days with NUMA, cache coherency, PCI transactions, etc. You’ll have to know how to use the right kind of atomic memory ordering, debug lockless algorithms, and know how the OS scheduler works at a deep level. If this doesn’t excite you, the job might not be a good fit.
I'm familiar with the things you mentioned, except PCI transactions. In this context, do you also have to deal with distributed transactions, lost responses, eventual consistency, and the like? Do you have to account for "response lost, don't know if it went through" situations?
Definitely going for it after all the responses here :)
Re: Ask HN: Has anyone here worked on the Windows kernel?
#158Earlier quoted context omitted.
> Windbg is actually great and I will die on this hill Came here to say this. (Actually kd, but lots of overlap.)
How did you learn it? I use it occasionally to debug memory dumps after an application crash but I know that WinDbg can do so much more than show me the stack just as the application crashed.
I learned it at Microsoft. When debugging issues in Windows, it was pretty common to leave a remote server open and pass it around among colleagues to find the right person who knows what code is failing. You could see what the remote party was typing and doing. It was a pretty social way to learn your way around.
Re: Ask HN: Has anyone here worked on the Windows kernel?
#159I'm pretty burned out too with my work (mostly web dev stuff). I've been meaning to get "AWS certified" or learn Kubernetes, but like you, it all seems so crazy to me. I used to love old school "linux administration", but this new wave of tech gives me no interest. Curious how you get an offer such as this? I've thought about changing job roles, but I really suck at leet coding so I've never really bothered. I figure…
Feeling the same. Tired of web stack. Don’t want to do Kubernetes. Don’t want to do ML. All I want is descending to lower level programming. Rust would be nice. C would do too. LLVM IR might be my ultimate language. I don’t waste time on Leetcode. I spend most of my free time on side projects on GitHub which helps me build up a portfolio for my next job. I also watch MIT courses to learn system programming. I don’t c…
Would you mind sharing a link or two to these courses?
Re: Ask HN: Has anyone here worked on the Windows kernel?
#160I spent about ten years on the Windows Kernel team. In the 90's, so likely very different than what you would experience today, but probably still a lot of the same problems. (Funny to hear people are still complaining about windbg!) I think you are coming at this from the wrong perspective. Rather than thinking about how to avoid work you DON'T like, think about what you DO like and then decide if the new job would…
Don't you find it difficult to transition from one domain to another?
I like to learn new things; I find it energizing, especially when I am feeling burned out. Sure, it can be difficult but that is also what makes it rewarding. It's also really interesting to see the different tradeoffs that have been made between different solutions. (Linux vs Windows for example)
Ultimately, it's all just code and problem solving. And you can usually find a way to leverage your expertise in one domain into a different domain.