Live data from Hacker News

Dtrace for Linux

gnu.wildebeest.org

91–100 of 108 posts

Re: Dtrace for Linux

#91
post #67

Thanks God, eBPF is a security nightmare as we saw with Spectre, they even got arrays now. and there is no infrastructure as with dtrace. Will compile it into my kernel for sure for proper dev work.

Doesn't this criticism apply to any software that opens a communications channel between the kernel and user space? Also, how does eBPF relate to Spectre?

Yes, but syscalls are properly designed. drivers are a huge problem, always have been. eBPF on the other hand lacks proper security design. They added it afterwards, to some extent.

> how does eBPF relate to Spectre?

Please read any spectre paper. Besides the known javascript attacks, eBPF is the easiest way to bypass kernel ASLR. google is your friend.

Re: Dtrace for Linux

#92
post #32

Earlier quoted context omitted.

Well, none of my dtrace knowledge works with eBPF, and eBPF is only useful on linux. This is still a very welcome change :)

Yeah, the compatibility element is really important. I've been doing tons of research around illumos, SmartOS, etc., lately and I'm hoping to deploy it in some projects soon. DTrace knowledge becoming properly portable across systems is a great thing. I've been very impressed with illumos from a technical standpoint, as a Linux admin for over 10 years whose never really had cause to seriously wander into the now-esot…

For what it's worth, I've been at Joyent for six years now. It's still a great place to work, and we're still fully invested in SmartOS. Samsung has some pretty seriously large computing workloads and they bought us to use our technology.

I would note also that while we have had staff come and go over the years, I don't believe there was a mass exodus of Sun alumni -- indeed, since the recent ahem reprioritisation of Solaris at Oracle we've picked up a bunch of new Sun alumni!

In short, we're keeping the dream alive and doing the work we think is important. The world has space for more than one approach to systems software, and we're keen to keep working on ours!

Re: Dtrace for Linux

#93

Earlier quoted context omitted.

I'm sorry, but Bryan did not "address" the real reasons I left Joyent. What he said was inaccurate. I no longer believe in illumos. It made a lot of sense in 2010 when competing with Linux, a lot less by 2014, and very little today. When we created illumos in August 2010, the latest Linux version was 2.6.35. The cutting-edge latest. Most people were still running older versions, 2.6.32, 2.6.27, etc. Linux had many pe…

Brendan, thanks very much for replying to this. It's great to get clarity around it. Your blog and HN posts have been great resources as I've gone down this rabbit hole, and since I've read so many of them, I know that's not the first time you've heard it. Your work has been hugely valuable to this community. Since the options for illumos employment are and were scant, the story that you were looking for a change of…

I'm definitely not trying to be "hand-wavy"; apologies if it's taken that way.

So let me make this slightly blunter: our goal in SmartOS and illumos is to solve systems problems -- and yes, we do it with the courage to go our own way where we see fit. If that worldview is a match for you, that's awesome. If that's not a match for you, that's fine too -- world domination is not, in fact, our goal. Yes, we are a small community, but that's fine with us: this is all open source; it's not going away.

As an aside, I personally don't understand why anyone would choose to spend their time denigrating small communities; if you want to write your software in (say) OCaml and deploy it on OpenBSD via a Nix derivative, why should anyone spend their time telling you that you are wasting yours? (These are three examples of communities that -- like SmartOS -- are small but technically interesting.)

And more importantly: why should you (or anyone, really) listen to a person who has defined themselves so negatively and so rigidly? Your technology choices -- like the books you like or the movies you enjoy or the ideas you have -- are your business; don't let anyone browbeat your choices out of you simply because they don't match their own.

Re: Dtrace for Linux

#94
post #77

Earlier quoted context omitted.

Better? OK, could you please show me eBPF oneliner code which is analogue of the DTrace command like: # dtrace -qn 'syscall::write:entry /execname == "mysqld"/ {self->stime = timestamp;} syscall::write:return /self->stime != 0/ {@LWrite = quantize(timestamp - self->stime);} tick-10s {printa(@LWrite);}' ?? Above is now working not only under Solaris but under Oracle UEK Linux kernel as well.

# funclatency -i 10 -p `pgrep -n mysqld` sys_write Whoops, that was easy. I think argdist can do this as well. I'd have picked a different one-liner. It's not hard to come up with DTrace one-liners that are currently horribly hard with eBPF and bcc. Which is why I said we could have DTrace's D as a front end to bcc/eBPF. Or the bpftrace project will do that, and already has similar one-liners. Check it out[1]. Either…

Oh wow, I had no idea that funclatency and biolatency existed. There aren't enough hours in the day to learn about everything that could be saving me time. Huge thanks!

Re: Dtrace for Linux

#95
post #79
post #75

Earlier quoted context omitted.

You mean, like systemd?

systemd still has a race condition when handling forking servers. There's no way to atomically send a signal to a cgroup, so what systemd does is read the PIDs in the cgroup and then iteratively send SIGKILL to each one. However, between reading the PID and sending the signal is the classic PID file race. There is a way to atomically send a signal to a traditional process group, however. What I do for my daemons--at…

Right. I implemented the same controlling pty approach in Buck. It's amazing how quickly forget robust older solutions and jump immediately to the new hotness.

Re: Dtrace for Linux

#96
post #85

Earlier quoted context omitted.

For ages now, I've wanted NT-style process handles. open(2)ing a process (maybe via its proc directory) should keep the corresponding process alive, even if as a zombie. This way, you'd be able to write perfectly robust versions of pkill without annoying TOCTOU PID reuse races. Linus has rejected this mechanism due to the ability of an outstanding process handle to prevent process reclaim, permitting users to fill th…

We've talked before about FreeBSD's process descriptors and EVFILT_PROC kevents, ne?

Process descriptors are unfortunately a little incomplete -- pdwait() was never implemented. They don't seem to be getting much use.

Re: Dtrace for Linux

#97

Earlier quoted context omitted.

Brendan, thanks very much for replying to this. It's great to get clarity around it. Your blog and HN posts have been great resources as I've gone down this rabbit hole, and since I've read so many of them, I know that's not the first time you've heard it. Your work has been hugely valuable to this community. Since the options for illumos employment are and were scant, the story that you were looking for a change of…

I'm definitely not trying to be "hand-wavy"; apologies if it's taken that way. So let me make this slightly blunter: our goal in SmartOS and illumos is to solve systems problems -- and yes, we do it with the courage to go our own way where we see fit. If that worldview is a match for you, that's awesome. If that's not a match for you, that's fine too -- world domination is not, in fact, our goal. Yes, we are a small…

I see the small community in illumos as a benefit, more or less. I'm not trying to denigrate it for its size. The issue is the attrition, the apparently-negative momentum, and the loss of prominent proponents and distributors, including Brendan, OmniTI, etc. That's bad when something is already small. It makes people wonder why others are fleeing.

My concern, as it relates to this thread, is around the spectacular loss of faith that Brendan has exhibited. You don't go from being a world-recognized authority on DTrace like Brendan to a Linux refugee overnight, especially not 3 years ago when you have to lay out all the plumbing for eBPF yourself.

So something happened. That's clear. I want to know what it is, at least proximally / vaguely, because if it can drive Brendan away from illumos, I assume it can also drive away those of us who are less involved.

If it was simply a personal dispute and had nothing to do with technical direction, that, of course, is one thing that doesn't reflect badly on illumos at all. People burn out sometimes, they want a change of scenery, whatever. But Brendan appears to stringently believe that illumos is now a bad choice in virtually all circumstances. It feels like there's something substantial at play. I don't think he's petty enough to bash the illumos kernel just because he had a spat or wanted to mix things up, especially considering it contains a not-insignificant portion of his own work.

I don't necessarily expect either of you to reveal details, and obviously, I respect that you have both moral and legal obligations as it relates to confidentiality surrounding personnel matters. I'm just talking about my read on the situation, and why I'm skittish, despite what I feel are some strong positives in SmartOS's favor. A world where Bryan Cantrill is the lone SmartOS holdout, after his compatriots have declared it a lost cause, is just nerve-wracking. I'm sure you can understand an outsider's perspective on that.

It'd be cool if you could discuss some of the performance ground that illumos has covered since 2014, as Brendan gave a pretty thorough list of ground that Linux has covered. His tone always seems to be "Solaris is a sad situation, but look guys, there's light at the end of the tunnel." He gives absolutely no intimation that illumos is a viable alternative, though he will entertain FreeBSD. Always phrases it like everyone knows Solaris and its derivatives are fully dead, and he alternates between appearing relieved about it and appearing disappointed about it (I'm sure it's both).

I just want to know why he rules it out so strictly. It's weird, it doesn't add up. Missing variables. I don't think this attitude is sufficiently justified by "Linux caught up, that's all". When stuff is weird, I approach it cautiously, especially since my goal with SmartOS is something simple, stable, and powerful, no fuss. Weird things are fuss.

Re: Dtrace for Linux

#98

Earlier quoted context omitted.

I'm definitely not trying to be "hand-wavy"; apologies if it's taken that way. So let me make this slightly blunter: our goal in SmartOS and illumos is to solve systems problems -- and yes, we do it with the courage to go our own way where we see fit. If that worldview is a match for you, that's awesome. If that's not a match for you, that's fine too -- world domination is not, in fact, our goal. Yes, we are a small…

I see the small community in illumos as a benefit, more or less. I'm not trying to denigrate it for its size. The issue is the attrition, the apparently-negative momentum, and the loss of prominent proponents and distributors, including Brendan, OmniTI, etc. That's bad when something is already small. It makes people wonder why others are fleeing. My concern, as it relates to this thread, is around the spectacular lo…

OmniTI put OmniOS out to its active community: https://omniosce.org/

FYI.

Re: Dtrace for Linux

#99

Earlier quoted context omitted.

I'm definitely not trying to be "hand-wavy"; apologies if it's taken that way. So let me make this slightly blunter: our goal in SmartOS and illumos is to solve systems problems -- and yes, we do it with the courage to go our own way where we see fit. If that worldview is a match for you, that's awesome. If that's not a match for you, that's fine too -- world domination is not, in fact, our goal. Yes, we are a small…

I see the small community in illumos as a benefit, more or less. I'm not trying to denigrate it for its size. The issue is the attrition, the apparently-negative momentum, and the loss of prominent proponents and distributors, including Brendan, OmniTI, etc. That's bad when something is already small. It makes people wonder why others are fleeing. My concern, as it relates to this thread, is around the spectacular lo…

> A world where Bryan Cantrill is the lone SmartOS holdout, after his compatriots have declared it a lost cause, is just nerve-wracking.

I don't see where you are getting this idea. There are many contributors to Illumos from all over the globe. https://github.com/illumos/illumos-gate/graphs/contributors . Bryan stated in a previous post that Joyent now employs more former Sun engineers than it ever has.

> If it was simply a personal dispute and had nothing to do with technical direction

From the outside looking in when Brendan left, reading both the mailing lists and Twitter, this is exactly what it seemed like.

Post reply on HN