Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dtrace for Linux

gnu.wildebeest.org

Post reply on HN
Print view View original

21–30 of 108 posts

Re: Dtrace for Linux

#21

Post by byefruit »

burke wrote: post #19
brendangregg wrote: post #9

Unfortunately for DTrace, this is too late. Oracle should have done this years ago. Now Linux has a more powerful tracer builtin, eBPF, and it would be a backwards step to switch the kernel code to DTrace (assuming the DTrace port is completed, which it is not). I'm sure this will not be lost on the maintainers, who have the ultimate say as to what is included in Linux mainline. The only hope for DTrace is to have th…

As a non-expert, I still find this exciting. While eBPF may be more powerful, I find it far less approachable than DTrace. Perhaps this is just a failure to discover the right documentation. I use DTrace now and then on macOS and have resented that I can't really use it on Linux as well.

With regards to tracing eBPF has actually moved at a fairly brisk pace the last year or two and it will probably take a while for the documentation and knowledge to permeate.

The usability side you mention is what Brendan's suggesting is worth porting from DTrace, just the front-end that the users sees. That would just back onto eBPF and the user would know little different.

Re: Dtrace for Linux

#22

Post by brendangregg »

burke wrote: post #19
brendangregg wrote: post #9

Unfortunately for DTrace, this is too late. Oracle should have done this years ago. Now Linux has a more powerful tracer builtin, eBPF, and it would be a backwards step to switch the kernel code to DTrace (assuming the DTrace port is completed, which it is not). I'm sure this will not be lost on the maintainers, who have the ultimate say as to what is included in Linux mainline. The only hope for DTrace is to have th…

As a non-expert, I still find this exciting. While eBPF may be more powerful, I find it far less approachable than DTrace. Perhaps this is just a failure to discover the right documentation. I use DTrace now and then on macOS and have resented that I can't really use it on Linux as well.

eBPF lacks a high level front end, in part because that's optional. Engineering effort has gone into the essentials first: kernel development of eBPF, basic user-level frameworks to use it (bcc), and lots and lots of testing and bug fixes to make that rock solid. bcc is not as easy to code in, but I've been able to port over many of my DTraceToolkit scripts already.

We're only more recently looking at higher level languages, as a nice to have. Here's one project:

https://github.com/ajor/bpftrace

We can also look at taking the /usr/sbin/dtrace code (licenses permitting), and calling into libbcc and emitting BPF bytecode. That'd give us a D language front-end to BPF, so I can run my old DTrace scripts directly (plus sell more copies of my DTrace book -- I joke ;-)

Help on either of those projects would be appreciated (bpftrace or DTrace-BPF).

Re: Dtrace for Linux

#23

Post by nailer »

burke wrote: post #19
brendangregg wrote: post #9

Unfortunately for DTrace, this is too late. Oracle should have done this years ago. Now Linux has a more powerful tracer builtin, eBPF, and it would be a backwards step to switch the kernel code to DTrace (assuming the DTrace port is completed, which it is not). I'm sure this will not be lost on the maintainers, who have the ultimate say as to what is included in Linux mainline. The only hope for DTrace is to have th…

As a non-expert, I still find this exciting. While eBPF may be more powerful, I find it far less approachable than DTrace. Perhaps this is just a failure to discover the right documentation. I use DTrace now and then on macOS and have resented that I can't really use it on Linux as well.

IIRC one of the big things about dtrace vs systemtap is that dtrace could trace a process from userspace to kernel space and then back again. Can eBPF do this? How else do they compare?

Re: Dtrace for Linux

#24

Post by Gracana »

jsiepkes wrote: post #7
rhinoceraptor wrote: post #5

Earlier quoted context omitted.

If ZFS were also relicensed, would Illumos be able to relicense OpenZFS as well, even with the divergence?

If Oracle's sources would be GPL'ed all OpenZFS changes would also have to be put under the GPL (or dual licensed) if you want to use those. OpenZFS doesn't have a CLA I think so that might be somewhat of a problem. Then again most ZFS changes were made by people for their employers (Joyent, Delphix, etc.) so it might be that only a few companies (...and Allan Jude ;-) need to give their blessing.

Solaris developers have an unhappy history with CLAs, you are unlikely to see them in that community ever again.

Re: Dtrace for Linux

#25

Post by brendangregg »

nailer wrote: post #23
burke wrote: post #19

Earlier quoted context omitted.

As a non-expert, I still find this exciting. While eBPF may be more powerful, I find it far less approachable than DTrace. Perhaps this is just a failure to discover the right documentation. I use DTrace now and then on macOS and have resented that I can't really use it on Linux as well.

IIRC one of the big things about dtrace vs systemtap is that dtrace could trace a process from userspace to kernel space and then back again. Can eBPF do this? How else do they compare?

That would not be recalling correctly: SystemTap does that, and so does eBPF (via uprobes and kprobes).

A large problem with SystemTap is that it was developed in the open, and in the early days people were scared off with its kernel panics. SystemTap built a negative brand: first impressions matter. DTrace, however, was largely built before its public release, and with the help from many other engineers and test teams. People's first impressions with DTrace was something that was already awesome. Of course, there are other problems with SystemTap: its process of compiling code and loading kernel modules was always risker than an interpreter.

As for how eBPF compares: I summarized it last year http://www.brendangregg.com/blog/2016-10-27/dtrace-for-linux...

Re: Dtrace for Linux

#26

Post by rollcat »

HankB99 wrote: post #20
4ad wrote: post #8

Earlier quoted context omitted.

OpenZFS can't be relicensed as GPL since FreeBSD won't be able to use it. Perhaps it could be dual-licensed, not sure, but that would be bad for FreeBSD as well. If it would be integrated into the mainline Linux kernel, I don't believe FreeBSD could merge back subsequent ZFS changes done on the Linux side. If OpenZFS were relincensed and developed outside the mainline Linux tree (as it is now), FreeBSD would be able…

> ... but it would still be annoying to actually use ZFS in Linux for the same reason it is annoying to use it now. For those of us who are ignorant of what is annoying to use now (I raise my hand) can you elaborate? I'm fooling around with it on Debian Stretch and Ubuntu 16.04 LTS and haven't yet found it annoying (except for the challenge of finding where the various command line switches are documented...) I was p…

Running ZFS (boot&root) on both Ubuntu 17.10 on my home server and Debian Jessie on my laptop.

The ZFS experience between the two is heaven and hell. Ubuntu just ships the zfs.ko and basically everything works 100% of the time, zero effort.

Debian OTOH has to compile the module via DKMS for every minor kernel release... You better track those apt upgrades carefully, and run `dkms status` after each one, or you might end up having to drop into a rescue shell on next reboot, or try booting older kernels until you find the one that works. Also hopefully you didn't remove the last working kernel and/or zfs.ko. ZFS module upgrades are a roulette too (although none have failed... so far). Full disk encryption (including /boot) only adds to the complexity.

I set this up in January 2017, at the time it seemed like the coolest thing on Earth. In retrospect, it was foolish. I wish I went with ext4 for /boot & /, unencrypted /boot, and left ZFS for /home & maybe /var/lib.

Re: Dtrace for Linux

#27

Post by loeg »

4ad wrote: post #8
rhinoceraptor wrote: post #5

Earlier quoted context omitted.

If ZFS were also relicensed, would Illumos be able to relicense OpenZFS as well, even with the divergence?

OpenZFS can't be relicensed as GPL since FreeBSD won't be able to use it. Perhaps it could be dual-licensed, not sure, but that would be bad for FreeBSD as well. If it would be integrated into the mainline Linux kernel, I don't believe FreeBSD could merge back subsequent ZFS changes done on the Linux side. If OpenZFS were relincensed and developed outside the mainline Linux tree (as it is now), FreeBSD would be able…

There is plenty of dual GPL-BSD (or even MIT) licensed code in the Linux kernel that can be merged back and forth with FreeBSD. Examples are most Intel-contributed drivers (BSD+GPL) and graphics code (MIT).

Re: Dtrace for Linux

#28

Post by AceJohnny2 »

brendangregg wrote: post #9

Unfortunately for DTrace, this is too late. Oracle should have done this years ago. Now Linux has a more powerful tracer builtin, eBPF, and it would be a backwards step to switch the kernel code to DTrace (assuming the DTrace port is completed, which it is not). I'm sure this will not be lost on the maintainers, who have the ultimate say as to what is included in Linux mainline. The only hope for DTrace is to have th…

Tangential, but I feel the need to quote Dan Colascione:

"Ah Linux, in which a thread is a process, a semantic patch is a lint rule, and a packet filter is a system profiler."

;)

(but seriously, your work in exploring and expanding it has been wonderful. Thank you!)

Re: Dtrace for Linux

#29

Post by loeg »

jsiepkes wrote: post #4

Really cool! However isn't it a bit too late for DTrace to gain market share on Linux now SystemTap and all have gotten such a head start? For me personally I like the idea of being able to use my DTrace skills (which I gained on FreeBSD and SmartOS) on Linux!

It would be very nice to be able to use the same Dtrace tracing skills and interface across a variety of modern Unix systems. I hope it isn't too late.

Re: Dtrace for Linux

#30

Post by fche »

brendangregg wrote: post #25
nailer wrote: post #23

Earlier quoted context omitted.

IIRC one of the big things about dtrace vs systemtap is that dtrace could trace a process from userspace to kernel space and then back again. Can eBPF do this? How else do they compare?

That would not be recalling correctly: SystemTap does that, and so does eBPF (via uprobes and kprobes). A large problem with SystemTap is that it was developed in the open, and in the early days people were scared off with its kernel panics. SystemTap built a negative brand: first impressions matter. DTrace, however, was largely built before its public release, and with the help from many other engineers and test tea…

Note for freshness that systemtap can also emit eBPF directly now. So kernel modules are the fastest & most powerful way, but not the only one.
Post reply on HN