DTrace and Python
github.com
DTrace and Python
1–10 of 21 posts
Re: DTrace and Python
#2Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.
Re: DTrace and Python
#3Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.
> Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux?
Never. Due to dtrace being licensed under the CDDL it will never be mainlined into the upstream kernel. I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS kernel module in recent releases of Ubuntu. I shall say no more on the matter as a refuse to be drawn into a licensing debate...
Getting back on topic, those seeking more on user space providers should check out this video from Bryan Cantrill.
https://www.youtube.com/watch?v=6chLw2aodYQ
Full disclosure I never had to run Solaris in production so I can look back on Sun with rose tinted glasses. That said I still long for many of the features it pioneered. Seriously ZFS would eliminate half of my production problems and dtrace would make the remaining half solvable.
Re: DTrace and Python
#4Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.
> Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Never. Due to dtrace being licensed under the CDDL it will never be mainlined into the upstream kernel. I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS…
> I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS kernel module in recent releases of Ubuntu.
AFAIK it's only packaged as a dkms-package:
Package: zfs-dkms
Version: 0.6.5.6-0ubuntu8
Priority: extra
Section: universe/kernel
Source: zfs-linux
Origin: Ubuntu
Maintainer: Ubuntu Developers...
Original-Maintainer: Darik Horn ...
Bugs: https://bugs.launchpad...
Installed-Size: 8 325 kB
Provides: lustre-backend-fs,
lzfs, lzfs-dkms
Pre-Depends: spl-dkms (>= 0.6.5.6)
Depends: dkms (>= 2.2.0.2), zfs-doc (= 0.6.5.6-0ubuntu8)
Conflicts: lzfs, lzfs-dkms
Replaces: lzfs, lzfs-dkms
(...)
xenial/universe amd64 Packages
From DKMS(8): dkms is a framework which allows kernel modules to be dynamically built for each kernel on your system in a simplified and organized fashion.
> I shall say no more on the matter as a refuse to be drawn into a licensing debate...
I really don't want to start that debate, but I also think the "Ubuntu distributes (binary) ZFS" isn't quite accurate.
Now, how this all shakes out in terms of whether or not ZFS is bundled with Ubuntu ("on the ISO so to speak"), and if it then really makes a difference that it's distributed as source-code that can be automatically compiled -- I don't know. I wouldn't think shipping pythons source (to be later compiled to byte code, pyc) is enough to dodge GPL restrictions on bundling other code/binaries as part of a distribution.
But, yeah, whichever it is, CDDL code isn't likely to ever be upstreamed to the kernel proper.
Re: DTrace and Python
#5Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.
FreeBSD already has DTrace :)
Re: DTrace and Python
#6Shameless plug: if you are looking for something similar to DTrace but for Windows that can be also used in Python or any other COM capable programming language... you can take a look at our Deviare Hooking Engine: https://github.com/nektra/Deviare2
Additionally, we have other open source instrumentation engines like https://github.com/nektra/Deviare-InProc (better and more secure than Detours, check [1]), RemoteBridge: https://github.com/nektra/RemoteBridge. SpyStudio will also be open sourced very soon: http://www.nektra.com/products/spystudio-api-monitor/
[1] https://www.blackhat.com/docs/us-16/materials/us-16-Yavo-Cap... and https://www.blackhat.com/docs/us-16/materials/us-16-Yavo-Cap...
Re: DTrace and Python
#7Great deck! In particular, for the incredible magic on slide 14, a debt of thanks is owed to John Levon[1] and to whomever has maintained that work and brought it forward.
On slide 28, the presentation asks "What is 'Speculative Tracing'?" It's unclear if that's a rhetorical question, but just to answer it here: speculative tracing is a DTrace facility that allows for data to be traced speculatively, and only committed to the trace buffer if and when some other (later) condition is met.[2] My original inspiration for this was a case that we had back in the day at Sun on the Performance and Application Engineering (PAE) team, when Yufei Zhu (now at Facebook) described a case she had in which one out of every 10,000 mmap()'s was failing with EINVAL -- and it was really tough to use DTrace when she was only interested in its output 0.01% of the time. For Yufei's (motivating) example, speculative tracing offered a way of capturing all of the necessary data on every mmap request, but only emitting that data when the entire operation was found to have failed. Speculative tracing is one of those you-don't-need-it-until-you-need-it features of DTrace (to which I would certainly add anonymous tracing) -- but when you need it, it's a lifesaver, and I have it used it as recently as last week to nail a particularly nasty bug that very much needed it.[3]
[1] https://blogs.oracle.com/levon/entry/python_and_dtrace_in_bu...
Re: DTrace and Python
#8Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.
Enhanced BPF is integrated in Linux 4.x series -- as in, it is part of the kernel, and everyone who deploys on Linux will be getting it once they upgrade -- and provides similar capabilities.
Re: DTrace and Python
#9Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.
> Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Never. Due to dtrace being licensed under the CDDL it will never be mainlined into the upstream kernel. I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS…
That's Linux today. Upgrade. ZFS + DTrace capabilities.
Re: DTrace and Python
#10Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.
Just give FreeBSD a shot!