Python utility for tracking third party dependencies within a library
1–10 of 23 posts
Re: Python utility for tracking third party dependencies within a library
#2This looks like a real useful tool for large projects, it can be quite possible to loose track of what a specific dependancy is used for. I also like the idea of making an import lazy so in monolithic app you could have a deployment that excludes functionality, and exclude its dependancies.
When I read the title I was hoping for something else though, what I would love is a tool that logs and potentially blocks unexpected IO operations on a library basis. With the increasing common supply chain attacks we are seeing (there was a PyPI one just the other day), having a way to at least report on unexpected activity if not help prevent it would be brilliant. Has anyone ever found a tool like that?
(Obviously the ultimate solution would be an outbound firewall, but it seems be that although you can easily do this in a VM or bare metal, I haven't seen any PAAS platforms have that sort of capability)
Re: Python utility for tracking third party dependencies within a library
#3Tried this on one of my projects, it's neat.
python3 -m import_tracker --name datasette --recursive | jq
{
"datasette": [
"aiofiles",
"click",
"markupsafe",
"mergedeep",
"pluggy",
"yaml"
],
"datasette.version": [],
"datasette.utils.shutil_backport": [
"click",
"markupsafe",
"mergedeep",
"yaml"
],
"datasette.utils.sqlite": [
"click",
"markupsafe",
"mergedeep",
"yaml"
],
"datasette.utils": [
"click",
"markupsafe",
"mergedeep",
"yaml"
],
"datasette.utils.asgi": [
"aiofiles",
"click",
"markupsafe",
"mergedeep",
"yaml"
],
"datasette.hookspecs": [
"aiofiles",
"click",
"markupsafe",
"mergedeep",
"pluggy",
"yaml"
]
}
Re: Python utility for tracking third party dependencies within a library
#4[deleted]
Re: Python utility for tracking third party dependencies within a library
#5Could I use the lazy import to define a single set of dependencies of a monorepo and then load only the required subset for each project?
Re: Python utility for tracking third party dependencies within a library
#6Could I use the lazy import to define a single set of dependencies of a monorepo and then load only the required subset for each project?
That’s how Bazel Python works
Re: Python utility for tracking third party dependencies within a library
#7Tried this on one of my projects, it's neat. python3 -m import_tracker --name datasette --recursive | jq { "datasette": [ "aiofiles", "click", "markupsafe", "mergedeep", "pluggy", "yaml" ], "datasette.version": [], "datasette.utils.shutil_backport": [ "click", "markupsafe", "mergedeep", "yaml" ], "datasette.utils.sqlite": [ "click", "markupsafe", "mergedeep", "yaml" ], "datasette.utils": [ "click", "markupsafe", "mer…
Is this the equivalent of Poetry's `poetry show --tree`?
Re: Python utility for tracking third party dependencies within a library
#8This looks like a real useful tool for large projects, it can be quite possible to loose track of what a specific dependancy is used for. I also like the idea of making an import lazy so in monolithic app you could have a deployment that excludes functionality, and exclude its dependancies. When I read the title I was hoping for something else though, what I would love is a tool that logs and potentially blocks unexp…
> When I read the title I was hoping for something else though, what I would love is a tool that logs and potentially blocks unexpected IO operations on a library basis. With the increasing common supply chain attacks we are seeing (there was a PyPI one just the other day), having a way to at least report on unexpected activity if not help prevent would be brilliant. Has anyone ever found a tool like. that?
You could do something close to that with Python's audit hooks, which were introduced with 3.8[1]. One massive caveat: audit hooks can be disabled by an attacker with the ability to control the interpreter, and are not perfect (there's plenty of things they don't cover.)
(More generally: this kind of auditing/restriction falls under the umbrella of "capability management." OpenBSD's pledge[2] is another example.)
Re: Python utility for tracking third party dependencies within a library
#9You can use Syft [1] which generates the full software bill of materials, which includes package names, licenses for a broad set of tech stack ranging from OS level (Alpine, Debian), through Go, Ruby, Python, Java, JavaScript, etc.
Re: Python utility for tracking third party dependencies within a library
#10You can use Syft [1] which generates the full software bill of materials, which includes package names, licenses for a broad set of tech stack ranging from OS level (Alpine, Debian), through Go, Ruby, Python, Java, JavaScript, etc. [1] https://github.com/anchore/syft
Since this is about Python specifically, I'll go ahead and and highlight `pip-audit`[1] as a specialized tool for generating Python SBOMs and running audits against the official PyPI vulnerability feed.
FD: My company, my work.