People here focus on Python, but to me, a bioinformatician, conda is much more, it provides 99.99% of the tools I need. Like bwa, samtools, rsem, salmon, fastqc, R. And many, many obscure tools.
[deleted]
Conda: A package management disaster?
151–160 of 234 posts
Re: Conda: A package management disaster?
#152Earlier quoted context omitted.
Agreed. But the problem is now fully solved by https://docs.astral.sh/uv/ .
I'm enjoying uv but I wouldn't say the problem is "fully" solved -- for starters it's not uncommon to do `uv add foo` and then 5K lines of gobbledygook later it says "missing foo-esoterica.dll" and I have to go back to the multiplatform drawing board.
Could it be a problem with a specific Python package being installed rather than uv itself?
Re: Conda: A package management disaster?
#153Impossible to read on mobile, least of all because of the lack of any word breaking.
How does one even do that? Browsers try quite hard to not break words and make text readable by default.
The blog author explicitly requested it, with `word-break: break-all`.
Now why you would do that … IDK.
Weirdly, it's the second post on HN this quarter to do it, from a completely different site. Makes me wonder if there's some viral piece of CSS advice out there …? (And nobody looks at their site…?) Bad LLM output?
Re: Conda: A package management disaster?
#154Re: Conda: A package management disaster?
#155People here focus on Python, but to me, a bioinformatician, conda is much more, it provides 99.99% of the tools I need. Like bwa, samtools, rsem, salmon, fastqc, R. And many, many obscure tools.
I wish you luck with tracking down versions of software used when you're writing papers... especially if you're using multiple conda environments. This is pretty much the example used in the article -- version mismatches.
But, I think this illustrates the problem very well.
Conda isn't just used for Python. It's used for general tools and libraries that Python scripts depend on. They could be C/C++ that needs to be compiled. It could be a Cython library. It could be...
When you're trying to be a package manager that operates on-top of the operating system's package manager, you're always going to have issues. And that is why Conda is such a mess, it's trying to do too much. Installation issues are one of the reason why I stopped writing so many projects in Python. For now, I'm only doing smaller scripts in Python. Anything larger than a module gets written in something else.
People here have mentioned Rust as an example of a language with a solid dependency toolchain. I've used more Go, which similarly has had dependency management tooling from the begining. By and large, these languages aren't trying to bring in C libraries that need to be compiled and linked into Python accessible code (it's probably possible, but not the main use-case).
For Python code though, when I do need to import a package, I always start with a fresh venv virtual environment, install whatever libraries are needed in that venv, and then always run the python from that absolute path (ex: `venv/bin/python3 script.py`). This has solved 99% of my dependency issues. If you can separate yourself from the system python as much as possible, you're 90% of the way there.
Side rant: Which, is why I think there is a problem with Python to begin with -- *nix OSes all include a system level Python install. Dependencies only become a problem when you're installing libraries in a global path. If you can have separate dependency trees for individual projects, you're largely safe. It's not very storage efficient, but that's a different issue.
Re: Conda: A package management disaster?
#156conda was for scientific python, but had to solve for everything below python to make that work. There was no generic binary solution before python for multiple architectures and operating systems.
Re: Conda: A package management disaster?
#157Earlier quoted context omitted.
Isn't uv to conda what uv is to pip?
`uv` is not a drop-in replacement for `conda` in the sense that `conda` also handles non-python dependencies, has its own distinct api server for packages, and has its own packaging yaml standard. `pixi` basically covers `conda` while using the same solver as `uv` and is written in Rust like `uv`. Now is it a good idea to have python's package management tool handle non-python packages? I think that's debateable. I p…
I have been using pixi for half a year and it has been fantastic.
It’s fast, takes yml files as an input (which is super convenient) and super intuitive
Quite surprised it isn’t more popular
Re: Conda: A package management disaster?
#158People here focus on Python, but to me, a bioinformatician, conda is much more, it provides 99.99% of the tools I need. Like bwa, samtools, rsem, salmon, fastqc, R. And many, many obscure tools.
I wish you luck with tracking down versions of software used when you're writing papers... especially if you're using multiple conda environments. This is pretty much the example used in the article -- version mismatches. But, I think this illustrates the problem very well. Conda isn't just used for Python. It's used for general tools and libraries that Python scripts depend on. They could be C/C++ that needs to be c…
> I wish you luck with tracking down versions of software used when you're writing papers... especially if you're using multiple conda environments.
How would you do this otherwise? I find `conda list` to be terribly helpful.
As a tool developer for bioinformaticians, I can't imagine trying to work with OS package managers, so that would leave vendoring multiple languages and libraries in a home-grown scheme slightly worse and more brittle than conda.
I also don't think it's realistic to imagine that any single language (and thus language-specific build tools or pkg manager) is sufficient. Since we're still using fortran deep in the guts of many higher level libraries (recent tensor stuff is disrupting this a bit, but it's not like openBLAS isn't still there as a default backend).
Re: Conda: A package management disaster?
#159Earlier quoted context omitted.
imagine being a beginner to programming and being told "use venvs" or worse, imagine being a longtime user of shells but not python and then being presented a venv as a solution to the problem that for some reason python doesn't stash deps in a subdirectory of your project
You don't need to stash deps in a subdirectory, IMHO that's a node.js design flaw that leads to tons of duplication. I don't think there's any other package manager for a popular language that works like this by default (Bundlers does allow you to version dependencies which can be useful for deployment, but you still only ever get one version of any dependency unlike node). You just need to have some sort of wrapper/…
What was unique to node.js was the decision to not only store the dependencies in a sub-folder, but also to apply that rule, recursively, for every one of the projects you add as a dependency.
There are many dependency managers that use a project-local flat storage, and a global storage was really frowned upon until immutable versions and reliable identifiers became popular some 10 years ago.
Re: Conda: A package management disaster?
#160Can somebody please eli5 why it is so unanimously accepted that Python's package management is terrible? For personal projects venv + requirements.txt has never caused problems for me. For work projects we use poetry because of an assumption that we would need something better but I remain unconvinced (nothing was causing a problem for that decision to be made).
> For personal projects venv + requirements.txt has never caused problems for me.
That means you don't use Windows.
What is great. Keep not using it. But most people will have a different experience.