Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ease of maintenance is a feature

ronakgothi.com

Post reply on HN
Print view View original

21–30 of 39 posts

Re: Ease of maintenance is a feature

#21

Post by bunderbunder »

aaronblohowiak wrote: post #8
sethammons wrote: post #5

Earlier quoted context omitted.

I have a couple decades of experience and have ridden a small start up through public and I have worked intimately with 6 companies. I know about taking a product and ultra scaling it in both technical and organizational scale. I will never recommend Python outside of a small team. It is organizational molasses. My current company has multiple teams striving to keep our Python tech stack serving our growing technical…

The hoops that people go through to solve this sometimes creates something even more complex and not great, like forcing all development into a docker container.. Ever try conda though? I’ve had moderate success with pipenv, but tbh I don’t love it as it hides too many things when installing a package fails.

Doing a Python development environment inside of Docker can get particularly obnoxious in the long run because there are approximately a zillion ways that a base image upgrade can break things by changing something about the system Python packages.

(And by the time it happens you might have a real mess on your hands because by that point the dev container's dockerfile has quite possibly grown into an undocumented spaghetti tangle of band-aids as a result of every dev on the team tweaking things in whatever way seemed to make the most sense at the time without a whole lot of regard for the end-to-end cohesiveness of the situation.)

The standard advice in the Python community is "never trust the system Python", but tools like pyenv that we have for protecting ourselves from the operating system aren't always straightforward to get working sensibly inside of a container. It seems like it should be easy, but I've seen people get it wrong far more often than I've seen them get it right.

A big part of the problem is that the Python community has developed an extremely severe case of TMTOWTDI when it comes to dependency management, packaging and deployment. It's led to a situation where, if you're just googling around for problem solutions in an ad-hoc manner, you're likely to end up with a horrible chimera of different philosophies of how to do Python devops, and they won't necessarily mesh well together.

Re: Ease of maintenance is a feature

#22

Post by AlienRobot »

cglan wrote: post #2

Super true. One of the best tests of this is setting up a new laptop. Some of the best experiences are when you get a new laptop, and just clone the codebase and everything works as it did before, no special magic. Golang with vendored dependencies seems to be wonderful for this but I've had relatively decent experiences with newer java projects. My worst experiences universally have always been python projects. I do…

I love Python but it always amazes me how hard it is for it to just... work.

So there is virtualenv, built in, but... if there is a venv directory, Python doesn't just use it.

Like you have app.py, and you python app.py, that doesn't run it with the venv python. This leads to all sorts of problems with scripts that assume they're running under venv. Which means you probably want to write a script that sources venv just so you don't forget, but if you place it in the same directory you may forget you need to call the script, so you probably want to add an extra directory to hide all the python code so you only see the shell script that you need to run to properly setup the environment to run the python code. Or just use an IDE.

Just "pip install." But pip isn't installed and ensure pip doesn't work? What do I even do then?

I recall downloading a project that required a library that wasn't available for the newest version of python, so when you tried to install the requirements pip wouldn't find it. I discovered this, naturally, because I updated my operating system so the python version changed which means the project that used to work stopped working! What is the solution for installing multiple python versions side by side? Hint: it's not an official project by the Python organization but something you can find on github.

Re: Ease of maintenance is a feature

#23

Post by uludag »

I was reflecting on this article, thinking about what software tools and languages I use that reflect this property, and a weird realization came to mind: Emacs lisp is by far the best language I use in this regard. I literally copy-pasted 20+ year code, eval it, and every time it just works. Then if I want to debug it: C-u C-M-x, and I'm instantly stepping through the code.

Something this old shouldn't have this property. Nothing "modern" even comes close. Look at the top languages, Python, JavaScript, and Java, and you don't even have to consider too much how abysmal these languages are in this regards.

Re: Ease of maintenance is a feature

#24

Post by nzach »

cglan wrote: post #2

Super true. One of the best tests of this is setting up a new laptop. Some of the best experiences are when you get a new laptop, and just clone the codebase and everything works as it did before, no special magic. Golang with vendored dependencies seems to be wonderful for this but I've had relatively decent experiences with newer java projects. My worst experiences universally have always been python projects. I do…

> My worst experiences universally have always been python projects.

Do you mind sharing why do you think this happens ? Although I never worked professionally with python, this sentiment matches with my experiences as a user. So I don't have a lot of context why this is the case.

Some siblings in this thread provided some explanations that mostly boils down to 'bad tooling' in one form or another. But this doesn't feel right.

In my opinion if it was just bad tooling this problem would be solved by now.

Re: Ease of maintenance is a feature

#26

Post by ronakjain90 »

cglan wrote: post #2

Super true. One of the best tests of this is setting up a new laptop. Some of the best experiences are when you get a new laptop, and just clone the codebase and everything works as it did before, no special magic. Golang with vendored dependencies seems to be wonderful for this but I've had relatively decent experiences with newer java projects. My worst experiences universally have always been python projects. I do…

Every time I setup a JS project which is older than a few years, it's

1. Extremely difficult to setup the code base, because of dependency spaghetti 2. Lot of breaking changes across different libraries, making maintenance not so easy.

Easiest projects to maintain were written on Go, Java, Ruby,

Re: Ease of maintenance is a feature

#27

Post by aaronblohowiak »

bunderbunder wrote: post #21
aaronblohowiak wrote: post #8

Earlier quoted context omitted.

The hoops that people go through to solve this sometimes creates something even more complex and not great, like forcing all development into a docker container.. Ever try conda though? I’ve had moderate success with pipenv, but tbh I don’t love it as it hides too many things when installing a package fails.

Doing a Python development environment inside of Docker can get particularly obnoxious in the long run because there are approximately a zillion ways that a base image upgrade can break things by changing something about the system Python packages. (And by the time it happens you might have a real mess on your hands because by that point the dev container's dockerfile has quite possibly grown into an undocumented spa…

TMTOWTDI is antithetical to python as a language it’s so odd to me that its packaging and env management is the opposite.

Do you have a suggested solution? Im solo dev for now but will be adding more folks in foreseeable future. Stuck with python for some things (notebooks, model development.)

Re: Ease of maintenance is a feature

#28

Post by aaronblohowiak »

imp0cat wrote: post #20
aaronblohowiak wrote: post #8

Earlier quoted context omitted.

The hoops that people go through to solve this sometimes creates something even more complex and not great, like forcing all development into a docker container.. Ever try conda though? I’ve had moderate success with pipenv, but tbh I don’t love it as it hides too many things when installing a package fails.

I quite like docker for local development. Docker and docker compose do make it incredibly easy to start everything that's required for local development and testing. Your service A needs B and C? Grab those images of B and C and run them all on your machine. The only limitation is the amount of RAM you have available locally. And if you think a bit about your Dockerfiles (ie. have the layers set-up to take advantage…

Yea all of that is stuff I do not want to bother with. At all. “It’s nice if you take on maintenance burden of a bunch of additional moving parts” is the opposite of what I want. If you have to support a diverse set of languages / runtimes / environments and you deploy using containers, maybe it makes sense, but that seems like a use of the complexity budget I’d rather spend on… something else

Re: Ease of maintenance is a feature

#29

Post by stevepike »

I think the kind of application here matters a lot, specifically whether you're trying to make a change to a web app or if you're hacking on library code.

In ruby, for example, I can pretty trivially clone any open source gem and run the specs in On the other hand, getting a development environment running for a company's proprietary web app is often a hassle. Mostly though this isn't because of the language or dependencies, it's because of:

  - Getting all the dependent services up and running (postgres version X, redis Y, whatever else) with appropriate seed data. 
  - Getting access to development secrets
My company (infield.ai) upgrades legacy apps, so we deal with setting up a lot of these. We run them in individual siloed remote developer environments using devcontainers. It works OK once we've configured the service containers.

Re: Ease of maintenance is a feature

#30

Post by bunderbunder »

aaronblohowiak wrote: post #27
bunderbunder wrote: post #21

Earlier quoted context omitted.

Doing a Python development environment inside of Docker can get particularly obnoxious in the long run because there are approximately a zillion ways that a base image upgrade can break things by changing something about the system Python packages. (And by the time it happens you might have a real mess on your hands because by that point the dev container's dockerfile has quite possibly grown into an undocumented spa…

TMTOWTDI is antithetical to python as a language it’s so odd to me that its packaging and env management is the opposite. Do you have a suggested solution? Im solo dev for now but will be adding more folks in foreseeable future. Stuck with python for some things (notebooks, model development.)

Honestly I don't think there's any quick answer, and providing specific advice on Hacker News would be playing into exactly the problem I was pointing out in the last paragraph of my previous post.

My usual advice is to just bite the bullet and invest the time it takes to understand how Python package management and resolution really works under the hood, and how all the various devops approaches that are built on top of it work with it, so that you can make informed decisions and truly own your own stack.

Post reply on HN