Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Best Practices for Working with Configuration in Python Applications

tech.preferred.jp

Post reply on HN
Print view View original

61–69 of 69 posts

Re: Best Practices for Working with Configuration in Python Applications

#61

Post by thomk »

alanfranz wrote: post #60
thomk wrote: post #59

Earlier quoted context omitted.

Configuration, by definition, modifies the software. How much it modifies the software is determined by the software author. If there aren't limits imposed in your example (json) you're still going to get unexpected results. You'll have to limit what's accepted. Also, what if I allow a config.py file then translate that to .json then back to python? That is no better than just allowing the consumption of a python sou…

I don't know whether it's vogue or not, I think it's just a bad idea, because you can't restrict HOW MUCH is is modifying the software, and then people needs to know the nuisances of your programming language. What if I need some configuration to be user (not admin) configurable? What if I don't want to learn programming language X just to use a software? > If there aren't limits imposed in your example (json) you're…

I appreciate the thoughtful response and I respectfully disagree.

I agree with your point about non-technical users, I addressed that in the top level comment on this thread.

Limiting what is accepted isn't hard at all, here's how you do it:

In your config.py module, have a Config class that subclasses a superclass named, say, ConfigBase. ConfigBase implements __init_subclass__ and in there, you can dictate how a subclass is configured and raise descriptive errors if your rules are not followed. You can do the same thing with a metaclass.

With this approach the class does not have to be mainline executable. You can pluck out what you need and use those plucked items however you see fit.

You are essentially consuming a python script as a config file and not actually running it.

Re: Best Practices for Working with Configuration in Python Applications

#62

Post by brewmarche »

alanfranz wrote: post #58
brewmarche wrote: post #38

Earlier quoted context omitted.

Actually UTC + timezone is exactly the wrong thing for "wall clock times" (things like meetings or departures where the time at the location is relevant). The conversion to UTC will lose the original local time so you cannot retrieve it once time zone data changes, unless you perform reconversions every time you detect such a change in tzdata. And countries changing time zones happens more often than we think (and al…

> The conversion to UTC will lose the original local time so you cannot retrieve it once time zone data changes, unless you perform reconversions every time you detect such a change in tzdata I don't agree/disagree with your point, and neither I do agree/disagree with GP on the topic, but why couldn't I retrieve the original time? If, for an event, I save UTC + event TimeZone, I can always get back to the original ti…

Offset timezones (e.g. UTC+2) don't change, what changes are local timezones (e.g. Europe/Rome).

For example here Turkey decided to change daylight savings time: https://github.com/JodaOrg/joda-time/issues/403 (if you have a look at the tzdata database you will find more, this one I remember because Turkey went back and forth about this).

If your timezone database changes you cannot retrieve the original wall clock time, unless you have a temporal timezone database and remember the date of conversion to UTC.

And if you used offsets instead of local timezones to begin with, you cannot even infer which offsets to change unless you have location data saved as well.

Here is a blog post by Jon Skeet https://codeblog.jonskeet.uk/2019/03/27/storing-utc-is-not-a... where he says:

> For me, the key difference between the options is that in option 3, we store and never change what the conference organizer entered. The organizer told us that the event would start at the given address in Amsterdam, at 9am on July 10th 2022. That’s what we stored, and that information never needs to change (unless the organizer wants to change it, of course). The UTC value is derived from that “golden” information, but can be re-derived if the context changes – such as when time zone rules change

Re: Best Practices for Working with Configuration in Python Applications

#63

Post by brewmarche »

thomaslkjeldsen wrote: post #50
brewmarche wrote: post #38

Earlier quoted context omitted.

Actually UTC + timezone is exactly the wrong thing for "wall clock times" (things like meetings or departures where the time at the location is relevant). The conversion to UTC will lose the original local time so you cannot retrieve it once time zone data changes, unless you perform reconversions every time you detect such a change in tzdata. And countries changing time zones happens more often than we think (and al…

I believe this "wall clock time" approach is broken by design as it pushes the burden of figuring out timezone details to those who are not located in that particular timezone. A fair and therefore safer approach is to decide that by protocol the legally binding time is defined in UTC. Your system will translate UTC times to and from any given local time using the IANA time zone database which is regularly updated. E…

I think whether UTC or wall clock time is binding is a problem in the legal and planning (so the business) domain and has to be treated as an external input to the software engineering problem.

Although you are of course free to advocate for UTC. I remember Swatch trying to establish something similar and it never took off: https://en.wikipedia.org/wiki/Swatch_Internet_Time

Re: Best Practices for Working with Configuration in Python Applications

#64

Post by rochacbruno »

fermigier wrote: post #11

Quick list of Python libraries that help with application configuration: - Python application configuration -> https://github.com/edaniszewski/bison - Configuration with env variables for Python -> https://github.com/hynek/environ_config - Configuration library for python projects -> https://github.com/willkg/everett - Strict separation of config from code -> https://github.com/henriquebastos/python-decouple This is…

https://github.com/rochacbruno/dynaconf

Re: Best Practices for Working with Configuration in Python Applications

#65

Post by kissgyorgy »

ggolu2 wrote: post #3
alanfranz wrote: post #2

While most points are valid, I feel some pieces are missing in this article. What should I finally do? How to put everything together without creating an hard-to-maintain mess of casting/parsing/configuration? Should I manually cast strings to integers (or other types) for all and each value I parse? Where do I keep default values? It's cumbersome to have them embedded in code for get(). I usually want a) a default c…

What’s naive and dangerous about Python’s Datetime objects?

Basically everything.

Re: Best Practices for Working with Configuration in Python Applications

#66

Post by alanfranz »

brewmarche wrote: post #62
alanfranz wrote: post #58

Earlier quoted context omitted.

> The conversion to UTC will lose the original local time so you cannot retrieve it once time zone data changes, unless you perform reconversions every time you detect such a change in tzdata I don't agree/disagree with your point, and neither I do agree/disagree with GP on the topic, but why couldn't I retrieve the original time? If, for an event, I save UTC + event TimeZone, I can always get back to the original ti…

Offset timezones (e.g. UTC+2) don't change, what changes are local timezones (e.g. Europe/Rome). For example here Turkey decided to change daylight savings time: https://github.com/JodaOrg/joda-time/issues/403 (if you have a look at the tzdata database you will find more, this one I remember because Turkey went back and forth about this). If your timezone database changes you cannot retrieve the original wall clock t…

I'm sorry for the late response; yes, you are right that for future events the "right way to do it" is saving the place + local time. I think we were speaking of slightly different things.

Re: Best Practices for Working with Configuration in Python Applications

#67

Post by brewmarche »

alanfranz wrote: post #66
brewmarche wrote: post #62

Earlier quoted context omitted.

Offset timezones (e.g. UTC+2) don't change, what changes are local timezones (e.g. Europe/Rome). For example here Turkey decided to change daylight savings time: https://github.com/JodaOrg/joda-time/issues/403 (if you have a look at the tzdata database you will find more, this one I remember because Turkey went back and forth about this). If your timezone database changes you cannot retrieve the original wall clock t…

I'm sorry for the late response; yes, you are right that for future events the "right way to do it" is saving the place + local time. I think we were speaking of slightly different things.

No problem. Yes this is for future events where we want to coordinate people at a certain place.

For recording when something happened use UTC or UTC plus fixed offset.

Do you have a third context of using time?

Re: Best Practices for Working with Configuration in Python Applications

#68

Post by gimboland »

pydry wrote: post #20
alanfranz wrote: post #4

Earlier quoted context omitted.

Your question implies that you don't know about the nuisances of the datetime library :-) (see https://docs.python.org/3/library/datetime.html it's the first paragraph!) Python datetime objects, by design, can be naive or timezone-aware. Timezone-aware datetime objects are OK; they identify a certain instant in time. Naive datetime objects are Python-only abstractions (AFAIK) that don't identify anything in the real…

>What does the library do? Throw an exception? Associate an arbitrary timezone (e.g. UTC)? Associate the local, current timezone? Naive datetime is what datetime.utcnow() returns. UTC is essentially a "default" timezone. I've always thought it made most sense in a library to assume it's UTC.

https://blog.ganssle.io/articles/2019/11/utcnow.html

Re: Best Practices for Working with Configuration in Python Applications

#69

Post by okomestudio »

jayjader wrote: post #48
okomestudio wrote: post #45

I have been working on a minimalistic application config library for Python, aiming to consolidate config loading from files, environment variable, and command-line argument parsing. It's an alpha, so please feel free to provide feedback if app configuration has been your pain point. https://github.com/okomestudio/resconfig

FYI that sounds like exactly the same feature set of 'python-decouple'.

Indeed "python-decouple" looks like it serves similar niche. (I didn't know of the package. Thanks for letting me know.) I think I'd like to target a smaller niche though, someone writing a small applications, with a little more flexibility in things like YAML support and dynamic loading. Unless "decouple" eventually supports similar features, I want to keep experimenting.
Post reply on HN