Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python at Scale: Strict Modules

instagram-engineering.com

Post reply on HN
Print view View original

71–80 of 259 posts

Re: Python at Scale: Strict Modules

#71

Post by coldtea »

TylerE wrote: post #23
miki123211 wrote: post #21

This is yet another example of the divide between wizarding and engineering[1]. When you're a small startup, what matters is the expressiveness of your language, and the ability do do a lot of things very very quickly. Type safety, performance, readability, those things don't matter. You're just a bunch of engineers who know the whole codebase inside out, you're pretty certain of what you're doing. In short, you're w…

Who is starting large-scale new projects in Java in 2019?

Countless companies, huge and small -- from Apple and Amazon, to Google and your friendly local startup, plus all the enterprise world that's not a .NET shop...

In what parallel universe is not Java immensely popular or not used for green projects?

Re: Python at Scale: Strict Modules

#72

Post by jedberg »

It's interesting to me that they are going down this path instead of the microservices path. This seems like something ripe for slowly breaking down into microservices.

Someone made a change that took down production because of non-deterministic outcomes? How about break out whatever they were changing into it's own service? With proper fallbacks, breaking that part shouldn't take down all of production again.

To be clear, I'm not saying microservices will solve all their problems or be less work. I'm just saying that with an equal level of effort, they would probably get more overall reliability by having multiple services, they'd be able to use multiple languages, whatever is suited to the task at hand, be able to deploy even more often with less risk, and be able to isolate these types of "change on import" behavior to a much smaller surface on any given deployment.

Re: Python at Scale: Strict Modules

#73

Post by SEJeff »

TylerE wrote: post #39
pjmlp wrote: post #29

Earlier quoted context omitted.

Plenty of companies free of magpie developers.

I asked for examples , not platitudes.

Twitter, and they write some really good open source stuff too. If you're writing rpc services in java I'd almost argue you should default to considering Finagle:

https://twitter.github.io/finagle/

Re: Python at Scale: Strict Modules

#74

Post by lucidone »

TylerE wrote: post #23
miki123211 wrote: post #21

This is yet another example of the divide between wizarding and engineering[1]. When you're a small startup, what matters is the expressiveness of your language, and the ability do do a lot of things very very quickly. Type safety, performance, readability, those things don't matter. You're just a bunch of engineers who know the whole codebase inside out, you're pretty certain of what you're doing. In short, you're w…

Who is starting large-scale new projects in Java in 2019?

I write JavaScript all day and wish I could write in Java.

Re: Python at Scale: Strict Modules

#75

Post by GordonS »

TylerE wrote: post #35
victor106 wrote: post #25

Earlier quoted context omitted.

One of my clients handles 90% of all the pbm routing in the US, which is millions of transactions per second. They started to completely modernize the application on java... primarily Spring Boot and Apache Geode. After some optimizations they are very happy with the performance and expressiveness of modern java.

I said new projects. Not upgrades.

Not the OP, but I understood their comment to mean 'a rewrite in Java of a legacy system written in some other language'?

Re: Python at Scale: Strict Modules

#76

Post by joshuamorton »

scrollaway wrote: post #49
timothycrosley wrote: post #45

Earlier quoted context omitted.

> No importing basic type This at least is being solved in Python: https://www.python.org/dev/peps/pep-0563/ > inline interfaces Typed dicts in 3.8 look pretty similar at cursory glance https://www.python.org/dev/peps/pep-0589/ I'll definitely play around with it more!

Watch the difference: A variable that can be an object with two elements (one of them being a list of strings), or a tuple of exactly two strings. // typescript let t: {a: string[], b: number} | [string, string] # python 3.8 from typing import TypedDict, Tuple, Union class SomeTypedDict(TypedDict): a: List[str] b: Union[float, int] t: Union[SomeTypedDict, Tuple[str, str]] I had to google a bunch to figure out how to…

Granted, in python, I'd call the use of a typed dict a smell. If you're able to spend the time creating the typed dict, just promote it to a dataclass. Using python ~3.9, this will look like

    @dataclasses.dataclass  # or @attr.s
    class MyStruct:
        a: List[str]
        b: int|float

    t: MyStruct|Tuple[str,str]
But MyStruct will be an actual object that can be manipulated as an object. And if you want to accept any object that fits that interface, instead of just instances of MyStruct,

    class MyStructTmpl(Protocol):
        a: List[str]
        b: int|float
Then

    def f(thing: MyStructTmpl|Tuple[str,str]) -> bool:
        return True

    f(MyStruct(a=['a'], b=2))
would typecheck.

In JS having the typed-dict type makes sense because you're often working with arbitrary objects with who knows what attributes, but in python that isn't the case. There's fairly succinct and powerful tools (now, anyway) to define record types.

Re: Python at Scale: Strict Modules

#77

Post by orf »

allan_s wrote: post #11

> This means that just by importing this module, we're mutating global state somewhere else. Yes, this ! That's why I hate Django and some flask app the most for, the fact that by importing a module, you're implicitly creating a database connection, and a lot of other magic stuff, which mean that now I can't import a constant defined in said module outside of `python manage.py` Also as said below in the article, sudd…

Django doesn’t create database connections on import. That would be madness.

It does create an object that can (lazily) connect to the database, so it needs the required database drivers installed. It also needs the required information about _how_ to connect to the database, so it needs the settings loaded.

That's why you need to use `django.setup()` before, to tell it what settings to load. You should never be importing random Django models without this configured, simply because they cannot be used and will not work. We think an exception saying "don't do this, call django.setup()" is less confusing at import time is than "Databases not configured" at runtime. Not that it would even reach that, because you might be using a field from a third party application that needs to be initialized (i.e INSTALLED_APPS configured) or that relies on a configured settings (maybe an encrypted field that needs your SECRET_KEY available).

Stop making it hard, just write a management command. It's super easy.

Re: Python at Scale: Strict Modules

#78

Post by lern_too_spel »

TylerE wrote: post #23
miki123211 wrote: post #21

This is yet another example of the divide between wizarding and engineering[1]. When you're a small startup, what matters is the expressiveness of your language, and the ability do do a lot of things very very quickly. Type safety, performance, readability, those things don't matter. You're just a bunch of engineers who know the whole codebase inside out, you're pretty certain of what you're doing. In short, you're w…

Who is starting large-scale new projects in Java in 2019?

For large scale projects, Java and C++ remain the go-to languages. I've seen a little bit of Go start to show up but no others. Other languages are used for libraries (Rust, C), only at certain employers (OCaml, Erlang), or for small-scale projects (nearly everything else).

Re: Python at Scale: Strict Modules

#79

Post by coldtea »

TylerE wrote: post #39
pjmlp wrote: post #29

Earlier quoted context omitted.

Plenty of companies free of magpie developers.

I asked for examples , not platitudes.

And yet you only offered a misinformed platitude in the form of a question.

Google does all kind of new Java work (Golang contrary to myth, is just one of the languages Google uses for internal stuff, and niche at that), Amazon of course, most of Apple's backend services are Java, Twitter, AirBnB, Uber, LinkedIn, TripAdvisor, and tons of others use Java, and write green stuff in it all the time...

Post reply on HN