Writing and linting Python at scale • Hacker News Board

Re: Writing and linting Python at scale

#21

Post by corethree » Wed, Nov 22, 2023, 4:20 PM UTC

lagt_t wrote: post #5

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

Python not only has types but it's type system is superior to typeScript.

Get this python has sum types and exhaustive pattern matching exactly like rust or haskell.

The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor.

edit: (rate limiter is preventing from replying to everyone... I will respond to everyone.)

Re: Writing and linting Python at scale

#22

Post by seabrookmx » Wed, Nov 22, 2023, 4:27 PM UTC

corethree wrote: post #21

lagt_t wrote: post #5

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

Python not only has types but it's type system is superior to typeScript. Get this python has sum types and exhaustive pattern matching exactly like rust or haskell. The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor. edit: (rate limi…

Yes but the if you want to do ahead-of-time type checking you need to run a tool like mypy, and none of those tools are as comprehensive or performant as Typescript. Also the ecosystem of libraries with type annotations is much smaller (Typescript has the first mover advantage, after all).

Re: Writing and linting Python at scale

#23

Post by crabbone » Wed, Nov 22, 2023, 4:27 PM UTC

lagt_t wrote: post #5

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

I've been writing in Python for over ten years, in different roles, for wildly different projects (research, infra, Web, testing, education).

I'm yet to find anything Python was good for. On engineering merits alone Python isn't best for anything, nor is it best for combinations of things. It's silly to think that any tool that works with Python does so because Python was the best language for the job, and they only needed that tool to make it even better.

Most stuff written around Python is yet another layer of band aids on top of a huge ball of band aids that's already there.

So, you may wonder, with all those band aids, didn't they all make it better? And, in a sense, yes, that's what they are for. The band aids improve the experience of the user end. But this isn't how I use the word "better". When I use "better" I mean the quality of execution, not the satisfaction it gives to the user.

Re: Writing and linting Python at scale

#24

Post by crabbone » Wed, Nov 22, 2023, 4:35 PM UTC

jerf wrote: post #14

DarkNova6 wrote: post #10

Earlier quoted context omitted.

The engineers at meta worte a language + runtime on top of PHP to make it work "at scale". If Facebook wants to make something work they have enough resources to throw at the problem to solve it. Regardless of whether it makes sense or not.

I mean, as far as I'm concerned, everything Facebook has done has done nothing but reconfirm how unwise it is to build large-scale infrastructure on dynamic scripting languages. They have the resources to move heaven and earth to do what amounts to turning their dynamic scripting language back into static languages in everything but name, and they have the hole they've dug themselves into that justifies it. I have ne…

> dynamic scripting languages.

Why keep repeating this nonsense? "Dynamic" or "scripting" aren't features of languages. When anyone says something like this, it's like talking about square chicken... (i.e. a category error). Obviously, you had some idea in your mind, and you wanted to communicate it somehow, but your readers will not know what it was unless you make an effort to analyze what you want to say and make sure it doesn't have internal contradictions and that readers can within reason understand what you are trying to say.

You seem to be complaining about something. My guess is that your problem isn't even with the language. Your problem is that some tools for working with your kind of programs are missing or aren't effective.

Re: Writing and linting Python at scale

#25

Post by devjab » Wed, Nov 22, 2023, 4:35 PM UTC

crabbone wrote: post #23

lagt_t wrote: post #5

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

I've been writing in Python for over ten years, in different roles, for wildly different projects (research, infra, Web, testing, education). I'm yet to find anything Python was good for. On engineering merits alone Python isn't best for anything, nor is it best for combinations of things. It's silly to think that any tool that works with Python does so because Python was the best language for the job, and they only…

I’m not really a fan of Python as such, but after a few decades in the industry, I’m beginning to think that being good at being bandaid is “better”. I can’t think of a single tech where we don’t have a bunch of duct tape (as we refer to it), not so much because we want to but because that’s just how things end up in the imperfect world of organisations. I value the techs that fit into this reality more than the ones which don’t, but you’re right, Python isn’t really great for any technology based merits, it’s good because the world is a messy place where being productive with your band aid is often more valuable than using the “better” programming language.

Re: Writing and linting Python at scale

#26

Post by jerf » Wed, Nov 22, 2023, 4:39 PM UTC

crabbone wrote: post #24

jerf wrote: post #14

Earlier quoted context omitted.

I mean, as far as I'm concerned, everything Facebook has done has done nothing but reconfirm how unwise it is to build large-scale infrastructure on dynamic scripting languages. They have the resources to move heaven and earth to do what amounts to turning their dynamic scripting language back into static languages in everything but name, and they have the hole they've dug themselves into that justifies it. I have ne…

> dynamic scripting languages. Why keep repeating this nonsense? "Dynamic" or "scripting" aren't features of languages. When anyone says something like this, it's like talking about square chicken... (i.e. a category error). Obviously, you had some idea in your mind, and you wanted to communicate it somehow, but your readers will not know what it was unless you make an effort to analyze what you want to say and make…

Because it a common term used for a category of languages that everybody understands.

If you have a problem with it, you can take it up with the aforementioned everybody. Personally I think it gets perilously close to the error that if you argue definitions enough you can change reality. It doesn't matter what labels we slap on clearly related languages Python/Perl/Javascript/Lua/PHP/Javascript, I consider it a mistake to build a large system with them in 2023 because of their deficiencies, and given the extreme efforts being exerted trying to fix those very same deficiencies with things like gradual typing and strongly-typed languages like TypeScript that compile down into them, which includes the effort Facebook has exerted, clearly it is not a terribly heterodox opinion.

Re: Writing and linting Python at scale

#27

Post by corethree » Wed, Nov 22, 2023, 4:39 PM UTC

crabbone wrote: post #23

lagt_t wrote: post #5

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

I've been writing in Python for over ten years, in different roles, for wildly different projects (research, infra, Web, testing, education). I'm yet to find anything Python was good for. On engineering merits alone Python isn't best for anything, nor is it best for combinations of things. It's silly to think that any tool that works with Python does so because Python was the best language for the job, and they only…

I've been writing python for a longer amount of time.

What makes python popular is it's the ease of use and debugging. It may not have been type safe but every time it crashed it's extremely easy to find out where and why it crashed. This is not the case languages like JavaScript or for even something like golang which is ironically type safe. I like to emphasize this here. It is more important for a language to be extremely clear about the origin and nature of a runtime error then it is to be type and statically safe. Look at C++. It has powerful static typing but is marred by seg faults and memory leaks which are hidden errors that are extremely hard to tease out.

That being said, modern python now has types. The bandaids on top of python from sheer coincidence turns it into a powerful scalable language for the web rather then an ugly patchwork.

The main problem with python right now would be the performance and the library ecosystem. Performance is of course mainly the gil and libraries are often written with tricks that make static typing impossible. It's just old tech debt getting in the way not the modern language itself. Linters should prevent anyone starting new code using those patterns again.

Other then that, pythons type system + checkers supports even sum types with exhaustive pattern matching added with extreme ease of debugging runtime errors makes python one of the best languages for big projects. That level of safety isn't even offered by go or typescript. The only alternatives are rust or haskell for sum types.

I highly disagree with you. Python is not without old tech debt warts holding it back, but overall it is a superior language.

Re: Writing and linting Python at scale

#28

Post by insanitybit » Wed, Nov 22, 2023, 4:42 PM UTC

crabbone wrote: post #20

n00j wrote: post #18

Earlier quoted context omitted.

In the C#/.NET world this is the standard and it works very well. It definitely increases productivity to not have to double check each and every instance of violating a lint error, as you can just have the fix applied to an entire project without having to worry to much.

> increases productivity to not have to double check each and every instance of violating a lint error, At what cosmic speed should you be pumping out code for this to be a concern? Also, in C#/.NET, where programmers predominantly use MSVS, which is an atrocious editor with MSBuild, which is an atrocious build system, both hampering productivity... Also, plenty of linter errors are actual errors that need non-trivia…

> Also, plenty of linter errors are actual errors that need non-trivial fixing.

These sound like they're not lints at all.

Re: Writing and linting Python at scale

#29

Post by kstrauser » Wed, Nov 22, 2023, 4:45 PM UTC

_whiteCaps_ wrote: post #7

There's a lot of neat Python research coming from Facebook. libcst (which Fixit uses) is super cool - I use it in https://gitlab.com/harford/logzy

Why do you use that instead of flake8, which also finds that?

(Not meant to be a pointed question; genuinely curious.)

Re: Writing and linting Python at scale

#30

Post by insanitybit » Wed, Nov 22, 2023, 4:45 PM UTC

corethree wrote: post #21

lagt_t wrote: post #5

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

Python not only has types but it's type system is superior to typeScript. Get this python has sum types and exhaustive pattern matching exactly like rust or haskell. The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor. edit: (rate limi…

You can't even represent `Json` in Python's type system because it would require a recursive type.

edit: I think this is actually a Python type annotation limitation but it's possible that's mypy, although I think in 99% of cases those are fine to conflate (I have used other Python type systems).