Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python types have an expectations problem

medium.com

Post reply on HN
Print view View original

101–110 of 115 posts

Re: Python types have an expectations problem

#101

Post by gpderetta »

threePointFive wrote: post #48
robertlagrant wrote: post #33

Earlier quoted context omitted.

It has something, but not what you'd call modern type checking: #include int main(void) { unsigned int positive_number = -1; printf("%d", positive_number); return 0; } Prints -1.

I had to check this because my intuition told me that this would generate a warning. GCC and Clang both warn about that assignment with -Wsign-conversion, however that doesn't seem to be enabled with any of -Wall, -Wextra, or -Wpendatic and only clang's -Weverthing would catch it if you weren't specifically looking for it.

that's because assigning -1 to an unsigned number is an idiomatic, safe and portable way to setting all bits to 1. You could argue that an explicit cast should be warranted in this case, and I sort of agree, but it would break way too much code.

Re: Python types have an expectations problem

#102

Post by gpderetta »

SAI_Peregrinus wrote: post #61
maxfurman wrote: post #54

Earlier quoted context omitted.

I must be missing something here. Wouldn't an unsigned int be unable to represent -1 since the sign bit is part of the value?

C implicitly converts integer types as needed. The specific conversion rules are rather convoluted, see section 6.3 of the ISO C standard for the details. Before C23 this was implementation defined, but since C23 now mandates two's complement representation of signed integers and wrapping for unsigned overflow (as before) the `-1` is converted to UINT_MAX. Then the `%d` format specifier performs a conversion from `un…

I'm pretty sure that this was well defined and portable well before C23. Although the representation was implementation defined, IIRC signed to unsigned has always been guaranteed to be as-if in two-complement.

Re: Python types have an expectations problem

#103

Post by ReflectedImage »

nickm12 wrote: post #93
ReflectedImage wrote: post #59

Using static typing in Python is a serious mistake. The point of static typing is to allow an compiler to compile your code for performance reasons and nothing else. People do not normally compile Python code with a compiler. Static typing significantly increases code length compared to duck typing, significantly increases development times and significantly increases bug count per delivered software feature. It addi…

I previously worked at Dropbox, which had server codebase consisting of over 2 million lines of (untyped) Python. While I was there, Dropbox introduced type annotations into the codebase and used it as a testing group for Mypy, which was being developed largely at Dropbox, with both Jukka Lehtosalo and Guido van Rossum on staff. I can say from that experience that pretty much everything you are claiming is wrong. Dev…

I've seen both the statically typed Python 1Mloc monolithic and the ducked typed 1Mloc micro-service based approach in practice.

I'm telling you the ducked typed micro-service based approach is the way to go.

But you sort of know that "not to write a 2Mloc server in Python", don't you? If we encourage developers to use static typing they will be writing 2Mloc monolithic servers. That's just the type of code that static typing encourages.

Encouraging static typing in Python is putting them down the path of disaster.

Re: Python types have an expectations problem

#104

Post by SAI_Peregrinus »

gpderetta wrote: post #102
SAI_Peregrinus wrote: post #61

Earlier quoted context omitted.

C implicitly converts integer types as needed. The specific conversion rules are rather convoluted, see section 6.3 of the ISO C standard for the details. Before C23 this was implementation defined, but since C23 now mandates two's complement representation of signed integers and wrapping for unsigned overflow (as before) the `-1` is converted to UINT_MAX. Then the `%d` format specifier performs a conversion from `un…

I'm pretty sure that this was well defined and portable well before C23. Although the representation was implementation defined, IIRC signed to unsigned has always been guaranteed to be as-if in two-complement.

You're probably correct, but I didn't have the earlier standards as quickly to hand to cite. The storage representation being twos complement is new, I didn't mean to imply that the conversion was as well.

Re: Python types have an expectations problem

#105

Post by ReflectedImage »

_dain_ wrote: post #85
ReflectedImage wrote: post #83

Earlier quoted context omitted.

Over the years, plenty of people have wanted to show that static typing gives code correctness benefits over dynamic typing. Everyone who tries just falls flat on their faces.

again, specifics please. which languages are you thinking of? I don't see how e.g. [1] or [2] are "falling flat on their faces". [1] https://fsharpforfunandprofit.com/posts/designing-with-types... [2] https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va... or even something as basic as non-nullable types, enforced at compile time. a clear, obvious win.

Those are some nice blog articles but the general consensus is that on a per-line basis duck typed and statically typed code have the same amount of bugs.

The duck typed code is also significantly shorter, less lines of code per software feature means less bugs per released software feature.

It's very difficult to compete against just having less code. You only needed to write 1000 lines of code instead of 3000 lines of code? Then you just eliminated 66% of the bugs.

Static typing on the other hand catches less than 1% of bugs.

So you can go for the duck typing approach and remove 66% of the bugs in your code or the static typing approach and remove 1% of the bugs in your code.

Which approach do you think works better?

Re: Python types have an expectations problem

#106

Post by ReflectedImage »

frou_dh wrote: post #97
ReflectedImage wrote: post #59

Using static typing in Python is a serious mistake. The point of static typing is to allow an compiler to compile your code for performance reasons and nothing else. People do not normally compile Python code with a compiler. Static typing significantly increases code length compared to duck typing, significantly increases development times and significantly increases bug count per delivered software feature. It addi…

> The point of static typing is to allow an compiler to compile your code for performance reasons and nothing else. The relevant term is Type Checker. Static Type Checker: Something that checks types statically (i.e. by inspecting the source code without running it). Doesn't necessarily say anything about compilation, execution strategy, or indeed if the user program will ever be run at all.

Static type checking catches The majority of bugs in code are about the behavior of the code rather than the typing. You require unit tests or some other more advanced technique.

Re: Python types have an expectations problem

#107

Post by gpderetta »

nerdponx wrote: post #79
hermitdev wrote: post #17

Earlier quoted context omitted.

In practice, this would be solved with `typing.overload`[0]. Using you example: from typing import overload @overload def fooify(x: int) -> int: ... @overload def fooify(x: list[int]) -> list[int]: ... def fooify(x: list[int] | int) -> list[int] | int: if isinstance(x, list): return [fooify(_x) for _x in x] return x * 2 [0] https://docs.python.org/3/library/typing.html#overload

And for an example of the practical limits of @overload, take a look at the Pandas type hints: https://github.com/pandas-dev/pandas/blob/dc5586baa9e4731805... Meanwhile it's not even possible to express such things in other static type systems. So I'm not exactly an unhappy customer, but it does put certain things tantalizingly close, but still out of reach without a ton of clunky boilerplate and LoC explosion.

> it's not even possible to express such things in other static type systems

what do you mean? It seems relatively straightforward.

Re: Python types have an expectations problem

#108

Post by ReflectedImage »

pjmlp wrote: post #96
ReflectedImage wrote: post #59

Using static typing in Python is a serious mistake. The point of static typing is to allow an compiler to compile your code for performance reasons and nothing else. People do not normally compile Python code with a compiler. Static typing significantly increases code length compared to duck typing, significantly increases development times and significantly increases bug count per delivered software feature. It addi…

> The point of static typing is to allow an compiler to compile your code for performance reasons and nothing else Sure, if one doesn't care about their code quality, after all everyone loves to write unit tests for every possible use case.

If you don't test the behavior of your code then it is wrong. Whether the code could in theory be compiled isn't a very good test case.

Static type checking has a measured bug catch rate of <1%. So the quality of any code where the programmer depends on static typing to verify correctness rather than unit tests is very very low.

Re: Python types have an expectations problem

#109

Post by pjmlp »

ReflectedImage wrote: post #108
pjmlp wrote: post #96

Earlier quoted context omitted.

> The point of static typing is to allow an compiler to compile your code for performance reasons and nothing else Sure, if one doesn't care about their code quality, after all everyone loves to write unit tests for every possible use case.

If you don't test the behavior of your code then it is wrong. Whether the code could in theory be compiled isn't a very good test case. Static type checking has a measured bug catch rate of <1%. So the quality of any code where the programmer depends on static typing to verify correctness rather than unit tests is very very low.

Measured by whom?

Re: Python types have an expectations problem

#110

Post by ReflectedImage »

drekipus wrote: post #88
ReflectedImage wrote: post #82

Earlier quoted context omitted.

We assume in good faith that statements made in comments on social media are true. If this was an academic paper, then perhaps you would be correct. But on social meda, asking for sources for what is basically common programming knowledge is borderline trolling. If s/he thinks that what I'm saying is wrong so strongly then they can go back that up with something. They will obviously fail terribly because I'm correct…

I've thought that adding the type information is to help less bugs, so I'd be curious to see evidence of the contrary. I use python add my day job, and I hate writing and dealing with typing. I do admit it makes it easier to reason about other people's code though, especially when faced with libraries that don't have any hints.

People get this weird idea in their heads that if they do more stuff than what they have done is better in some way. In practice however simpler code usually wins out.

Docstrings are usually a lot better than type hints for that purpose.

Post reply on HN