Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

codereviewdoctor.medium.com

Post reply on HN
Print view View original

191–200 of 339 posts

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#191

Post by jstx1 »

andi999 wrote: post #188
fault1 wrote: post #150

Earlier quoted context omitted.

the zen of python was written in the 90s. from that context it makes sense, because the only goal of python in the 1990s was to be more popular than perl, which was notorious in having many ways of doing the same thing. but yeah, python had had significant feature creep over the years, it's nowhere near the small clear lang it used to be.

And still no expressive switch/case statement, breaking out of loops and ending scripts early (for explorative programming).

> And still no expressive switch/case statement

There's match/case in 3.10 - https://www.python.org/dev/peps/pep-0636/

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#192

Post by bcrl »

rikatee wrote: post #47
bcrl wrote: post #24

Earlier quoted context omitted.

It's a class of error that would be caught by even the most basic testing. A better title for the article is that 5% of 666 Python repos have typos that demonstrate the code in them that is completely untested. It doesn't matter which language it is: untested code is untested code in any language.

unfortunately like 10% of the bugs were in the tests themselves. e.g., the sentry one https://codereviewdoctor.medium.com/5-of-666-python-repos-ha... the tests are only as good as the code they're written with, and as good as the code review process they were merged under.

One of the habits I have when writing kernel code is to intentionally break code in the kernel to verify that my test is checking what I think it's checking. That's because of a lesson I learned a long, long time ago after someone reviewed my code and caught a problem: when your code has security implications, you need to make sure the boundary conditions that your tests are supposed to cover actually get tested. Having implemented a number of syscalls exposted to untrusted userland over the years, this habit has saved my bacon several times and avoided CVEs.

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#194

Post by voussoir »

onphonenow wrote: post #164
thrdbndndn wrote: post #32

Earlier quoted context omitted.

I luckily never accidently used this space-concatenation thing, but I've been bitten by the fact a=(1) doesn't create 1-element tuple multiple times in my early days learning Python.

I still don't understand why it doesn't! So I still get bit from time to time.

If a person decides to add parentheses to some booleans or arithmetic,

    (4 + 5) * (8 + 2)

    (this and that) or (theother)
These elements should not become 1-tuples after the interior contents are evaluated. I sometimes add parentheses even around single variables just for visual clarity.

Also, this allows you to do dot-access on int / float literals, if you want to

    # doesn't work
    4.to_bytes(8, 'little')

    # works
    (4).to_bytes(8, 'little')

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#195

Post by rikatee »

joatmon-snoo wrote: post #147
karolkozub wrote: post #5

I really like the idea of automated code review tools that point out unusual or suspicious solutions and code patterns. Kind of like an advanced linter that looks deeper into the code structure. With emerging AI tools like Github Copilot, it seems like the inevitable future. Programming is very pattern-oriented and even though these kinds of tools might not necessarily be able to point out architectural flaws in a co…

I actually recently joined a startup working on this problem! One of our products is a universal linter, which wraps the standard open-source tools available for different ecosystems, simplifies the setup/installation process for all of them, and a bunch of other usability things (suppressing existing issues so that you can introduce new linters with minimal pain, CI integration, and more): you can read more about it…

cool product :) it is just linting or do any of the tools do code transformation to offer the fix for the lint failure? (code review doctor also offers the fix if you add the github PR integration)

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#196

Post by ehsankia »

Nice! Internally we have a PCRE support on our code search and I regularly run a regex to find and fix these. I've also found a ton on opensource project which I've been trying to fix:

https://github.com/YosysHQ/prjtrellis/pull/176

https://github.com/UWQuickstep/quickstep/pull/9

https://github.com/tensorflow/tensorflow/pull/51578

https://github.com/mono/mono/pull/21197

https://github.com/llvm/llvm-project/pull/335

https://github.com/PyCQA/baron/pull/156

https://github.com/dagwieers/pygments/pull/1

https://github.com/zhuyifei1999/guppy3/pull/12

https://github.com/pyusb/pyusb/pull/277

https://github.com/KhronosGroup/Vulkan-ValidationLayers/pull...

It is indeed a very common mistake in Python, and can be very hard to debug. It bit me once and wasted a whole day for me, so I've been finding/fixing them ever since trying to save others the same pain I went through.

EDIT: I will point out that I've found this error in other non-Python code too, such as c++ (see the 2nd PR for example).

Here's the regex for anyone curious:

[([{]\s*\n?(\s*['"](\w)+['"],\n)+(\s*['"]\w+['"]\n)(\s*['"]\w+['"],\n)*

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#197

Post by aeturnum »

iooi wrote: post #179
aeturnum wrote: post #138

Earlier quoted context omitted.

I love how simple and forgiving Python is for small projects. The "trailing comma creates a tuple" situation comes out of, as far as I can tell, a desire to create maximally convenient syntax in the scenarios where tuples are intended. I think that's great for small code! I just wish that the core team would take that same zeal for a "pythonic" experience with small code and use it to develop more scaled-up systems f…

If you use mypy (as anyone should for any non-hobby Python usage) then Python has one of the strongest type systems available. Optional types, generics, "Any" escape hatches, everything you could want.

mypy is a great project and I agree that basically every project at scale should use it. However, I think you're wrong about the strength of the Python type system and what a good type system can "get" you. I think mypy both does an amazing job at static checking and that more powerful type systems go far beyond static checks and into changing how you structure and write code. The newly introduced "structural pattern matching" they just introduced[1] is an example of the kind of feature that could be usefully expanded by making type a first-class part of the Python runtime.

Again - the dynamism of Python means teams can write amazing extensions to Python (like mypy), but that isn't a replacement for the core team having a plan for how they think typing information should be used at runtime. Their current answer seems to be "nothing," which disappoints me.

[1] https://www.python.org/dev/peps/pep-0622/

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#198

Post by scbrg »

onphonenow wrote: post #164
thrdbndndn wrote: post #32

Earlier quoted context omitted.

I luckily never accidently used this space-concatenation thing, but I've been bitten by the fact a=(1) doesn't create 1-element tuple multiple times in my early days learning Python.

I still don't understand why it doesn't! So I still get bit from time to time.

Presumably because parantheses don't really have anything to do with tuples, it's commas that do. Parantheses are there to help the parser group things in case of ambiguity, and to support expressions spanning multiple lines.

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#199

Post by a_t48 »

voltagedivider wrote: post #125
xigoi wrote: post #76

Earlier quoted context omitted.

I can think of at least 2 obvious ways to iterate through something: for loops and comprehensions.

You're right that both iterate through something but `for` loops and comprehensions aren't used as if they were interchangeable. For example, you'll sometimes see people do bad stuff like this: >>> lst = [] >>> >>> [lst.append(i + i) for i in range(10)] [None, None, None, None, None, None, None, None, None, None] >>> >>> lst [0, 2, 4, 6, 8, 10, 12, 14, 16, 18] >>> When they should be doing this: >>> lst = [] >>> >>>…

    lst = [range(0, 10, 2)]

Re: 5% of 666 Python repos had comma typo bugs (inc V8, TensorFlow and PyTorch)

#200

Post by orlp »

a_t48 wrote: post #199
voltagedivider wrote: post #125

Earlier quoted context omitted.

You're right that both iterate through something but `for` loops and comprehensions aren't used as if they were interchangeable. For example, you'll sometimes see people do bad stuff like this: >>> lst = [] >>> >>> [lst.append(i + i) for i in range(10)] [None, None, None, None, None, None, None, None, None, None] >>> >>> lst [0, 2, 4, 6, 8, 10, 12, 14, 16, 18] >>> When they should be doing this: >>> lst = [] >>> >>>…

lst = [range(0, 10, 2)]

That's wrong in multiple ways. You want

    lst = list(range(0, 20, 2))
Post reply on HN