Live data from Hacker News

Ask HN: Good Python codebases to read?

news.ycombinator.com

61–70 of 153 posts

Re: Ask HN: Good Python codebases to read?

#61
post #4

Requests - https://github.com/kennethreitz/requests . How to make a usable api. The decisions that went into each method call were fantastic. Great test coverage as well. I use package in most python development.

requests is very useful, but it always gets mentioned as a good python codebase and I'm not sure I agree. One example:

The first thing many users will do is

  requests.get?
Which tells them that it takes some kwargs, but doesn't tell them what those kwargs are. It's easy, especially for a newcomer, to read "optional arguments that `request` takes" and fail to understand that they should look up the docs on (not-really-encouraged-as-part-of-public-API) function `request`. That's pretty bad; those kwargs are important! (The reason is because requests.get is implemented as a call to request(method, ..., kwargs) but the user doesn't care what the implementation-level reason is.)

Beyond that I did look into the codebase once to investigate a possible bug and there were a few python style things I wanted to fix, but I don't remember them so this comment probably sounds kind of annoying (it would annoy me if I were reading it not writing it...). It didn't strike me a really clean codebase. But yes the library is very useful and I'm sure it's a pretty decent python codebase.

  >>> requests.get?
  Signature: requests.get(url, params=None, **kwargs)
  Docstring:
  Sends a GET request.
  
  :param url: URL for the new :class:`Request` object.
  :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
  :param \*\*kwargs: Optional arguments that ``request`` takes.
Here are all the kwargs the user probably wanted to know but failed to find out and was forced to either browse online docs or the source code:

  def request(method, url, **kwargs):
      """Constructs and sends a :class:`Request `.
      :param method: method for the new :class:`Request` object.
      :param url: URL for the new :class:`Request` object.
      :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
      :param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`.
      :param json: (optional) json data to send in the body of the :class:`Request`.
      :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.
      :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.
      :param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': ('filename', fileobj)}``) for multipart encoding upload.
      :param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.
      :param timeout: (optional) How long to wait for the server to send data
          before giving up, as a float, or a (`connect timeout, read timeout
          `_) tuple.
      :type timeout: float or tuple
      :param allow_redirects: (optional) Boolean. Set to True if POST/PUT/DELETE redirect following is allowed.
      :type allow_redirects: bool
      :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy.
      :param verify: (optional) if ``True``, the SSL cert will be verified. A CA_BUNDLE path can also be provided.
      :param stream: (optional) if ``False``, the response content will be immediately downloaded.
      :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair.
      :return: :class:`Response ` object
      :rtype: requests.Response

Re: Ask HN: Good Python codebases to read?

#62
post #29

The django project is a good example of a large opensource project which has aged well. http://github.com/django/django

I know it is a good project, but would you recommend it for someone to read through the codebase? (Honestly, looking at the repo, I don't even know where to start if I wanted to do a read through. Has anyone created a map? :)

I would start with django.core.handlers.base.BaseHandler (https://github.com/django/django/blob/master/django/core/han...), which is Django's Front Controller.

Re: Ask HN: Good Python codebases to read?

#63

Peter Norvig's examples. They are quite short and include much explanation in addition to code. They also include tests and benchmarking code. http://norvig.com/lispy.html http://norvig.com/lispy2.html (Lisp interpreter) http://www.norvig.com/spell-correct.html (Spelling corrector) http://norvig.com/sudoku.html (Sudoku solver) Also his online course Design of Computer programs includes many short, well-explained Pyth…

His coding style is not common to most writers of Python.

He's using overly terse, poorly descriptive variable names, not using multiline strings for docstrings, not indenting where he should, and one-lining if/elif statements and function definitions. This style does not contribute to readability.

This is not how I would want someone just learning Python to learn it.

Re: Ask HN: Good Python codebases to read?

#64

Earlier quoted context omitted.

Openstack is so large it also has a plethora of examples of bad code. Especially in vendor-contributed drivers and such. I wouldn't pay attention to the code so much as the incredible infrastructure and process that manages it and improves it over time. Seriously, I've seen "sychronization threads" that use no synchronization primitives that have a single bare exception handler trap the critical section. The handler…

I happen to agree, Ive been working on it for the past year and its sort of killed my drive to work with something 'big' again. That being said Barbican is quite good, which I think is due to its proof of concept being written in Go[1] before being ported. If you want to learn decent testing Openstack is a good example, the code as a whole not so much. This also only applies if you want to use unittest or their custo…

I still don't get it why they only write in Python (Horizon also has Less, JS) However a lot of code would be way cleaner / clearer when they wouldn't use Python that much. Especially when looking at Horizon. They use Django mainly to use Django, they nearly need nothing from Django at all. The most things they did were quite hacky. I mean horizon is full of magic to do really simple things. And especially a lot of code got glued together over time..

Re: Ask HN: Good Python codebases to read?

#66
post #11

Earlier quoted context omitted.

I have been debating whether or not to read the standard library, pro - it's the standard library, if I learn it I may save myself innumerable hours from reinventing wheels, con - it may not be the best example of code (thinking about the 400 line function from vim to see if there is input from the keyboard. Justified in context, but would be a horrible way to learn some code) Bazaar I may look into, as I know it's a…

I have some experience with the Mercurial code base, which I thought was pretty well engineered. It's not PEP 8, though, so that makes it slightly idiosyncratic. Might make an interesting comparison with Bazaar, though!

In my opinion mercurial's codebase is much better than bazaar's. It's concise and to the point, uses classes when necessary and functions in other cases, following logic is usually straightforward task and it educates well. On the other hand, bazaar's codebase is a mess of hundreds of classes, relations of which are often hard to understand, they are hard to navigate and IMO it's over-engineered. Even writing a plugin for bzr is hard because there is plenty of docs which usually say nothing substantial and codebase is so hard to navigate.

Disclaimer: contributed to mercurial (and wrote some plugins), wrote few plugins for bzr (it was used in one company I worked for).

P.S. Didn't notice right away, hey, Dirkjan! :)

Re: Ask HN: Good Python codebases to read?

#67
post #4

Requests - https://github.com/kennethreitz/requests . How to make a usable api. The decisions that went into each method call were fantastic. Great test coverage as well. I use package in most python development.

requests is very useful, but it always gets mentioned as a good python codebase and I'm not sure I agree. One example: The first thing many users will do is requests.get? Which tells them that it takes some kwargs, but doesn't tell them what those kwargs are. It's easy, especially for a newcomer, to read "optional arguments that `request` takes" and fail to understand that they should look up the docs on (not-really-…

Looks like you did the legwork of listing the params already, so you might as well send a PR with that docstring.

Re: Ask HN: Good Python codebases to read?

#68

Peter Norvig's examples. They are quite short and include much explanation in addition to code. They also include tests and benchmarking code. http://norvig.com/lispy.html http://norvig.com/lispy2.html (Lisp interpreter) http://www.norvig.com/spell-correct.html (Spelling corrector) http://norvig.com/sudoku.html (Sudoku solver) Also his online course Design of Computer programs includes many short, well-explained Pyth…

His coding style is not common to most writers of Python. He's using overly terse, poorly descriptive variable names, not using multiline strings for docstrings, not indenting where he should, and one-lining if/elif statements and function definitions. This style does not contribute to readability. This is not how I would want someone just learning Python to learn it.

While that's true, he also tends to write fairly idiomatic Python -- using base data types a lot, list comprehensions, etc, and his comments and overall insight make them Very worthwhile to read -- especially once you're familiar enough reading it to recognize what he's doing.

The spellchecker still blows my mind, and I don't yet understand the Sudoku puzzle. We can always take the code and re-format / re-name variables to aid our understanding.

Re: Ask HN: Good Python codebases to read?

#70

The django project is a good example of a large opensource project which has aged well. http://github.com/django/django

I don't know what would be of my life right now without Django and I can't praise it enough... but "good example of readable python-code", it is not.

However, I think it's one of the best examples of phenomenal documentation.
Post reply on HN