Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Types for Python HTTP APIs

instagram-engineering.com

Post reply on HN
Print view View original

21–30 of 55 posts

Re: Types for Python HTTP APIs

#21

Post by rytill »

joshuamorton wrote: post #16
dna_polymerase wrote: post #4

The whole Type thing in Python and all the workarounds to make Python type-safish looks and feels absolutely pathetic. I have no idea why a company like Facebook takes so much time to apologize for the inadequacy of Python as a programming language in the broader sense. Especially since they are utilizing a service based architecture they easily could make the switch to a language that actually supports types. Also t…

Python does support types though. It's type system is, imo more pleasant and powerful than Javas.

Can you elaborate on this? I’d love to actually use types in Python.

Re: Types for Python HTTP APIs

#23

Post by acemarke »

rytill wrote: post #21
joshuamorton wrote: post #16

Earlier quoted context omitted.

Python does support types though. It's type system is, imo more pleasant and powerful than Javas.

Can you elaborate on this? I’d love to actually use types in Python.

Python 3 has optional static typing:

https://docs.python.org/3/library/typing.html

https://realpython.com/python-type-checking/

Re: Types for Python HTTP APIs

#24

Post by ChrisSD »

dna_polymerase wrote: post #4

The whole Type thing in Python and all the workarounds to make Python type-safish looks and feels absolutely pathetic. I have no idea why a company like Facebook takes so much time to apologize for the inadequacy of Python as a programming language in the broader sense. Especially since they are utilizing a service based architecture they easily could make the switch to a language that actually supports types. Also t…

Just to clarify, Python has always been strongly typed. Every object has to have an explicit concrete type. However this type is only known at runtime and variables can be bound to any type of object. Aka dynamic typing.

What the "whole Type thing" adds is type hints. That is to say they tell the programmer what types are expected. These hints can also be used in static analysis to try to restrict the types that can be bound to any particular variable. This can be useful to detect bugs in large code bases.

Re: Types for Python HTTP APIs

#25

Post by dragonsh »

In our startup we use marshmallow [1] to validate and make REST API type aware. So marshmallow models validates the data types based on validation rules for request/response.

In place where we need client to be able to define the validation the request/response will consist of data, validation json schema. In this case use jsonschema validator [2] [3].

So for every json request/request data, it's passed through a marshmallow model which validates the json. In some cases we use jsonschema validator when we have dynamic json data with schema definition for validations.

For response the function return value pass through a marshmallow model. We are moving towards making all of internal functions/methods with type annotations to generate documentation using Sphinx [4] plugin.

We are not Instagram but are very happy with it and can replace flask library with bottle or other wsgi compatible framework and it will still work.

[1] https://marshmallow.readthedocs.io/en/stable/

[2] https://python-jsonschema.readthedocs.io/

[3] https://pydantic-docs.helpmanual.io/

[4] https://pypi.org/project/sphinx-autodoc-typehints/#data

Re: Types for Python HTTP APIs

#26

Post by henryfjordan »

dragonsh wrote: post #25

In our startup we use marshmallow [1] to validate and make REST API type aware. So marshmallow models validates the data types based on validation rules for request/response. In place where we need client to be able to define the validation the request/response will consist of data, validation json schema. In this case use jsonschema validator [2] [3]. So for every json request/request data, it's passed through a mar…

Marshmallow is good but it can be slow. We also use it over a Flask API for input/output serialization and in the worse cases it can take a significant amount of time if the objects are large enough (maybe a hundred milliseconds). We also use the Marshmallow models in conjunction with a project called `flask-apispec` by the same authors to generate Swagger docs.

I've wanted to explore using the Typing module to replace Marshmallow since it started making the rounds to see if it results in better performance, but haven't had a chance. I would have liked to see Instagram release a library to go with this blog post so I don't have to do as much legwork.

Re: Types for Python HTTP APIs

#27

Post by j88439h84 »

henryfjordan wrote: post #26
dragonsh wrote: post #25

In our startup we use marshmallow [1] to validate and make REST API type aware. So marshmallow models validates the data types based on validation rules for request/response. In place where we need client to be able to define the validation the request/response will consist of data, validation json schema. In this case use jsonschema validator [2] [3]. So for every json request/request data, it's passed through a mar…

Marshmallow is good but it can be slow. We also use it over a Flask API for input/output serialization and in the worse cases it can take a significant amount of time if the objects are large enough (maybe a hundred milliseconds). We also use the Marshmallow models in conjunction with a project called `flask-apispec` by the same authors to generate Swagger docs. I've wanted to explore using the Typing module to repla…

> Toasted Marshmallow implements a JIT for marshmallow that speeds up dumping objects 10-25X (depending on your schema).

https://github.com/lyft/toasted-marshmallow

Re: Types for Python HTTP APIs

#28

Post by timothycrosley »

Been doing this before the typing module was released, using hug: http://www.hug.rest/

FastAPI has a lot of advantages at this point for someone looking to do the same: https://github.com/tiangolo/fastapi

hug has some catching up to do, some of it just because it got in so early (It needs to be updated to be compatible with mypy and the types defined typing.py!) in any case using typing on API endpoints - independent of how you feel about Dynamic vs Static typing in general, just makes a lot of sense IMHO.

Re: Types for Python HTTP APIs

#29

Post by kigdihshbjdf »

dna_polymerase wrote: post #4

The whole Type thing in Python and all the workarounds to make Python type-safish looks and feels absolutely pathetic. I have no idea why a company like Facebook takes so much time to apologize for the inadequacy of Python as a programming language in the broader sense. Especially since they are utilizing a service based architecture they easily could make the switch to a language that actually supports types. Also t…

Like Islam and Judaism, Python and Go are immune to criticism on hacker news. You will be shadow banned or IP range banned soon.
Post reply on HN