Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Future of Asynchronous IO in Python

medium.com

Post reply on HN
Print view View original

11–20 of 51 posts

Re: The Future of Asynchronous IO in Python

#11

Post by untothebreach »

rubiquity wrote: post #10
mamcx wrote: post #6

Ok, and if we can start clean? (I'm in the process of build a toy language). Is this more a problem for a language with baggage, or is general? I wonder if for example in Go, Erlang is less of a issue...

The entire premise of this article is that the author has adopted the microservices disease and is suffering from not having asynchronous IO. You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. Oh, and microservices clearly create other problems, such as messaging. 0 - I…

a lot of the articles I read about microservices seem to come at them more from an organization point-of-view than a concurrency one. That is, encapsulating the logic for one specific component of a system, both for ease of scaling up only the parts of the system that need to be scaled, as well as for a different way for dev teams to interact with other parts of the system.

I'm not saying I agree, but I do understand why some golang programmers might be evangelizing microservices as well

Re: The Future of Asynchronous IO in Python

#12

Post by dragonwriter »

rubiquity wrote: post #10
mamcx wrote: post #6

Ok, and if we can start clean? (I'm in the process of build a toy language). Is this more a problem for a language with baggage, or is general? I wonder if for example in Go, Erlang is less of a issue...

The entire premise of this article is that the author has adopted the microservices disease and is suffering from not having asynchronous IO. You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. Oh, and microservices clearly create other problems, such as messaging. 0 - I…

> You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0].

The architecture of an idiomatic Erlang-based system is essentially a microservice architecture.

> - I do hear Go-lang users talk about microservices, which is weird. Because they have decent concurrency primitives to where they shouldn't need to split web applications up into microservices.

Microservice architecture has motivation (loose coupling, distribution, independent scalability of components) that go considerably beyond "my language doesn't have decent concurrency primitives".

Re: The Future of Asynchronous IO in Python

#13

Post by rubiquity »

dragonwriter wrote: post #12
rubiquity wrote: post #10

Earlier quoted context omitted.

The entire premise of this article is that the author has adopted the microservices disease and is suffering from not having asynchronous IO. You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. Oh, and microservices clearly create other problems, such as messaging. 0 - I…

> You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. The architecture of an idiomatic Erlang-based system is essentially a microservice architecture. > - I do hear Go-lang users talk about microservices, which is weird. Because they have decent concurrency primitives to…

> The architecture of an idiomatic Erlang-based system is essentially a microservice architecture.

:)

> loose coupling, distribution, independent scalability of components

If you have good abstractions for concurrency then distribution and independent scalability should be trivial in a single codebase. Loose coupling is usually a false dream. Microservices tend to get coupled at the network level instead of the code level. Yuck!

Re: The Future of Asynchronous IO in Python

#14

Post by rubiquity »

untothebreach wrote: post #11
rubiquity wrote: post #10

Earlier quoted context omitted.

The entire premise of this article is that the author has adopted the microservices disease and is suffering from not having asynchronous IO. You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. Oh, and microservices clearly create other problems, such as messaging. 0 - I…

a lot of the articles I read about microservices seem to come at them more from an organization point-of-view than a concurrency one. That is, encapsulating the logic for one specific component of a system, both for ease of scaling up only the parts of the system that need to be scaled, as well as for a different way for dev teams to interact with other parts of the system. I'm not saying I agree, but I do understand…

Good point about the organizational point of view. I'm still torn on whether more smaller teams is better for organizations or worse. Smaller is usually better, but now along with coordinating between the applications at the network level you also have to coordinate between applications at the human level. Some organizations probably do a lot better at this than others.

Re: The Future of Asynchronous IO in Python

#15

Post by avita1 »

rubiquity wrote: post #10
mamcx wrote: post #6

Ok, and if we can start clean? (I'm in the process of build a toy language). Is this more a problem for a language with baggage, or is general? I wonder if for example in Go, Erlang is less of a issue...

The entire premise of this article is that the author has adopted the microservices disease and is suffering from not having asynchronous IO. You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. Oh, and microservices clearly create other problems, such as messaging. 0 - I…

Microservices are touted as being fault tolerant. Erlang provides fault tolerance as part of the OTP. While I haven't programmed much in Go, I haven't seem much in the way of fault tolerance.

Re: The Future of Asynchronous IO in Python

#16

Post by plainOldText »

rubiquity wrote: post #13
dragonwriter wrote: post #12

Earlier quoted context omitted.

> You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. The architecture of an idiomatic Erlang-based system is essentially a microservice architecture. > - I do hear Go-lang users talk about microservices, which is weird. Because they have decent concurrency primitives to…

> The architecture of an idiomatic Erlang-based system is essentially a microservice architecture. :) > loose coupling, distribution, independent scalability of components If you have good abstractions for concurrency then distribution and independent scalability should be trivial in a single codebase. Loose coupling is usually a false dream. Microservices tend to get coupled at the network level instead of the code…

A system which serves millions of users spans multiple machines. It's much easier to scale and evolve your stack if you're keeping it decoupled. Plus if you use something like nanomsg you can keep the services on the same host, and use something like ipc:// for minimal latency, then just switch the protocol and move the service to a new host with no other changes when it's time to scale up.

No matter how good your concurrency primitives, you cannot escape the network. Yes, I know, Erlang is awesome and has excellent distribution and concurrency capabilities out of the box, but you cannot write all the things in Erlang, nor should you want to. Different languages/services have to talk to each other somehow and at some point.

Re: The Future of Asynchronous IO in Python

#17

Post by Animats »

It's frustrating. Many people want an interprocess subroutine call, and few OSs have the right primitives for it. (QNX does, and Windows sort of does, but the Linux/Unix world does not.) There's an endless collection of kludges so program A can call program B implemented on top of pipe/socket like mechanisms.

OpenRPC and CORBA are out of fashion. Google protocol buffers help, but don't come with a standard RPC mechanism. HTTP with JSON is easy to do but has high overhead. Then there's the handling of failure, the curse of callback-oriented systems. ("He said he'd call back. Why didn't he call back? Doesn't he like me any more? Should I dump him?")

The lack of a good, standardized interprocess call mechanism results in interprocess call logic appearing at the application level. At that level, it's intertwined with business logic and often botched.

Re: The Future of Asynchronous IO in Python

#18

Post by rumcajz »

tailhook wrote: post #3
rumcajz wrote: post #2

This article raises many relevant points. But the real issue behind all of that is that we lack means to easily implement protocol stacks. Implementing a new protocol (especially in the user space) is a task that can easily eat months or years of your precious time.

Well, for many protocols it easy. I had written (a useful subset of) mysql protocol parser in a weekend. Sure, this is in Python, so reimplement it in C is much harder, still not years or so. Many protocols: mongodb, redis, memcached, beanstalk, etc, are much simpler. There are protocols that take months and years, but they are not so ubiquitous (with the obvious exception of HTTP, which is really complex and ubiquit…

Parsers are generally easy, I was thimking more about implementation stuff like: How can we pipe input from one protocol to another protocol on top of it? How to poll on a user-space protocol? How to integrate it with foreign event loops? If in user space, how to handle process termination if there's still data in tx queue? How to handle many different peers at once without having to create a separate thread for each of them? Et c.

Re: The Future of Asynchronous IO in Python

#19

Post by cbsmith »

Animats wrote: post #17

It's frustrating. Many people want an interprocess subroutine call, and few OSs have the right primitives for it. (QNX does, and Windows sort of does, but the Linux/Unix world does not.) There's an endless collection of kludges so program A can call program B implemented on top of pipe/socket like mechanisms. OpenRPC and CORBA are out of fashion. Google protocol buffers help, but don't come with a standard RPC mechan…

So often an RPC mechanism is not what you really want in the end. You tend to be better off with a MOM-style interface.

Re: The Future of Asynchronous IO in Python

#20

Post by woah »

rubiquity wrote: post #10
mamcx wrote: post #6

Ok, and if we can start clean? (I'm in the process of build a toy language). Is this more a problem for a language with baggage, or is general? I wonder if for example in Go, Erlang is less of a issue...

The entire premise of this article is that the author has adopted the microservices disease and is suffering from not having asynchronous IO. You'll never hear an Erlang programmer even talk about microservices because they are a solution to a problem that doesn't exist in Erlang, or any sufficiently concurrent language for that matter[0]. Oh, and microservices clearly create other problems, such as messaging. 0 - I…

Maybe I'm dumb, but doesn't splitting up a web application into micro services have many other benefits such as modularity, constraining complexity, code reuse etc.?
Post reply on HN