Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happens if you write a TCP stack in Python?

jvns.ca

Post reply on HN
Print view View original

71–80 of 125 posts

Re: What happens if you write a TCP stack in Python?

#71

Post by cbhl »

tptacek wrote: post #23

The idea that Python is so slow that it's confusing TCP sounds wrong to me. I think it's more likely that your packet capture scheme is slow. It looks like you're using scapy, which I assume is in turn using libpcap... which may be buffering (in high-performance network monitoring, the packet capture interface goes out of its way to buffer). Which is something you can turn off. About 13 years ago, I wrote my own prog…

The Google homepage is only about 20000 bytes... if we assume an maximum segment size of ~1400 bytes, then 14 or 15 packets is about right.

I wouldn't be surprised if Google is sending the packets all at once and ignoring the ACKs altogether.

Heck, there's even a 2010 paper from Google on the subject of sending a bunch of packets at the beginning of the connection: An Argument for Increasing TCP's Initial Congestion Window[0]

[0]http://research.google.com/pubs/pub36640.html

Re: What happens if you write a TCP stack in Python?

#73

Post by milkshakes »

danudey wrote: post #69
tptacek wrote: post #32

Earlier quoted context omitted.

This is a great project. Keep playing with it! You might find that the serverside of TCP is more useful to have in Python than the clientside (having a userland IP/TCP serverside allows you to create fake hosts out of thin air).

This sounds super handy for situations like honeypots. Take a /24, assign the hosts you actually have, and then spoof the rest to another server which fakes open SMTP, HTTPS, etc. connections, or which replies to every connection attempt (slowly!) with a successful (if slow!) open. There's an iptables module called tarpit[1], which takes advantage of some peculiarities of the TCP protocol to essentially prevent the r…

this is in fact exactly how honeyd works: http://www.honeyd.org/

Re: What happens if you write a TCP stack in Python?

#74

Post by latiera »

It's not Python that's slow, but scapy, which is dog slow. In fact, it is so slow that it should come with big WARNINGs that it's only really meant for interactive use. Do the dissection yourself or use something built for that purpose.

It's really surprising to me that lots of ppl are using scapy for things that require performance but then again if you look at the scapy website or the docs, it's not immediately apparent that their tool is not meant for this. Which I guess says a lot about the scapy developers rather than the scapy users.

tl;dr Scapy is a joke, performance-wise.

Re: What happens if you write a TCP stack in Python?

#75

Post by js2 »

All three volumes of TCP/IP Illustrated may be found on the Internet in pdf form, but they are well worth buying.

Tangent: One of my favorite interview questions is to ask how traceroute works. The question works best when the candidate doesn't actually know. Then we can start to discuss bits and pieces of how TCP/IP works, until they can puzzle out the answer.

Re: What happens if you write a TCP stack in Python?

#76

Post by peterwwillis »

michaelhoffman wrote: post #30
cookiemonster11 wrote: post #20

Earlier quoted context omitted.

It confuses me that people make such a big deal of their little 20 lines of code toy projects.

It's not a little 20 line of code toy project. It's an engaging and accessible writeup of some basic parts of TCP that happens to include some easy-to-understand code. It's pointless to people who understand how TCP works in depth, but the majority of programmers don't.

It's an engaging and accessible writeup, true. Unfortunately, there are also several glaringly incorrect/misleading points in the article. The fact that it's getting upvoted is just..... strange.

Re: What happens if you write a TCP stack in Python?

#77

Post by corysama »

blutoot wrote: post #60
tedchs wrote: post #56

Earlier quoted context omitted.

Software Defined Networking is about moving the control plane out of switching/routing devices and into general purpose servers, so that they can make better forwarding decisions such as improved convergence time. In the sense that I understand SDN, I don't think doing TCP in userspace is part of it.

But if it can be done easily in a high-level language, then the control plane can be integrated better with a lot of applications written in the same language, no? I was thinking if that can serve as an impetus for SDN adoption.

You should have a look over here :)

https://github.com/SnabbCo/snabbswitch/wiki

Re: What happens if you write a TCP stack in Python?

#78

Post by jacquesm »

Dylan16807 wrote: post #67
sly010 wrote: post #33

If an 8MHz microcontroller is fast enough to implement TCP, then Python should be fast enough too. Here is my two cents on the expirement: 1. You don't really have to ack every packet, you have to order them, drop duplicates and ack the last one. 2. Google ignores the TCP flow control algorithm and sends the first few hundred packets very quickly without waiting for acks. They do this to beat the latency of the flow…

Even 8MHz is massive overkill. It's almost fast enough to bit-bang ethernet.

I'd like to see you bit-bang a 10MHz signal with an 8 MHz clock.

You'd need something substantially faster (Nyquist and such).

Re: What happens if you write a TCP stack in Python?

#79

Post by digitalsushi »

tptacek wrote: post #23

The idea that Python is so slow that it's confusing TCP sounds wrong to me. I think it's more likely that your packet capture scheme is slow. It looks like you're using scapy, which I assume is in turn using libpcap... which may be buffering (in high-performance network monitoring, the packet capture interface goes out of its way to buffer). Which is something you can turn off. About 13 years ago, I wrote my own prog…

Our tcp stack is written in tcl. The window size is set to 1. It works fine.
Post reply on HN