Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happens if you write a TCP stack in Python?

jvns.ca

Post reply on HN
Print view View original

81–90 of 125 posts

Re: What happens if you write a TCP stack in Python?

#81

Post by fivre »

kashif wrote: post #52

I have actually implemented a TCP stack in python, unfortunately I can't share it publicly just yet. The authors problems are because she is not using RAW_IP_SOCKETS. Making TCP packets using pythons struct module is a breeze. I can post specific examples in code if anyone is interested. Finally you can write a proper TCP stack in python, there is no reason not to. Your user-space interpreted stack won't be as fast a…

If you have code examples already, I'd be grateful--I ran into server that had Python but not nmap or nc and while I was already familiar enough with socket to do what I needed, I wasn't so familiar with struct and wasn't able to make much headway manually building TCP/UDP packets before having to move onto other work.

Re: What happens if you write a TCP stack in Python?

#82

Post by fivre »

js2 wrote: post #75

All three volumes of TCP/IP Illustrated may be found on the Internet in pdf form, but they are well worth buying. Tangent: One of my favorite interview questions is to ask how traceroute works. The question works best when the candidate doesn't actually know. Then we can start to discuss bits and pieces of how TCP/IP works, until they can puzzle out the answer.

Do you know if there's a good IPv6 equivalent? I have IPv6 Core Protocols Implementation but like the writing style TCP/IP Illustrated much more.

Re: What happens if you write a TCP stack in Python?

#83

Post by sohaib2y »

cbhl wrote: post #71
tptacek wrote: post #23

The idea that Python is so slow that it's confusing TCP sounds wrong to me. I think it's more likely that your packet capture scheme is slow. It looks like you're using scapy, which I assume is in turn using libpcap... which may be buffering (in high-performance network monitoring, the packet capture interface goes out of its way to buffer). Which is something you can turn off. About 13 years ago, I wrote my own prog…

The Google homepage is only about 20000 bytes... if we assume an maximum segment size of ~1400 bytes, then 14 or 15 packets is about right. I wouldn't be surprised if Google is sending the packets all at once and ignoring the ACKs altogether. Heck, there's even a 2010 paper from Google on the subject of sending a bunch of packets at the beginning of the connection: An Argument for Increasing TCP's Initial Congestion…

I just checked google page is of 149K.

Re: What happens if you write a TCP stack in Python?

#84

Post by blutoot »

blutoot wrote: post #57
kashif wrote: post #52

I have actually implemented a TCP stack in python, unfortunately I can't share it publicly just yet. The authors problems are because she is not using RAW_IP_SOCKETS. Making TCP packets using pythons struct module is a breeze. I can post specific examples in code if anyone is interested. Finally you can write a proper TCP stack in python, there is no reason not to. Your user-space interpreted stack won't be as fast a…

I believe the author is a "she". It says at the banner of the blog.

I think this downvoting thing is out of control. What was even remotely offensive about this correction?

Re: What happens if you write a TCP stack in Python?

#85

Post by phaker »

Dylan16807 wrote: post #67
sly010 wrote: post #33

If an 8MHz microcontroller is fast enough to implement TCP, then Python should be fast enough too. Here is my two cents on the expirement: 1. You don't really have to ack every packet, you have to order them, drop duplicates and ack the last one. 2. Google ignores the TCP flow control algorithm and sends the first few hundred packets very quickly without waiting for acks. They do this to beat the latency of the flow…

Even 8MHz is massive overkill. It's almost fast enough to bit-bang ethernet.

Some guy actually did it on a 20MHz avr: http://www.cesko.host.sk/IgorPlugUDP/IgorPlug-UDP%20%28AVR%2... (hope the url doesn't get borked).

Sadly 10BASE-T ethernet uses manchester coding, so you need 20MHz :( you could pull it off with a slower processor if you had a fifo that can reach 20Mbaud on the serial end, but the usart in avr micros runs at a fraction of the system clock.

Re: What happens if you write a TCP stack in Python?

#86

Post by berdario »

milesvp wrote: post #50
kyllo wrote: post #28

Earlier quoted context omitted.

So you're saying that late binding is the most significant reason why Python is slow? Is slowness just an inherent tradeoff in using a language that supports this powerful feature?

There's probably something to this statement. there was a great article I saw on HN about a year ago that was talking about this (wish I'd bookmarked it). Crux of it was he looked at idiomatic python, saw all the hash lookups that entailed and said if you wrote C like that, it'd be slow as hell too. he then proceeded to speed up an algorithm to near C speeds by removing structures relying on these lookups. Was quit a…

You're probably thinking of this:

https://speakerdeck.com/alex/why-python-ruby-and-javascript-...

Re: What happens if you write a TCP stack in Python?

#87

Post by kyllo »

berdario wrote: post #86
milesvp wrote: post #50

Earlier quoted context omitted.

There's probably something to this statement. there was a great article I saw on HN about a year ago that was talking about this (wish I'd bookmarked it). Crux of it was he looked at idiomatic python, saw all the hash lookups that entailed and said if you wrote C like that, it'd be slow as hell too. he then proceeded to speed up an algorithm to near C speeds by removing structures relying on these lookups. Was quit a…

You're probably thinking of this: https://speakerdeck.com/alex/why-python-ruby-and-javascript-...

This is also relevant, someone implemented early-binding virtual method tables in Python and benchmarked the performance speedup: http://legacy.python.org/workshops/1998-11/proceedings/paper...

Re: What happens if you write a TCP stack in Python?

#88

Post by TickleSteve »

jacquesm wrote: post #78
Dylan16807 wrote: post #67

Earlier quoted context omitted.

Even 8MHz is massive overkill. It's almost fast enough to bit-bang ethernet.

I'd like to see you bit-bang a 10MHz signal with an 8 MHz clock. You'd need something substantially faster (Nyquist and such).

TCP != ethernet. A TCP stack can run over serial and othe link layers (PPP, SLIP, etc). PPP is absolutely doable on very small microcontrollers.

Re: What happens if you write a TCP stack in Python?

#89

Post by wmf »

fivre wrote: post #82
js2 wrote: post #75

All three volumes of TCP/IP Illustrated may be found on the Internet in pdf form, but they are well worth buying. Tangent: One of my favorite interview questions is to ask how traceroute works. The question works best when the candidate doesn't actually know. Then we can start to discuss bits and pieces of how TCP/IP works, until they can puzzle out the answer.

Do you know if there's a good IPv6 equivalent? I have IPv6 Core Protocols Implementation but like the writing style TCP/IP Illustrated much more.

There is a 2nd edition.

Re: What happens if you write a TCP stack in Python?

#90

Post by wmf »

beering wrote: post #10

I wonder if it would have a better success rate on a site other than Google's, since I've heard that Google's done extensive tuning of their TCP stack to send page data faster. Somebody's oversubscribed $3/month shared PHP hosting might not ramp up the speed as quickly.

TCP is TCP. If you advertise a certain window size you better be prepared to receive that data.
Post reply on HN