And in Squeak (Smalltalk) ? take a look at: http://www.squeaksource.com/@Hl1Cdo4NwCmLqQl0/Im9cEg0J
I'm guessing you were linking to http://www.squeaksource.com/Net.html - Seaside uses non-persistent URLs by default, leading to problems like this...
What happens if you write a TCP stack in Python?
101–110 of 125 posts
Re: What happens if you write a TCP stack in Python?
#102Earlier quoted context omitted.
Your "userland TCP/IP" comment got me excited and I started digging around. I wonder if there's some benefit in trying to port something like TCP Daytona [1,2] in a high-level language like Python. I'm curious if that will somehow advance the adoption of SDNs. [1] https://github.com/jamesbw/tcp-daytona [2] http://nms.lcs.mit.edu/~kandula/data/daytona.pdf
Like writing your own emulator, writing a full TCP stack is a project that is intrinsically worth doing. You can probably also come up with real-world applications for it (as a security tester, there are lots of applications for having full control over a TPC stack, regardless of how performant it is), but just having done it offers a huge learning return on a modest investment. You probably don't fully grok what TCP…
> (as a security tester, there are lots of applications for having full control over a TPC stack, regardless of how performant it is)
And sometimes you even want your stack to be slow, eg in a slow loris attack.
Re: What happens if you write a TCP stack in Python?
#103The idea that Python is so slow that it's confusing TCP sounds wrong to me. I think it's more likely that your packet capture scheme is slow. It looks like you're using scapy, which I assume is in turn using libpcap... which may be buffering (in high-performance network monitoring, the packet capture interface goes out of its way to buffer). Which is something you can turn off. About 13 years ago, I wrote my own prog…
I was surprised by this as well. The common quote is that Python is 10 - 100 times slower than C, yet computers have been doing TCP for ages, and Moores law has meant that my phone is probably more powerful / faster than my laptop from ten years ago. It didn't quite add up.
Python may be slow, but it's only relatively slow. It's still fast enough for the vast majority of use cases.
Re: What happens if you write a TCP stack in Python?
#104Earlier quoted context omitted.
I was surprised by this as well. The common quote is that Python is 10 - 100 times slower than C, yet computers have been doing TCP for ages, and Moores law has meant that my phone is probably more powerful / faster than my laptop from ten years ago. It didn't quite add up.
Python may be slow, but it's only relatively slow. It's still fast enough for the vast majority of use cases.
As with most things in software, speed can't be measured only by the running program -- the time required to write the program should be included in nearly all cases.
Python: fast to write, slow to run.
C/C++: slow to write, fast to run.
To put it succinctly, you can write fast programs, and you can write programs fast, but you can't write fast programs fast.
Re: What happens if you write a TCP stack in Python?
#105Earlier quoted context omitted.
Python may be slow, but it's only relatively slow. It's still fast enough for the vast majority of use cases.
As with most things in software, speed can't be measured only by the running program -- the time required to write the program should be included in nearly all cases. Python: fast to write, slow to run. C/C++: slow to write, fast to run. To put it succinctly, you can write fast programs, and you can write programs fast, but you can't write fast programs fast.
Now I'm genuinely interested in knowing which parts of python are 'slowing down' the application. With the help of Cython[0], one can give hints to the compiler to improve performance for most of the cases.
Re: What happens if you write a TCP stack in Python?
#106I have actually implemented a TCP stack in python, unfortunately I can't share it publicly just yet. The authors problems are because she is not using RAW_IP_SOCKETS. Making TCP packets using pythons struct module is a breeze. I can post specific examples in code if anyone is interested. Finally you can write a proper TCP stack in python, there is no reason not to. Your user-space interpreted stack won't be as fast a…
If you have code examples already, I'd be grateful--I ran into server that had Python but not nmap or nc and while I was already familiar enough with socket to do what I needed, I wasn't so familiar with struct and wasn't able to make much headway manually building TCP/UDP packets before having to move onto other work.
drop me an email, the address is in my profile. I will send you some examples.
Re: What happens if you write a TCP stack in Python?
#107Earlier quoted context omitted.
Python may be slow, but it's only relatively slow. It's still fast enough for the vast majority of use cases.
As with most things in software, speed can't be measured only by the running program -- the time required to write the program should be included in nearly all cases. Python: fast to write, slow to run. C/C++: slow to write, fast to run. To put it succinctly, you can write fast programs, and you can write programs fast, but you can't write fast programs fast.
I entirely agree, for most situations the trade off is in favor of high-level languages like Python. The scale tips even further in favor of CPython when you start using C extensions like Numpy. All of the benefits of Python, with speed approaching that of pure C (there's some overhead when calling from Python). For use-cases like web applications, the language is almost never the bottleneck anyways. Network latency and database queries usually eat up far more time than the glue code holding it all together. And this isn't even touching on the security benefits of memory safe languages.
Re: What happens if you write a TCP stack in Python?
#108I'm the author of hexcap(http://www.hexcap.org), an ncurses libpcap file editor and packet generator. I've also written many Scapy applications like this one(https://github.com/smutt/mcastClients). I rewrote the DHCPv4 client in Scapy since the stock one is broken. Also as part of hexcap have made numerous fixes to dpkt. Needless to say, I've done a lot with Python and packets.
If you're interested in writing a TCP/IP stack in Python I would recommend you use Python raw sockets, or possibly dnet[1] or pcapy[2]. The Scapy level of abstraction is too high for your needs.
I agree with other posters who mention buffering in libpcap. Read the man page for pcap_dispatch to get a better idea of how buffering works in libpcap. Also try capturing packets with tcpdump with and without the '-l' switch. You'll see a big difference if your pkts/sec is low.
Don't do arp spoofing. If you're writing a TCP/IP stack then you need to also code up iparp. If you don't want to do that, then use raw sockets and query the kernel's arp cache.
On second thought you really need to use raw sockets if you want this to work. Using anything pcap based will still leave the kernel's TCP/IP stack involved, which is not what you want.
[1] http://libdnet.sourceforge.net/pydoc/private/dnet-module.htm... [2] http://corelabs.coresecurity.com/index.php?module=Wiki&actio...
Re: What happens if you write a TCP stack in Python?
#109There's no reason to ACK every packet. It's very common to deal with things selectively. SACK: http://packetlife.net/blog/2010/jun/17/tcp-selective-acknowl...
Re: What happens if you write a TCP stack in Python?
#110Earlier quoted context omitted.
Like writing your own emulator, writing a full TCP stack is a project that is intrinsically worth doing. You can probably also come up with real-world applications for it (as a security tester, there are lots of applications for having full control over a TPC stack, regardless of how performant it is), but just having done it offers a huge learning return on a modest investment. You probably don't fully grok what TCP…
> (as a security tester, there are lots of applications for having full control over a TPC stack, regardless of how performant it is) And sometimes you even want your stack to be slow, eg in a slow loris attack.
Wrong. In these cases you want your stack to be as fast as possible. But you're introducing artificial delay on selected keypoints. If your attack code is slow, then you're basically attacking your self. Target is to consume (a lot if possible) more resources on the target side, than you're using on your own side.