Ever wondered how many open FTP servers there are? • Hacker News Board

Re: Ever wondered how many open FTP servers there are?

#21

Post by toast0 » Sun, Sep 18, 2016, 4:24 AM UTC

ryanmccullagh wrote: post #7

Scanning the IPv4 space. I know there are many different projects that do it. I was thinking about how I would do this today. I believe the first step would be to arrive at all the IPv4 blocks (/22, etc), then do a calculation to arrive at the address of each based on the prefix. Then in an array of threads or so, try to connect(2) to the address on some type of service in a timeout handler. If it succeeds, then cons…

If you want to have your scan done in a reasonable amount of time, you probably want to use raw packets, and not connect.

If you don't mind all the flak you'll get, just send a SYN on the port you care about to each IP (maybe skip rfc1918, multicast and reserved addresses; or only send to addresses included in bgp announcements). If you send one packet to each addresses, including the addresses you should probably skip that's 4 Billion packets; if you do it at 1M pps (should fit on a 1Gbps ethernet connection), that's less than two hours.

Re: Ever wondered how many open FTP servers there are?

#22

Post by api » Sun, Sep 18, 2016, 4:24 AM UTC

djsumdog wrote: post #4

I remember back in the 90s, people would hunt for FTP servers that allowed anonymous writes. Companies that didn't know how to secure servers would suddenly be hosting a ton-o-warez.

That still happens.

Re: Ever wondered how many open FTP servers there are?

#23

Post by minxomat » Sun, Sep 18, 2016, 4:30 AM UTC

achillean wrote: post #20

heydonovan wrote: post #15

Are there any legal implications for doing this? Was going to do something similar with Redis and MongoDB.

This might be a useful starting point: https://blog.shodan.io/its-still-the-data-stupid/ Shodan crawls for most nosql/ queueing software including mongodb and redistribution. And related to OP: we also crawl for FTP and attempt anonymous as well as a few other things to better understand FTP deployments on the Internet.

Related to your related: Nice. Is this research public or commercial?

Re: Ever wondered how many open FTP servers there are?

#24

Post by jftuga » Sun, Sep 18, 2016, 4:30 AM UTC

Side note: "xz -9 -e" compresses the file to 3,296,864 where as "lz5 -15" only compresses the original file to 4,643,261 bytes. The xz compressed file is 29% smaller.

Even "gzip -9" compresses the file to 4,035,858.

So I wonder why lz5 was chosen for compression.

Re: Ever wondered how many open FTP servers there are?

#25

Post by minxomat » Sun, Sep 18, 2016, 4:34 AM UTC

toast0 wrote: post #21

ryanmccullagh wrote: post #7

Scanning the IPv4 space. I know there are many different projects that do it. I was thinking about how I would do this today. I believe the first step would be to arrive at all the IPv4 blocks (/22, etc), then do a calculation to arrive at the address of each based on the prefix. Then in an array of threads or so, try to connect(2) to the address on some type of service in a timeout handler. If it succeeds, then cons…

If you want to have your scan done in a reasonable amount of time, you probably want to use raw packets, and not connect. If you don't mind all the flak you'll get, just send a SYN on the port you care about to each IP (maybe skip rfc1918, multicast and reserved addresses; or only send to addresses included in bgp announcements). If you send one packet to each addresses, including the addresses you should probably sk…

That's what masscan automates. To fully utilize ms, one requires a friendly ISP. And even then, they can only be ms-friendly as long as their peers are. If you annoy the peers enough, they'll just drop you (http://www.sudosecure.com/ecatels-harboring-of-spambots-and-...). Many datacenters classify port-scanning as an offensive action, even with low package thoughput.

Pulling data from research servers (such as Censys), reducing and then scanning is always a good idea.

Re: Ever wondered how many open FTP servers there are?

#26

Post by specialp » Sun, Sep 18, 2016, 4:47 AM UTC

djsumdog wrote: post #4

I remember back in the 90s, people would hunt for FTP servers that allowed anonymous writes. Companies that didn't know how to secure servers would suddenly be hosting a ton-o-warez.

And then you could use FXP to transfer to another "pub" Usually most open FTPs had their /pub public writable and thus they were called "pubs"

Re: Ever wondered how many open FTP servers there are?

#27

Post by sillysaurus3 » Sun, Sep 18, 2016, 4:50 AM UTC

  $ lz5
  -bash: lz5: command not found
  $ brew install lz5
  ==> Auto-updated Homebrew!
  Updated Homebrew from b5a6b4e to 7926114.
  Error: No available formula with the name "lz5" 
  ==> Searching for similarly named formulae...
  Error: No similarly named formulae found.
  ==> Searching taps...
  Error: No formulae found in taps.

So, rather than complain about this, how do I do something to fix it? Can I add lz5 as a tap to Homebrew?

EDIT: Let me try this again, in a more productive way.

lz5 does not currently exist in Homebrew. I'd like to fix this. I've never done this before. Does anyone have advice? Is it as simple as forking lz5, then adding a tap to Homebrew?

Thanks. And as a note, this edit occurred after specialp's replies. They were right: this originally wasn't a productive comment.

Re: Ever wondered how many open FTP servers there are?

#28

Post by specialp » Sun, Sep 18, 2016, 4:52 AM UTC

sillysaurus3 wrote: post #27

$ lz5 -bash: lz5: command not found $ brew install lz5 ==> Auto-updated Homebrew! Updated Homebrew from b5a6b4e to 7926114. Error: No available formula with the name "lz5" ==> Searching for similarly named formulae... Error: No similarly named formulae found. ==> Searching taps... Error: No formulae found in taps. So, rather than complain about this, how do I do something to fix it? Can I add lz5 as a tap to Homebrew…

Yes it doesn't exist in a binary package for OSX so thus it does not exist.

Re: Ever wondered how many open FTP servers there are?

#29

Post by sillysaurus3 » Sun, Sep 18, 2016, 4:54 AM UTC

specialp wrote: post #28

sillysaurus3 wrote: post #27

$ lz5 -bash: lz5: command not found $ brew install lz5 ==> Auto-updated Homebrew! Updated Homebrew from b5a6b4e to 7926114. Error: No available formula with the name "lz5" ==> Searching for similarly named formulae... Error: No similarly named formulae found. ==> Searching taps... Error: No formulae found in taps. So, rather than complain about this, how do I do something to fix it? Can I add lz5 as a tap to Homebrew…

Yes it doesn't exist in a binary package for OSX so thus it does not exist.

Again: Rather than complain, what can I do to make this a little easier for OS X people?

Re: Ever wondered how many open FTP servers there are?

#30

Post by leeoniya » Sun, Sep 18, 2016, 4:54 AM UTC

djsumdog wrote: post #4

I remember back in the 90s, people would hunt for FTP servers that allowed anonymous writes. Companies that didn't know how to secure servers would suddenly be hosting a ton-o-warez.

> would suddenly be hosting a ton-o-warez.

which could then be found by searching for "index of"...