The UK has an entire IPv4 /8 that it isn't using

I don't think you understand how the Internet works. Or business. There is no foreseeable point at which it makes sense to bill an "Internet" service that's IPv6-only. That's doable _after_ the transition, but expecting that to happen now is not a transition.

you are missing a big point. This isn't like real-estate. This is plumbing.

Go do a search on "routing table growth"

This is actually a much larger problem than IPv4 runout. I mean, it's a problem that is further out, but it is going to hit well within my expected lifetime, and it's going to be a dramatically more difficult problem to solve.

The thing is, every time you break up a larger block into a smaller block, every router on the internet needs an entry in their routing tables[1] Usually, in content-addressable memory; Expensive stuff.

The thing is, the size of the global routing table? it's growing faster than moore's law.[2] I mean, it's not a huge deal now; for the price of a new compact car, you can get a router that has enough content-addressable memory to handle full tables and two 10g uplinks with 48 1g downlinks. reasonable cost of entry, if you ask me, and if you use dram (good luck with line rate 10G. don't think about faster line-rate.) well, then the cost is trivial. The routing table is under 500,000 entries in IPv4, so it's a trivial amount of dram.

The problem is that if this continues? (and all indications are that it's going to /massively accelerate/ if IPv6 catches on. At a minimum, you're gonna see the same number of ipv6 routes as ipv4 routes... and because IPv6 addresses are larger, ipv6 routes are larger) those routers are going to get more and more expensive.

So yeah, really? we need a way of charging, not for each IP, but for each announced route. Of course, that comes down hard on the little guys, whereas a per-IP charge is much more progressive. (a /8 takes up as many routing table entries as a /24.)

Getting by with a reduced number of IPs would be way easier for me than getting by without announcing my own block. Without BGP announcing my own block, my ISP has my balls in a vice. If they go down? I go down. More importantly, I need to change IPs when I switch to a new ISP.

I know what this is like, because I started out on my ISPs IP addresses, like almost everyone does. My provider found out that it's hard for me to switch off those IPs, so now I'm paying them, in total, for all the services they provide me, about twice market rate.

(It's actually really interesting, watching the dynamics of this problem play out on PPML or the like. Small players, like me, want to keep it easy to announce smaller blocks. We are fucked if you can only announce large blocks. Large players want tight restrictions on what size of a block you can announce, because they have the address space and they would rather not pay for more super-expensive CAM every time a Luke Crawford gets it in his head to start an ISP.)

[1] This is not strictly true. "every router that matters" would be closer to the truth. If you don't have full tables from multiple upstreams, well, you will be effected by rather more partial outages than if you do.

[2]http://bgp.potaroo.net/

[3]http://bgp.potaroo.net/index-bgp.html

Hm. interesting. so, uh, for simplicity, we have 256^3 routes, right? each one is, uh, what,33 bits of data? I mean, you need 3 bytes for the network and then what, uh, 5 bits for the mask? then okay 4 bits for the dest so 33 bits per route, no? so (33*256^3)/8 is 69206016 bits, or what,66 megabytes? sweet jesus, you are right. I mean, you are off by an order of magnitude, but at this scale, who gives a shit about an order of magnitude.

Interesting. 'cause none of the commercial routers do this. which is fucking weird. With this optimization (only store the first 3 octets, as you aren't routing anything smaller, have a lookup table for dest. addresses.) you could take full tables in puny amounts of CAM that come on, say, l3 switches. I will ask around as to why this isn't done.

(of course, with IPv6, this does not come close to solving the problem. /32 is the 'standard' handout for ISPs, and usually the smallest prefix you accept is a /48. There are a lot of fucking /32s.)

Also, you'd have some complications, as /32s are commonly used to blackhole DDoS targets, but you only have a handful of those, so that would add complexity but it wouldn't kill the idea.

I bet I'm missing something- I mean, every two bit ISP would be really happy to give you ten grand for a 10G full-tables bgp router, and you can get l3 switches with enough cam for that and a few 10G ports for little more than half that.

Given your constraints, I wonder if what you actually want is one switch for each upstream, and a gigabit link from each of your physical Xen servers to each switch that goes to an upstream. If your switches have to route at all, they can each have a default route to the one respective upstream connected to that switch. Then, have a VM on each physical Xen server with full routing tables to control how the outbound packets from all the VMs on that physical host get routed. And hopefully your switches can deal with a /32 route for each VM or something, if they just have a route per VM plus a default route to the one ISP they're connected to.

There's also the variant where you have a /24 block (or something) which has one IP address which is your ISP's router, and then one IP address per physical Xen server, and if your ISP is willing to take individual /32 announcements to your servers (I think this is technically feasible, but may require more mental flexibility than some ISPs have; on the other hand, I think you're talking Cogent, and mentioned that Cogent is willing to peer directly with your customers, which may imply that you're dealing with a flexible ISP), you can have the ISP do the layer 3 routing to the right physical host for traffic going to each VM. (The idea is that the ISP would not propagate the /32s to their peers; you'd separately send them a broader announcement that they could pass along to their peers.)

One other issue in this scheme is that if a single gigabit link between one physical Xen host and the switch going to one ISP breaks, inbound traffic that happens to come to the ISP attached to the switch with the broken link ought to have some mechanism to find its way over to the other switch for the one physical Xen host that has the broken link. I think there are ways that OSPF and or iBGP with the non-default next-hop-self setting can be made to do this.