Posted on 2008-09-04 16:00:00
Tags: Networking, Peering
There are several kind of internet providers. One of them are the edge providers, who have their own IP space, AS number, provide application hosting and network services for their customers and have one or two uplinks to the internet which get charged per gigabyte of data. And a port on the local internet exchange of course! The other kind are the big ones, who have multiple links with other big ISPs and sell transit to the little ones. Oh, and also provide application hosting and network services for their own customers, and sometimes they are even on the local internet exchange because of that!
Local internet exchanges are places where multiple local ISP come together (you pay for the speed and for the port on the internet exchange, not for the traffic) and agree to route traffic between destined for the other ISPs on the internet exchange directly via the internet exchange instead of via the uplinks to the big ISPs. To push this behaviour, the address ranges advertised to the local internet exchanges are often /24s while the address ranges advertised on the internet uplinks are often /21s and bigger.
So euhm... What is the issue? Nothing yet, but it is coming :-)
If you are an edge provider, what should you do? Take an uplink which is also on the local internet exchange, advertise your /24s to the local internet exchange and your big /21 to the uplink provider. Why? Because the uplink provider will advertise your /21 to the rest of the internet, while it will internally route it via the /24s to the local internet exchange. Free inbound traffic! And if your port on the local internet exchange is 100Mbps or 1000Mbps and the link towards your uplink provider is less than 100Mbps, you will have a nice extra speed increase with it too. Of course this is only for the inbound traffic, the outbound traffic to the internet still goes via the slow uplink, but downloading goes fast. And since you are an edge provider, most of the traffic will be inbound.
The thing you have to take care about is that you have to monitor, specially in the first months, that the bill of your uplink provider does reflect the real traffic going over their uplink. Some providers run their accounting systems based on all the IP traffic going through their edge routers and will bill you for the traffic even if it doesn't go over the physical wire. Check your terms and conditions to see what you can do about this Layer 8 behaviour.
So what can the big ISPs do about this? Design their network properly. Consider the three different services which are being provided: Users, Services, Transit. Users traffic and Services traffic can go via the local internet exchange, but Transit traffic shouldn't. The network should be designed to have three routing clouds (call them Autonomous Systems if you want), and the exchange of routes between the three clouds and the outside world should be regulated carefully to make sure that the Systems and Users clouds are only providing the big ISPs IP space to the transit cloud. That way the /21 is in the Transit cloud and the /24s are in the Users and Services clouds.
Is this possible? I think so. Even if the Service networks and User networks are scattered around the world, with IP-over-IP tunnels between them it will make it look like one contiguous network. Doing proper routing traffic exchange between the three routing clouds internally and between the individual routing clouds and the external networks and the network behaves the way it was meant to work.