When the Internet hits 256K default routes, watch out
The Internet is nearing the magical number of 256K routes in forwarding tables of routers in the Default Free Zone. If that’s meaningless to you (it was previously to me), allow me to explain.
The Default Free Zone is the top level of Internet routing. There are millions, maybe even billions, of computers attached to the Internet, but the vast majority of them aren’t connected to the top level of the Internet; rather, they are aggregated into subnets by upstream providers. The Default Free Zone is where all of the upstream providers exchange traffic at the highest level. Depending on your perspective of the Internet, the Default Free Zone may have a different number of routes. The most noticeable reason for this is that some networks do not filter routes more specific than /24, but there are other reasons as well. Just note that the majority of top level routers still see fewer than 256K routes.
A fair number of Internet backbone routers (mostly older ones, and especially older ones made by Cisco) only support a forwarding table with a maximum size of 256K entries. Beyond that point, they either cannot add new entries or end up wiping older entries. This poses a huge problem, potentially leading to a cascading catastrophic failure of Internet routing. Even a single additional route above 256K could cause widespread failures if it caused an important route to get overwritten in some forwarding tables. And as the number of routes in the Default Free Zone exceeds 256K by more and more, just forget about it.
Nobody has really undertaken a comprehensive survey of the Internet to figure out how much older routing hardware is out there, but a quick Google search reveals that used 256K routers are still being sold, and presumably put into service. It will be interesting to watch the Internet over the coming months as the average number of routes in the Default Free Zone exceeds 256K. Forwarding tables may start spontaneously failing, and upstream providers that failed to anticipate the 256K crossover will be in a panic to replace all of their suddenly obsoleted hardware.
To give you an idea of how close we are to the 256K routes cross-over, AboveNet, a large upstream provider in the United States and Europe, currently sees 250,884 routes. Note that 256K is 2^18, or 262,144. So we’re pretty close. This next graph (click it to embiggify) gives a good idea of the growth of the number of routes in the DFZ over time from various perspectives. Note that it grew by 50,000 in the last year.
By contrast, check out the graphs by Team Cyrmu, a network which does not filter out /25 routes. Note that they have blown past 256K routes and are sitting at around 280K routes. Obviously they are running relatively modern routers, but for the many companies out there on the net who are not, things may get hairy in the next few months.
March 6th, 2008 at 12:43
While “embiggen” is a word, I draw the line at “embiggify”.
March 6th, 2008 at 17:44
Agreed. “Embiggify” is not cromulent.
March 6th, 2008 at 23:38
Aye, but the more important question is, are you worried about the 256K routes problem? You’ve all seized on that one little part of the post while ignoring the larger issues!
It’s like you’re trying to embiggify a molehill into a mountain.
March 7th, 2008 at 01:51
No, I’m not worried about it. What is worrying about it going to do for me? Or for the problem?
March 7th, 2008 at 10:47
Working as a software designer has taught me that the biggest problem with critical updates is convincing the customers and general public that they are indeed critical.
That said, I think that many networks have modern equipment capable of handling this problem. Many, but not all. We’ll have to wait and see what happens.
March 7th, 2008 at 11:54
William: Well there’s a chance it might cause widespread loss of Internet connectivity, so at least watch out for it.
Jens: Unfortunately it’s not as easy as getting people to install critical software updates. With those, you just have to download and install. To fix these older routers, they have to spend a lot of money acquiring new ones and then swap them out (that’s a lot of cable unplugging and replugging).
March 10th, 2008 at 04:20
Having to spend money makes the problem even worse than just having to download a software patch. After all, if customers cannot be bothered to install a “free” software update, why would they spend money on a hardware upgrade the purpose of which they they do not understand?
March 10th, 2008 at 09:02
Well they would definitely spend money on a hardware upgrade that they didn’t understand when their whole network was broken and that was the only way to fix things.
March 10th, 2008 at 09:38
Exactly, but they could prevent the problem from causing harm if they have the proper knowledge. The cost for replacing the hardware will be roughly the same, but the cost of having a broken network would be saved.
Still, I will be very surprised if no networks get problems of this kind. People tend to turn a deaf ear to knowledge when money is involved.
March 10th, 2008 at 16:39
Well, the difference between people “turning a deaf ear” and trying to save money by cutting down on frivolous expenses is a pretty fuzzy line. I mean, people are constantly trying to tell you that some part of your world is going to break and if you would only buy their product, it’ll all be okay.
It’s a hard call on which is which, you know?
Cyde: I’ll watch out for it, but I don’t think there’s anything I can do on my end even if it *does* happen. What would the effects look like? It seems as though little parts of the Internet would flicker out for a few minutes or hours at a time. I’m imagining one of those nighttime pictures of the States from space, with little bits getting darker every so often. But, y’know, with Internet instead of power.
Heh, ‘Net-powered lights. Sort of reminds me of the ansible from the Ender’s Game series.
May 4th, 2010 at 15:01
what is the internet route sizes today?