<videovortex> news.com: How Pakistan knocked YouTube offline

[Geert Lovink]

How Pakistan knocked YouTube offline (and how to make sure it never 
happens again)
Posted by Declan McCullagh  | 17 comments 

http://www.news.com/8301-10784_3-9878655-7.html?tag=nl.e498

The graph that network-monitoring firm Keynote Systems provided to us 
shows the worldwide availability of YouTube.com dropping dramatically 
from 100 percent to 0 percent for over an hour. It didn't recover 
completely until two hours had elapsed. (Credit: Keynote Systems)

A high-profile incident this weekend in which Pakistan's state-owned 
telecommunications company managed to cut YouTube off the global Web 
highlights a long-standing security weakness in the way the Internet is 
managed.

After receiving a censorship order from the telecommunications ministry 
directing that YouTube.com be blocked, Pakistan Telecom went even 
further. By accident or design, the company broadcast instructions 
worldwide claiming to be the legitimate destination for anyone trying 
to reach YouTube's range of Internet addresses.

The security weakness lies in why those false instructions, which took 
YouTube offline for two hours on Sunday, were believed by routers 
around the globe. That's because Hong Kong-based PCCW, which provides 
the Internet link to Pakistan Telecom, did not stop the misleading 
broadcast--which is what most large providers in the United States and 
Europe do.

This is not a new problem. A network provider in Turkey once pretended 
to be the entire Internet, snarling traffic and making many Web sites 
unreachable. Con Edison accidentally hijacked the Internet addresses 
for Panix customers including Martha Stuart Living Omnimedia and the 
New York Daily News. Problems with errant broadcasts go back as far as 
1997.

It's also not an infrequent problem. An automatically-updated list of 
suspicious broadcasts created by Josh Karlin of the University of New 
Mexico shows apparent mischief--in the form of dubious claims to be the 
true destination for certain Internet addresses--taking place on an 
hourly basis.

So why hasn't anyone done something about it? False broadcasts can 
amount to a denial-of-service attack and, if done with malicious 
intent, can send unsuspecting users to a fake bank, merchant, or credit 
card site.

To understand why this is both a serious Internet vulnerability and 
also difficult to fix requires delving into the technical details a 
little.

How to pretend to be YouTube.com
  When you type a domain like "news.com" into your Web browser, it uses 
the Domain Name System to cough up a numeric Internet address, which in 
our case is 216.239.113.101. That IP address is handed to your router, 
which uses a table of addresses to figure out the next hop toward the 
news.com server.

Network providers--called autonomous systems, or ASs--broadcast the 
ranges of IP addresses to which they'll provide access. One of the 
functions of the Internet Corporation for Assigned Names and Numbers is 
managing the master list of AS numbers, which it does by allocating 
large blocks of 1,000 or so at a time to regional address registries.

Kim Davies, ICANN's manager of route zone services, says ICANN isn't 
able to revoke the AS number of a misbehaving network provider. "It's 
best to think of them as similar to post codes or ZIP codes," Davies 
said. "We maintain a registry of them to ensure that they aren't 
conflicting."

If the address information provided by AS is reliable, all is well. But 
if an AS makes a false broadcast, because of a configuration mistake or 
for malicious reasons, all hell can break loose.

This is what happened with YouTube, which Pakistan's government ordered 
blocked because of offensive material, apparently a video depicting the 
cartoons about Muhammad that had been posted in a Danish newspaper. 
Some reports have said the video featured several minutes of a film 
made by Dutch politician Geert Wilders, an outspoken critic of Islam.

A spokesman for the Pakistani embassy said on Monday that the order to 
block access to YouTube came from the highest levels of the government. 
It would have been passed along to Pakistan's Electronic Media 
Regulatory Authority and then to Pakistan's telecom authority, the 
spokesman said, which in turn would have issued the formal order to the 
Internet providers.

Pakistan Telecom responded by broadcasting the false claim that it was 
the correct route for 256 addresses in YouTube's 208.65.153.0 network 
space. Because that was a more specific destination than the true 
broadcast from YouTube saying it was home to 1,024 computers, within a 
few minutes traffic started flowing to the wrong place.

A timeline created by Renesys, which provides real-time monitoring 
services, says that it took about 15 seconds for large Pacific-rim 
providers to direct YouTube.com traffic to the Pakistan ISP, and about 
45 seconds for the central routers on much of the rest of the Internet 
to follow suit.

YouTube took countermeasures within minutes, first trying to reclaim 
its network by narrowing its 1,024 broadcast to 256 addresses. Eleven 
minutes later, YouTube added an even more specific additional broadcast 
claiming just 64 addresses--which, under the Border Gateway Protocol, 
is more specific and therefore should overrule the Pakistani one. Over 
two hours after the initial false broadcast, Pakistan Telecom finally 
stopped.

How could this have been prevented? First, Pakistan Telecom shouldn't 
have broadcast to the entire world that it was hosting YouTube's IP 
addresses. Second, Hong Kong-based PCCW could have recognized the 
broadcast as false and filtered it out.

An employee of PCCW, who wished to remain anonymous because he is not 
authorized to speak for the company, said that as soon as the false 
broadcast occurred, PCCW started receiving a flurry of phone calls from 
global ISPs wondering what had gone wrong. A YouTube representative 
also called.

Even Pakistan Telecom contacted PCCW. "I don't think they understood 
what was going on," the employee said. A spokesman for PCCW's U.S. 
operations, based in Herndon, Va., declined to provide details.

At the moment, large network providers tend to trust that other network 
providers are behaving reasonably--and aren't intentionally trying to 
hijack someone else's Internet addresses. And errors that do arise tend 
to be fixed quickly by manual intervention.

But as the number of suspicious broadcasts grows, and the potential for 
fraud increases, so does the justification for more aggressive 
countermeasures. (Besides, some government will eventually order its 
network providers to broadcast false information about the Internet 
addresses of "offensive" Web sites. We've already seen domain name 
blocking in Finland and Web page blocking in the United States, both 
supposedly enlightened Western democracies.)

One way to handle this is for network providers to be automatically 
notified when the virtual location of an Internet address changes, 
which is what some researchers have suggested in the form of a "hijack 
alert system." Another is to treat broadcasts with changes of addresses 
as suspicious for 24 hours and then accept them as normal. Simple 
filtering of broadcasts may not always work because some networks 
provide connectivity to customers with thousands of different routes.

Probably the most extensive countermeasure would be a technology like 
Secure BGP, which uses encryption to verify which network providers own 
Internet addresses and are authorized to broadcast changes. But Secure 
BGP has been around in one form or another form since 1998, and is 
still not a widely-used standard, mostly because it adds complexity and 
routers that understand will add additional cost.

At least that's been the conventional view. A high-profile incident 
like YouTube being knocked offline may accelerate this process, said 
Steven Bellovin of Columbia University. "I know there are serious 
deployment and operational issues," Bellovin said. "The question is 
this: When is the pain from routing incidents great enough that we're 
forced to act? It would have been nice to have done something before 
this, since now all the world's script kiddies have seen what can be 
done."
News.com's Greg Sandoval contributed to this report.