NNSquad - Network Neutrality Squad
[ NNSquad ] [IP] An unusual denial of service attack
I would add that similar "automatic updating" situations also have potentially important implications vis-a-vis bandwidth caps. --Lauren-- NNSquad Moderator ----- Forwarded message from David Farber <dave@farber.net> ----- Date: Mon, 4 May 2009 11:56:35 -0400 From: David Farber <dave@farber.net> Subject: [IP] An unusual denial of service attack Reply-To: dave@farber.net To: ip <ip@v2.listbox.com> Begin forwarded message: From: Brett Glass <brett@lariat.net> Date: May 4, 2009 11:07:50 AM EDT To: dave@farber.net, "Ip ip" <ip@v2.listbox.com> Subject: An unusual denial of service attack Dave, and everyone: This weekend, my ISP suffered an unusual sort of denial of service attack. Starting on Saturday morning, users were reporting that their Web browsing had slowed to a crawl, though other services were working properly. I investigated, and saw that our upstream connection to the Internet backbone was being saturated -- but not by any one customer. So, I looked at the statistics on our Web cache (an activity, by the way, which I'm sure that certain privacy advocates would find tantamount to "snooping," even though it was for the purpose of managing the network). After awhile, I was able to figure out what was wrong. We were facing a distributed denial of service attack from the world's largest "botnet:" Microsoft's "Windows Update." Virtually every Windows machine on our network -- and most of our customers's machines are running Windows XP or Windows Vista -- was individually downloading many large updates. (See http://www.computerworld.com/action/article.do?command=printArticleBasic&taxonomyName=Security&articleId=9131573&taxonomyId=17 for a list of some of the many security holes that were being patched.) Fixing holes in Windows is a good thing, but to command more than 90% of all of the computers around the globe to "phone home" at the same time is, obviously, not. It's doubly bad when the updates are explicitly marked as non-cacheable, making our Web cache of no use to stem the flood. What's worse -- at least for our small ISP -- is that the updates are distributed for Microsoft by a company called Akamai. Akamai, as many of you know, places caches at the hubs of many ISPs' networks -- but, alas, only those of larger ones. Our smaller ISP, which has never been able to convince Akamai to place a cache at our location despite many years of requests, therefore must use backbone bandwidth to service all of these redundant requests. When I checked -- and it was not at the peak -- the traffic was consuming about half of our main DS-3 line to the Internet, leaving only half of its capacity available to carry all other traffic (including VoIP and bandwidth-intensive streaming video). Our cache's CPU utilization was above 95%, slowing response times still further. I solved the problem by telling the cache to throttle traffic to and from Akamai's upstream caches, which were serving up the updates. Instantly, the load dropped off and normal service was restored. As Spider-Man creator Stan Lee once noted, "with great power comes great responsibility." Microsoft, by virtue of its control over Windows-based PCs, has the ability to shut down the entire Internet at will -- and must be careful not to do it, inadvertently, by turning 90% of the world's PCs into a "zombie army." Furthermore, content delivery networks such as Akamai, which distributes Microsoft's updates, must not be allowed to discriminate against smaller providers by making updates uncacheable (at least by a standards-conforming Web cache) and then denying smaller ISPs access to a cache that WILL cache them. (Google, too, is also placing caches at the hubs of larger ISPs, thus giving them an edge when it comes to delivering Google services and video.) Small and competitive ISPs already have a tough row to hoe when competing with the telcos and cable companies. If they are further disadvantaged by prejudicial business practices of content providers and content delivery networks, Internet service will -- devastatingly for consumers -- become a duopoly. --Brett Glass ------------------------------------------- Archives: https://www.listbox.com/member/archive/247/=now RSS Feed: https://www.listbox.com/member/archive/rss/247/ Powered by Listbox: http://www.listbox.com ----- End forwarded message -----