Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem: Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem William B. Norton Co-Founder andamp; Chief Technical Liaison Equinix, Inc. Clean Slate Networking Research Workshop Wednesday, March 21, 2007 13:30-14:00 Frances C. Arrillaga Alumni Center v0.3


Slide2: Merit @ UMich – NSFNET Project Logistics NSF funded $ 1987-1994 Regional Techs Meetings Host Agenda Then port 80 grew, NSF stopped funding… Open sharing of Ops Data


Slide3: Merit @ UMich – NSFNET Project Logistics 1987-1994 Regional Techs Meetings Host Agenda 1994 North American Network Operators Group (NANOG) Regional Techs Meeting Slow transition Little sharing Weak SLAsandamp;NDAs 1998-andgt;EQIX


Equinix: Equinix Carrier Neutral Internet Exchange Bay Area, LA, CHI, DAL, DC, NYMetro Singapore, Hong Kong, Tokyo, Sydney IPO in 2001, Market Cap $2.5B today 1200 Customers, 200 Networks Tier 1 ISPs, Cable Cos, Content, etc. Efficient interconnection of services Catbird seat 90% documenting Internet Ops Practices


Internet Operations White Papers: Internet Operations White Papers 'Interconnection Strategies for ISPs' 'Internet Service Providers and Peering' 'A Business Case for Peering' 'The Art of Peering: The Peering Playbook' 'The Peering Simulation Game' 'Do ATM-based Internet Exchanges Make Sense Anymore?' 'Evolution of the U.S. Peering Ecosystem' 'The Asia Pacific Internet Peering Guidebook' 'The Great (Public vs. Private) Debate' 'The Folly of Peering Traffic Ratios?' 'Video Internet: The Next Wave….' Internet makes anyone a publisher, similar effect now emerging for video On the Internet, Everyone if a Publisher


Massive Disruption in U.S. Peering Ecosystem  Short Videos: Massive Disruption in U.S. Peering Ecosystem  Short Videos YouTube – founded 2005 Short video clips – 50 million view per day! 20Gbps of peering traffic Feb 2006 $1M/month in Sept 2006! Entering Peering Ecosystem 30 Other competitors600Gbps peerable? DoveTail Video may dwarf current peered traffic 2010 – 80-90% Internet is Video Inculcate video guys into peering ecosystem Now, On the Internet Everyone is a Broadcaster Short video clips…Full TV shows… Source: http://www.nanog.org/mtg-0606/norton.html Source: http://digg.com/tech_news/YouTube_Gets_Bandwidth_Boost_from_Level_3


Slide7:


Massive Disruption in U.S. Peering Ecosystem  Full Episodes: Massive Disruption in U.S. Peering Ecosystem  Full Episodes 'Desperate Housewives' – 210MB/hour For 320x240 H.264 Video iTunes image 10,000,000 households 2,100,000,000 MB = 2.1 peta-Bytes How long will that take to download? Source: http://www.pbs.org/cringely/pulpit/pulpit20060302.html 3 days @ 64Gbps non-stop ! Just one show Try 250M*180 Channels*HDTV The Point: Massive Wave of Incremental Traffic to document…..


The Research Questions: The Research Questions How to distribute video across the Internet ? How much does it cost per video?


Modeling: Modeling Transit =Metered pipe to the Internet CDN =Content Distribution Network Peering =free andamp; reciprocal access to each others Customers P2P =PeerToPeer Shift from Avg to more typical demand curve... Varying Sized Loads Small =Distribute 10 videos every 5 minutes on avg. Medium =Distribute 100 videos every 5 minutes on avg. Large =Distribute 1000 videos every 5 minutes on avg.


Demand Curve Modeling: Demand Curve Modeling α = 10 videos/5 min=400Mbps ω =1.6Gbps ρ = 2.64Gbps α = 100 videos/5 min=4,000Mbps ω =16Gbps ρ = 26.4Gbps α = 1000 videos/5 min=40,000Mbps ω =160Gbps ρ = 264Gbps ρ=6.6xα 95th % ω~ 4 x α α = average load ρ = peak load (6.6*α) ω =95th percentile (4*α)


Model 1C – Large Load Commodity Transit: Model 1C – Large Load Commodity Transit Server1 GigE Switch1 Upstream ISPs : Server24 8 * 10GE to upstreams each : : Router1 10G Server262 Server263 Server264 : : GigE Switch14 : Router2 Router2 Router4 10G


Model 2C: CDN Large Load: Model 2C: CDN Large Load


Model 3C: Transit/Peering Large Load: Model 3C: Transit/Peering Large Load


Model 4C: P2P Large Load: Model 4C: P2P Large Load


Summary: Summary Per Video Cost Of delivery


Observations: Observations Internet Transit Supply ▼Price ▲ Internet Transit Model  src/dst specific Bottlenecks IX Power, Router Capacity, Peer’s Capacity, Backbone Capacity, Last Mile bottleneck, 100G NIC? Do I need to upgrade $$$$ gear to support my competitor (peer)? Identify Players, Positions, Motivations, Behavior Geoff Huston: 'P2P has won. Telco/Cable co trying to keep its 1998 biz plan relevant.' Predictions from the Field


Slide18: Content Providers: 'We are sitting around this table in 2010 and we are commenting how remarkable the last few years have been, specifically that:' 1. Video streaming volume has grown 100 fold 2. Last mile wireless replaced local loop 3. Botnets (DDOS attacks) are still an issue 4. Non-mechanical (i.e. Flash) Drives replaced internal hard drives on laptops 5. 10% of all cell phones are now video phones 6. We have cell phones that we actually like 7. The U.S. is insignificant traffic wise relative to the rest of the world 8. Most popular question discussed around the table: 'How do we operate business in China?‘ 9. No online privacy. And the gov't watches everything 10. 18-25 demographic is best reached w/ads on the Internet 11. Next Gen 3D on-line Social Networks are so successful 12. No physical network interfaces are needed 13. We will big brother ourselves (video cams 'who scraped my car?') 14. So many special purpose Internet apps - in car google maps, livetraffic updates, etc. 15. So much of our personal information is on the net 16. Video IM emerged as a dominant app 17. P2P will emerge for non-pirated videos - DRM in place and embraced 18. Voice calls are free, bundled with other things Source: May 2006 Web Content Private Forum


Slide19: 'ISPs: We are sitting around this table in 2010 at NANOG and we are commenting how remarkable the last few years have been, specifically that:' 1. We have 10G network interface(s) on laptops (I assumed wired, but someone else might have been thinking wireless) 2. $5/mbps is the common/standard price of transit (other prediction was $30/mbps) 3. Internet traffic is now so heavily localized (as in 75% of telephone calls are across town type of thing but for the Internet) 4. Ad revenue will cover the cost/or subsidize significantly of DSL 5. 90% of Internet bits will be video traffic 6. VoIP traffic exceeds the PSTN traffic 7. Private networks predominantly migrate to overlays over the Internet Wireless Internet Service Providers (WISPs) are serious competitive threat to DSL and Cable Internet Sprint is bought by Time Warner 10. Cable companies form cabal andamp; hookup with Sprint or Level 3 11. Government passes Net Neutrality Law of some flavor 12. Earthlink successfully reinvents themselves as Wireless Metro player in Response to ATT and Verizon 13. 40% paid or subscription as opposed to Content Click Ads. Like Cable Company channel packages, folks will flock to subscriptions for Internet Content packages. 14. RIAA proposes surcharge on network access (like Canada tax on blank CDs) 15. NetFlix conversion to Internet delivery of movies to Tivo or PC,or open source set top box 16. ISPs will be in pain 17. Last mile (fiber, wireless, .) in metro will be funded by municipal bonds 18. Death of TV ads, Death of broadcast TV, Tivo andamp; Tivo like appliances all use the Internet with emergence of targeted ads based ondemographic profiles of viewer 19. Google in charge of 20% of ALL ads (TV, Radio, Billboards, .) 20. Ubiquitous wifi in every metro with wifi roaming agreements 21. Congestion issues drive selective customer acceptance of partial transit offerings 22. IPTV fully embraced by cable cos - VOD - no need for VDR and ala carte video services replace analog frequency 23. Near simultaneous release of movies to the theaters, DVDs for the home, PPV, and Internet download to meet needs of different demographics. Source: Discussions at NANOG 37 San Jose