CANARIE “Community Condo Fiber Networks”The Customer Empowered Networking Revolution: CANARIE “Community Condo Fiber Networks” The Customer Empowered Networking Revolution http://www.canarie.ca http://www.canet3.net Bill.St.Arnaud@canarie.ca Tel: +1.613.785.0426


CANARIE Inc: Mission: To facilitate the development of Canada’s communications infrastructure and stimulate next generation products, applications and services Canadian equivalent to Internet 2 and NGI private-sector led, not-for-profit consortium consortium formed 1993 federal funding of $300m (1993-99) total project costs estimated over $600 M currently over 140 members; 21 Board members CANARIE Inc


Canada & the Optical Age : Canada & the Optical Age World leader in optical networking - JDS Fitel, Nortel, Cambrian, Positron Fiber Systems, CISCO Canada, PMC Sierra, QNX, Tropic, Edgeflow, Accelight, Lumenon Over 75% of the world’s Internet traffic is carried on equipment made in Canada Nortel Optical Transport made in Montreal Newbridge ATM switches made in Ottawa JDS Fitel optical components made in Ottawa CISCO GSR12000 made in Ottawa, Toronto and Vancouver CA*net 3 - A network for basic research unparalleled anywhere in the world Canada could be poised to be a world leader in the “optical age” Silicon Valley was the capital of the “silicon age” Britain was the capital of the “industrial age” Canada has the critical industrial base for the optical age


CA*net 3 National Optical Internet: GigaPOP CA*net 3 National Optical Internet Vancouver Calgary Regina Winnipeg Ottawa Montreal Toronto Halifax St. John’s Fredericton Charlottetown ORAN BCnet Netera SRnet MRnet ONet RISQ ACORN Chicago STAR TAP CA*net 3 Primary Route Seattle New York CA*net 3 Diverse Route Deploying a 4 channel CWDM Gigabit Ethernet network – 400 km Deploying a 4 channel Gigabit Ethernet transparent optical DWDM– 1500 km Multiple Customer Owned Dark Fiber Networks connecting universities and schools 16 channel DWDM -8 wavelengths @OC-192 reserved for CANARIE -8 wavelengths for carrier and other customers Consortium Partners: Bell Nexxia Nortel Cisco JDS Uniphase Newbridge Condo Dark Fiber Networks connecting universities and schools Condo Fiber Network linking all universities and hospital


CA*net 3 & Community Networks: CA*net 3 & Community Networks E-research grids Researchers will want to use computing resources of schools and homes SETI@Home New grid projects in bio-informatics, pharmaceutical research, particle physics need access to millions of computers Democratization of research Next big discovery in cancer or particle physics could be made at your local high school


Customer Empowered Networks: Customer Empowered Networks School boards and municipalities throughout North America are building condominium dark fiber networks in partnership with next generation carrier Individual institutions – the customers – own and control their own strands of fiber Fiber are configured in point to point private networks; or Connect to local ISP or carrier hotel Private sector maintains the fiber Low cost LAN architectures and optics are used to light the fiber These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community. Customers will start with dark fiber but will eventually extend further outwards with customer control and ownership of wavelengths Extending the Internet model of autonomous peering networks to the telecom world


Examples of CENCustomer Empowered Networks: Examples of CEN Customer Empowered Networks Universities in Quebec are building their own 3500km “condominium” fiber network in partnership with 6 next gen carriers- $US 2million Will deploy and manage their own optics and long haul transmission gear Universities in Alberta are deploying their own 400 km 4xGbe dark fiber network - $US 200K Deploy and manage their own optics and long haul transmission gear Chicago is building a fiber networking linking all public sector institutions - $US 250m City of Montreal is second most fibered city in the world because of municipal owned open access conduit In Ottawa is deploying a 85km- 144 strand “condominium” network connecting 26 institutions – cost $1m US Peel County – Missassuaga & Brampton has built a 200km public sector fiber network - $US 5m Many other cities including Ashland OR, Halifax, Toronto are looking at similar initiatives


Market Drivers: Market Drivers First - low cost Up to 1000% reduction over current telecom prices. 6-12 month payback Second - LAN invades the WAN – no complex SONET or ATM required in network Network Restoral & Protection can be done by customer using a variety of techniques such as wireless backup, or relocating servers to a multi-homed site, etc Third - Enables new applications and services not possible with traditional telecom service providers Relocation of servers and extending LAN to central site Out sourcing LAN and web servers to a 3rd party because no performance impact IP telephony in the wide area (Spokane) HDTV video Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points Much easier to out source servers, e-commerce etc to a 3rd party at a carrier neutral collocation facility


What is condominium fiber?: What is condominium fiber? A number of organizations such as schools, hospitals, businesses and universities get together to fund and build a fiber network Carrier partners are also invited to be part of condominium project Several next generation carriers and fiber brokers are now arranging condominium fiber builds IMS, QuebecTel, Videotron, Cogeco, Dixon Cable, GT Telecom, etc etc Fiber is installed, owned and maintained by 3rd party professional fiber contractors – usually the same contractors used by the carriers for their fiber builds Each institution gets its own set of fibers, at cost, on a 20 year IRU (Indefeasible Right of Use) One time up front cost, plus annual maintenance and right of way cost approx 5% of the capital cost Institution lights up their own strands with whatever technology they want – Gigabit Ethernet, ATM, PBX, etc New long range laser will reach 120 km Ideal solution for point to point links for large fixed institutions Payback is usually less than 18 months


Slide10: Province wide network of condominium fiber to 420 communities in Alberta Guaranteed cost of bandwidth to all public sector institutions $500/mo for 10 Mbps, $700/mo for 100 Mbps Network a mix of fibre builds and existing supplier infrastructure (swap/buy/lease) Condominium approach: All suppliers can Buy (or swap) a share of the fibre (during build or after) Lease bandwidth at competitive rates GOA has perpetual right to use (IRU) Ownership will be held at arms length GOA/stakeholder rates are costs to run divided over users Because of fibre capacity, bandwidth can be made available to businesses at urban competitive rate Total cost $193m Bell Intrigna prime contractor Alberta SUPERnet


Slide11: Extended Area 372 communities GOA/stakeholder needs Proceeds from businesses (urban benchmarked rates) to GOA to further network Base Area 48 communities GOA/stakeholder needs Business proceeds to Bell (urban benchmarked rates) - $143 Million GOA - 100% GOA IRU - $50 Million GOA - 33%GOA IRU - $102 Million Bell - 67% Bell IRU Alberta SUPERnet IRUs


Slide12: Combination: Fibre build Use of Existing Infrastructure Communities with one or more school BUFFALO TRAILS SCHOOL DIVISION Alberta SUPERnet Example


Slide13: Current (Typical) Residences 56 Kbps dial Internet ($85/Month) No high speed Internet Businesses Some T1 Facilities ($2000/Month average - rates distance sensitive) Some high speed business service on special setup arrangement Future (Everywhere) Residences High speed DSL residential Internet at urban rates ($40/month) Businesses High speed business services available at competitive urban rates (eg $820/month - T1) Higher speeds at comparable rates RURAL COMMUNITIES Alberta SUPERnet Impact


Slide14: CivicNet - A City-Wide Condominium Fiber Project connecting up 1600 public sector institutions Oriented to Development of Backbone Infrastructure With Gateways to Tributary Systems More Fiber in More Places Faster Ubiquitous, Pervasive: 1,600 Locations E-Z High-Performance Low-Cost Internet Connectivity Foundation = Existing City Fiber Builds Chicago CivicNet


Slide15: Observatoire Mont-Mégantic Val d’Or/Rouyn Quebec University Condo Network


Slide16: St-Laurent/Vanier Lanaudière Maisonneuve Marie-Victorin Champlain Rosemont Sorel-Tracy Montmorency Édouard-Montpetit Vieux-Montréal Bois-de-Boulogne Ahuntsic Lionel-Groulx Vers Québec Gérald-Godin John-Abbott André-Laurendeau Dawson Montreal Public Sector Condominium Networks


Slide17: List of Schoolboard Fiber Builds


South Dundas : South Dundas IROQUOIS MORRISBURG


South Dundas Results: South Dundas Results Morrisburg , Iroquios Have Fibre Hung Electronics In and Fibre Lit ISP’s , ASP’s all Want In he Fibre Major Employers Inquiring Very Positive Attitude in Community Digital Desert to Digital Oasis


Peel County Municipal Fiber Network: Peel County Municipal Fiber Network Mississauga, Brampton, Pell 200 km of Fibre 96 strand backbone “Enough for small country” 12-60 strands elsewhere 12,000 strand-kilometers Laid end-to-end = Victoria to St. John’s …...and back again


Ottawa Fiber Condominium: Ottawa Fiber Condominium Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes 26 sites Point-to-point topology 144 fibre pairs Route diversity requirement for one member 85 km run $11k - $50K per site Total project cost $CDN 1.25 million Cost per strand less than $.50 per strand per meter 80% aerial Due to overwhelming response to first build – planning for second build under way


Ottawa Original Estimates: Ottawa Original Estimates Original Engineering Estimates Original estimates turned out to be 10% higher than RFP responses Estimated cost to connect 22 institutions with 6 fibers to each institution in a star configuration Total cost $615,000 or approximately $30,000 per institution “on average” Actual costs range from $5K to $60K depending on how far institution is from center of star in downtown Ottawa If condo fiber contractor were to double capacity of network (i.e.12 strands to each customer) cost of project would only increase by 10% Or doubling number of participants would increase cost by only 10% (plus cost of laterals for additional institutions) By doubling number of participants average cost would be less than $20,000 per institution Ultimately fiber costs could get as low as $1000 per institution if every building in the city was connected with fiber


Slide24: Newbridge CRC CISCO OCRI Nortel O-C School Board Algonquin O-C Catholic Carleton O Heart Civic Oconnor CO 55 Metcalfe Ottawa U Ottawa Carleton Region Conseil Des Ecoles NRC Telesat Ottawa General March Carling Baseline Greenbank Merivale Merivale Bronson Laurier Rideau St. Laurent Smythe Blair Rd 20 19 18b 18a 17 16 15 14 13 12 11a 11b 9b 10 9a 6 5a 5b 3 8 7a 4 2 1b 1c 1a 1d 7b Section 1a – 96 strands Section 1b – 12 strands Secion 1c – 12 strands Section 1d – 96 strands Section 1e – 12 strands Section 2 – 36 strands Section 3 – 12 starnds Section 4 – 24 strands Section 5a – 24 strands Section 5b – 12 strands Section 6 – 12 strands Section 7a- 12 strands Section 7b – 12 strands Section 8 – 12 strands Section 9a – 96 strands Section 9b – 72 strands Section 10 – 12 strands Section 11a – 12 strands Section 11b – 60 strands Section 12 – 12 strands Section 13 – 48 strands Section 14 – 12 strands Section 15 – 48 strands Section 16 – 12 strands Section 17 – 36 strands Section 18a – 36 strands Section 18b – 24 strands Section 19- 12 strands Section 20- 12 strands 1e Main Splice Box for Cross Connection Of Fibers Between Participating Institutions Splice Box Note: This a reference installation. Final Configuration will vary depending on number of participants and additional point to point fiber requirements.


Section Cost Detail: Section Cost Detail


Logical Layout of Topology: Logical Layout of Topology Newbridge CRC OCRI CISCO Nortel Carleton Ottawa U NRC Telsat In reference model each institution has been assigned 6 strands to terminate on, or about 55 Metcalfe St Example: Carleton U has 6 strands 2 would cross connect to NRC/ONet 2 strands would connect directly to OttawaU 2 strands would connect directly to CRC (At NRC Carleton could interconnect at layer 3 with other organizations


Typical Fiber Capital Costs: Typical Fiber Capital Costs Average total cost between $7 and $15 per meter as follows: Engineering and Design: $1 - $3 per meter for engineering, design, supervision, splicing Plus Installation: $7 to $10 per meter for install in existing conduit; or $3 to $6 per meter for install on existing poles Plus Premise termination: Average $5k each Plus cost of fiber: 15¢ per strand per meter for 36 strands or less 12¢ per strand per meter for 96 strands or less 10¢ per strand per meter 192 strands or less 5¢ per strand per meter over 192 strands


Condo Fiber Costs - Examples: Condo Fiber Costs - Examples Des affluents: Total cost $1,500,00 ($750,00 for schools) 70 schools 12 municipal buildings 204 km fiber $1,500,000 total cost average cost per building - $18,000 per building Mille-Isles: Total cost $2,100,000 ($1,500,000 for schools) 80 schools 18 municipal buildings 223km $21,428 per building Laval: Total cost $1,800,000 ($1,000,000 for schools) 111 schools 45 municipal buildings 165 km $11,500 per building Peel county: Total cost $5m – 100 buildings Cost per building $50,000


Typical Payback for school(Real example – des affluents – north of Montreal): Typical Payback for school (Real example – des affluents – north of Montreal) Over 3 years total expenditure of $1,440,000 for DSL service Total cost of dark fiber network for 75 schools $1,350,000 Additional condominium participants were brought in to lower cost to school board to $750,000 School board can now centralize routers and network servers at each school Estimated savings in travel and software upgrades $800,000 Payback typically 8 –16 months Independent Study by Group Secor available upon request


Slide30: Reduction in the number of servers


Big Cost Saving in VoIP for schools: Big Cost Saving in VoIP for schools Many schools are using dark fiber to enable VoIP telephones to each teacher’s desk Also free phones in hallways for kids to all kids in other scholl With dark fiber only cost is the one VoIP phone itself VoIP gateway to PSTN is located at school board office Most teachers have never had a telephone in their classroom Has a bigger impact than multimedia, tele-learning etc Schools are ripping out old copper telephone systems and leaving one copper telephone for emergency purpose For more details http://www.canarie.ca/press/publications/pdf/workshop99/schweikhardt.pdf


Condo fiber for Business: Condo fiber for Business Significant reduction in price for local loop costs No increase in local loop costs as bandwidth demands increase Ability to outsource LAN and web servers to distant location as LAN speeds and performance can be maintained over dark fiber Access to lower cost competitive service providers at carrier neutral hotels New entrants cannot afford high cost of building out their own fiber networks Even small businesses with less than 20 employees can realize significant savings and benefits Examples: Colgate-Palmolive build in Cincinnati Nortel, Cisco, Gov’t depts in Ottawa


Advantage of Condo Fiber: Advantage of Condo Fiber Central Office Central Office Today: Customer pays 2 telcos for SONET connections Carrier managed SONET ring Customer Owned Dark Fiber Long reach lasers SONET Mux and ADM ISP ISP ISP ISP Tomorrow: Multiple Customer owned dark fiber links to ISPs $50K one time Unlimited Bandwidth $50K one time Unlimited bandwidth Monthly cost Fixed Bandwidth


Condo Fiber versus SONET: Condo Fiber versus SONET Customer Owned Dark Fiber Low equipment costs – mostly fused fiber, rarely any active devices Low labour costs – only fiber maintenance Fiber technology does not change Costs can be amortized over 20 years Most costs are capital vs operational The only true future proof technology Reliability obtained through purchase of 2 separate dark fiber route or wireless link, etc Can take advantage of economies of scale with large fiber builds Simple extension of LAN network Unlimited bandwidth “Good enough” is adequate for single entity Carrier SONET network High equipment cost – SONET muxes, etc High labour costs for SONET muxes, CPE equipment etc Equipment changes every 5 years Costs must be amortized in 5 years SONET soon may be replaced by GbE, DPT Requires SONET ring to deliver 99.999 reliability Limited economies of scale with larger SONET OC-192 Customer can not “capitalize” carrier service CPE equipment required from LAN to WAN Common carrier needs “perfect” network


Condo fiber for office buildings: Condo fiber for office buildings Building risers increasingly becoming congested because so many new entrant carriers want access to building Building owners are now insisting they will install fiber in risers from basement to tenants Some building owners are extending fiber all the way to 2 or more collocation facilities Tenants get to lease 2 or more strands in the fiber bundle to the collocation facilities Tenants can then make independent deals to connect to the service provider of their choice Tenants can then out source their web, network servers to 3rd party


Condo fiber for cities: Condo fiber for cities In downtown core minimizes digging up streets If N carriers are trying to deploy service then number of times roads has to be torn up is N squared However with condominium fiber road only has to be torn up once Produces a competitive market place and level playing field New competitive carrier can meet customers at carrier neutral collocation facilities Eliminates market advantage of incumbents In suburban areas eliminates duopoly of cable and TV companies The first company to install fiber into suburban neighborhoods will likely have a natural monopoly In Stockholm home owners have a choice of 4 cable companies Makes cities a much more attractive place for new high tech businesses and service


2 Different Views: 2 Different Views Traditional Telco Network “Ring of Rings” 99.999% reliability only in the SONET Ideal for carriage of legacy traffic “Highest” common denominator technology and pricing Driver is bandwidth efficiency ISP A ISP B ISP C CO Customer Empowered Network “Warp and Woof” Separate dark fiber networks built by different competing organizations Customer responsible for building rings and establishing reliability Ideal for LAN and Internet traffic Driver is congestion avoidance SONET ring 3 separate dark fiber builds ACME Dark Fiber Inc Fiber R Us Inc Non Profit Fiber Inc ISP B ISP C


Facilities based competition in the residential neighborhood?: Facilities based competition in the residential neighborhood? Facilities based competition is alive and well in downtown core The biggest challenge for governments is manage and coordinate the digging up of streets Outside of downtown in big cities Usually only a monopoly telecom provider At best a duopoly How do we introduce facilities based competition into this market (or at least come as close as possible to true facilities based competition)? As well how can we assure scalable high speed Internet services to the home that eventually will support Gigabit speeds or higher?


Community Fiber Architecture: Community Fiber Architecture A community consortia would put together a plan to fiber up all public sector buildings in their community A community can be a province, a municipality, village, etc A fiber splice box that terminates the fiber at the street side nearby each public sector building such as school, hospital, library is called a “Node” Community should must insure that potential facilities exist near the for private sector equipment to connect up future home owners – colo facility Colo facility allows private sector to extend wireless, VDSL or HFC services to the neighbourhood around the school Public sector buildings will have dedicated fiber strands that connect to a “Supernode” which is a fiber splice box on the street beside outside of major public sector central facility such as school board office, city hall, university, etc Community should insure that facilities exist nearby the Supernode for the private sector to install equipment to service home owners and businesses – colo facility Additional fibers are made available from the Supernode to all Nodes such that competitive service providers can purchase fiber to the node at some future date


Possible architecture for large town: Possible architecture for large town School School board office School Telco Central Office Central Office For Wireless Company VDSL, HFC or Fiber Provisioned by service provider Condominium Fiber with separate strands owned by school and by service providers Carrier Owned Fiber Cable head end Average Fiber Penetration to 250-500 homes Colo Facility 802.11b


Marriage of wireless and fiber: Marriage of wireless and fiber Many companies building longer range and higher speed versions of 802.11b (11 Mbps) Devices are low power and can be attached to fiber cable to provide inexpensive high bandwidth service for approx 1 km No licensing requirements so can be installed easily and quickly Allows easy extension of school or university LAN into the community Student can access university LAN from just about anywhere in the community Ideal for low cost – high bandwidth Internet service to the community No complex traditional wireless systems to manage


Benefits to Industry: Benefits to Industry For cablecos and telcos it help them accelerate the deployment of high speed internet services into the community Currently deployment of DSL and cable modem deployment is hampered by high cost of deploying fiber into the neighbourhoods Cable companies need fiber to every 250 homes for cable modem service, but currently only have fiber on average to every 5000 homes Telephone companies need to get fiber to every 250 homes to support VDSL or FSAN technologies Wireless companies need to get fiber to every 250 homes for new high bandwidth wireless services and mobile Internet It will provide opportunities for small innovative service providers to offer service to public institutions as well as homes For e-commerce and web hosting companies it will generate new business in out sourcing and web hosting For Canadian optical manufacturing companies it will provide new opportunities for sales of optical technology and components


Historical Reference Points: There is a clear trend in all formerly monopoly services to move to unbundled competitive services Roads and highway systems vs railways: infrastructure was largely “public”, but the services (e.g. trucking) were private and competitive Electrical distribution systems: regulated monopolies (unbundling is on horizon) Gas distribution systems: regulated monopolies (unbundling is well underway) Legacy telecommunications systems: moving to unbundled fiber and facilities based competition Historical Reference Points


Carriers are not the only decision maker in the last mile: Carriers are not the only decision maker in the last mile Governments and consumers are becoming more active voice in determining the future of broadband to home Do not assume that carrier best technical solution is the only approach Open access is becoming a critical political issue Consumers want more than duopoly of cable and telco Facilities based competition the best Municipalities object to their streets being torn up Dig once – bury lots of fiber Residents object to street furniture and antennae


An important Role for Government: Governments promote the framework for GITH networks by funding schools, universities, libraries, hospitals and municipal buildings as first customers and early adopters of dark fiber and optical networks Private sector leverages that investment by government to promote high speed Internet access to schools and universities to extend the fiber to the home Electric utility companies, municipal governments, CLECs, SMEs, entrepreneurs, as well as traditional telcos and cablecos can participate as providers, provided they subscribe to the architecture of open access, facilities based competition through dark fiber (or wavelengths) Emphasize the development and use of technology that specifically addresses the new architecture and the last mile, which must therefore be open, cheap and Internet-only An important Role for Government


National Broadband Task Force: National Broadband Task Force Mandate:To map out a strategy and advise the Government on best approaches to make high-speed broadband Internet services available to businesses and residents in all Canadian communities by the year 2004. To ensure Canada’s competitiveness in a global economy To address the Digital Divide To create opportunities for all Canadians 35 members including carriers, educators, librarians, communities, equipment manufacturers, etc Chair – David Johnston


Conclusion: Conclusion Many governments have recognized the importance of access to low cost dark fiber as fundamental economic enabler It will be the 21st century equivalent to the roads and railways that were built in the 20th century