VSMM2001Zyda

Slide1: 

Michael Zyda, Director THE MOVES INSTITUTE zyda@movesinstitute.org The Future of Interactive Networked Entertainment

Outline: 

Outline Hardware - Graphics & Processors & Memory Broadband Network - DSL & Cable Modems A proposal for a laboratory to focus on the future of interactive networked entertainment A glimpse at where some of this work might be going on …

Hardware: 

Hardware

Hardware: 

Hardware Processors From 1 Ghz to 300 Ghz in the next 15 years We get this from Moore’s Law

Moore’s Law: 

Moore’s Law In 1965, Gordon Moore (co-founder of Intel) identified an empirical relationship between time and the number of transistors that could be placed on an integrated circuit. Moore’s Law states that the pace of microchip technological change is such that the number of transistors a microchip can hold doubles every 12 to 18 months.

Moore’s Law in Transistors: 

Moore’s Law in Transistors

Moore’s Law in MIPS: 

Moore’s Law in MIPS

Lots of processor cycles: 

Lots of processor cycles What will we do with all of these cycles? Well, this is a fairly common question in computer science when it appears we are entering an era of apparently limitless resources. We saw this with transistors in the early days of VLSI design - we had smart people question whether we could ever really use all those millions of transistors. The answer is, we always consume all available computing resources. And over the next 15 years, we will have many cycles to spend. So maybe we will compute things we haven’t been able to before …

Hardware - Memory: 

Hardware - Memory CPU memory and non-volatile disk storage are about to become nearly free. Disks Today - 30GB common (this laptop). Next two to three years - IBM technology - 400 GB 2.5 inch disks (200 GB for laptops will be able to store 42 DVD movies). 100B bits/square inch. We may no longer delete files or objects! But we will need to have some strategy for backups …

New memory technologies: 

New memory technologies Recent patent - solid state memory system with a capacity of 86GB per square centimeter of surface area. Technology - magneto-optical system similar to the CD-ROM and can be used as computer & processor memory for credit & smart cards. 3.4TB of memory within the surface area of a credit card. 3cm x 3cm x 1.5cm for $43.60/chip, available within 2 years. Source cnn.com 15 February 2001

How big is this?: 

How big is this? 3.4TB Memory Chip (3.4 x ) What if we put a small video camera and microphone on our shoulder & recorded full rate video 24/7? 60k bps per frame, at 30 fps (Media 100 natural video rate). 225k bytes per second. 1.51 x seconds at 8.64 x sec/day 1.748 x days --> 175 days! The real question is how do we index and use such a stream …

How big is this?: 

How big is this? 10TB --> the printed collection of the US Library of Congress 2TB --> the contents of an academic research library. 1TB --> the daily rate of all EOS (earth observatory satellite) data. Source http://www.cacr.caltech.edu/~roy/dataquan/ How many DVDs is this? 425 DVDs at 8GB/DVD movie.

Hardware - Graphics: 

Hardware - Graphics Graphics From 50M polygons per second to 5B polygons per second in three years? You can’t buy a motherboard from Intel without 3D from nVidia. And that is all the way down to $900 commodity computers. If you ask nVidia in private, they say they are heading towards 5B polygons per second …

nVidia GeForce3: 

nVidia GeForce3 GeForce3 specification (NV20) 57 million transistors 50 M polygons/second 0.15 technology 200 MHz graphics core clock speed 4 rendering pixel pipelines 2 texture blocks per rendering pipeline 4 textures on one pixel (two clocks are needed) 128 bit memory interface DDR SDRAM/SGRAM support When released the NV20 based cards will be equipped with 3.8 ns memory working at 230 (460) MHz over 7 Gbps peak memory bandwidth with 230 MHz DDR bus Up to 128 MBytes local memory (the majority of the first cards will have 64 MB) RAMDAC: 350 MHz Max resolution: 2048x1536@75 Hz

nVidia nForce invades Intel’s space?: 

nVidia nForce invades Intel’s space? Comparison 57M transistors in GeForce3. 42M transistors in Intel Pentium 4. We would expect to see nVidia start designing their own CPUs soon since they have access to a FAB that can make high transistor count parts.

Hardware - Graphics: 

Hardware - Graphics Computer graphics hardware developments are being driven by the demands of the entertainment industry, most specifically the 3D video game and console market. We are about to see a revolution on the future form factor of our computers (game console or desktop or laptop?)

Trends - Game Machine Platforms: 

Trends - Game Machine Platforms Playstation 2 --> Rasterize 75M polygons/second and transform 66M polygons/second. Playstation 3 --> 1,000 times faster than that in five years? 5B polygons/second? Faster?

Playstation-2: 

Playstation-2 Sony PlayStation 2 (October 2000): Processor: 128-bit "Emotion Engine" 300 MHz Floating point unit (FPU) co-processor Maximum bus transfer rate of 3.2 GB per second Includes current PlayStation CPU core Graphics: "Graphics Synthesizer" 150 MHz Embedded cache 4 MB VRAM 66 million polygons per second Audio: SPU2 (+CPU), 48 channels, 44.1- or 48-KHz sampling rate, 2 MB memory RAM: 32 MB RDRAM

Playstation-2: 

Playstation-2 Game medium: Proprietary 4.7-GB DVD and original PlayStation CDs Modem: 56K modem Controller: Two controller ports, "Dual Shock 2" analog controller Other features: Two memory card slots Optical digital output Two USB ports FireWire port Type III PCMCIA card slot Support for audio CDs and DVD-Video

Playstation 2 & Descendents: 

Playstation 2 & Descendents

Playstation-2 Online: 

Playstation-2 Online Playstation-2 60GB internal disk Additional CPU memory Broadband connectivity Mouse, keyboard Flat screen Source - E3 2001

Playstation-2 Online: 

Playstation-2 Online

XBox: 

XBox

XBox: 

XBox Microsoft Xbox (Fall 2001): Processor: Modified Intel Pentium III 733 MHz Maximum bus transfer rate of 6.4 GB per second Graphics: Custom nVidia 3-D graphics chip 300 MHz Approximately 150 million polygons per second Audio: Custom 3-D audio processor RAM: 64 MB (Xbox has a unified memory architecture -- the memory can be allocated to graphics, audio, textures or any other function as needed.)

XBox: 

XBox Game medium: Proprietary 4.7-GB DVD Modem/network:Media communications processor (MCP), 10/100-Mbps Ethernet, broadband enabled, 56K modem (optional) Controller: Four game controller ports Other features: 8-GB built-in hard drive 5X DVD drive with movie playback 8-MB removable memory card Expansion port

Nintendo GameCube: 

Nintendo GameCube Nintendo GameCube (October 2001): Processor: "Gekko" IBM Power PC microprocessor 405 MHz Cache: level 1: 32 KB Instruction and 32 KB Data level 2: 256 KB 32-bit address, 64-bit data bus Maximum bus transfer rate of 3.2 GB per second 0.18 micron copper interconnects Graphics: "Flipper" ATI graphics chip 202.5 MHz 1 MB embedded texture cache 3 MB Mosys 1T-SRAM (This static RAM uses a single transistor per cell, like DRAM.) Approximately 12 million polygons per second

Nintendo GameCube: 

Nintendo GameCube Audio: Special 16-bit digital signal processor, 64 channels, 48-KHz sampling rate RAM: 40 MB (24 MB 1T-SRAM, 16 MB of 100-MHz DRAM) Game medium: Proprietary 1.5-GB optical disc Modem: Conexant V90 56K modem (This will be replaced later by a broadband modem.) Controller: Four game controller ports, Wavebird wireless game controller Other features: Handle for carrying Two slots for 4-MB Digicard Flash memory cards or a 64-MB SD-Digicard adapter High speed parallel port Two high speed serial ports Analog and digital audio-video outputs

Visual Reality: 

Visual Reality Visual reality is 80M polygons/picture [Catmull,1984] & [NRC 95, pg. 252]. 80M polygons/picture at 60 pictures/second (fps) is 4.8B polygons/second. We are talking about machines that can visually display computer images indistinguishable from reality.

Hardware: 

Hardware Graphics With such graphics hardware, the problems we focus on become very different. We no longer worry about culling polygons from the graphics pipeline but rather more about how we reach out and grab a polygon and do something with it.

Broadband Network: 

Broadband Network By the year 2003, broadband connectivity will pass by 100 million homes in the United States. We hope - with the economic downturn, this may take longer. But it will happen. This is connectivity in the range of 10M bps to 100M bps to the home. What bandwidth do we need for streaming content? 100M bps What quality of service will we need? We need reliable, guaranteed bandwidth.

High Speed Access: 

High Speed Access

Wireless, handheld …: 

Wireless, handheld … Computers that are small enough to roll up in our sleeve & are fully connected to the net where ever we are,with connectivity in the 10M bps to 100M bps range. And these computers have the video camera & microphone & the 3.4TB chip for storage & have long lasting batteries …

So we have all the hardware & networking we need, soon …: 

So we have all the hardware & networking we need, soon … It is a simple mater of some research investments & a place for that research to be performed …

A proposal for a new laboratory: 

A proposal for a new laboratory The Future of Interactive Networked Entertainment Laboratory

Laboratory for the Future of Interactive Networked Entertainment - A Vision: 

Laboratory for the Future of Interactive Networked Entertainment - A Vision Examine, develop and guide the network infrastructure & software architectures necessary for interactive networked entertainment. Participate in the development of the technology for the digital distribution of motion pictures and home cinema. Guide the technology for the distribution of music. Examine, develop and select technologies for its content creation futures. Transition those developed technologies to the various corporate divisions.

FINE Laboratory Component Parts: 

FINE Laboratory Component Parts The Networked Entertainment Environments Group The Cinema of Tomorrow Group The Future of Music Group The Content Creation Futures Group The Technology Transition Group

The Networked Entertainment Environments Group: 

The Networked Entertainment Environments Group The Networked Entertainment Environments Group would look at all issues relating to networking infrastructure, and software architectures. A nexus for focusing on the network infrastructure and the software architectures for current and future networked entertainment environments. Current issues that would be examined by this group include wireless delivery systems, the advent of broadband, streaming solutions, and secure networked content management. Near future issues include the potential of fiber to the home, high bandwidth networks, the Next Generation Internet, streaming media, quality of service management, interoperability software architectures, architectures for dynamic extensibility, and VR Broadband.

The Cinema of Tomorrow Group: 

The Cinema of Tomorrow Group The Cinema of Tomorrow Group would look at all issues relating to technologies for digital and home theaters. We place Filmed Entertainment and Cable Networks both under the Cinema of Tomorrow Group as there are commonalities of technologies with respect to next generation cinemas and home theaters. Issues of immediate importance for this group include digital movie distribution, quality digital projection, quality sonic immersion (DTS, THX, SurroundSound), and DVD. Near future issues include interactive cinema, interactive television, HDTV, spatial sound, and set-top boxes.

The Future of Music Group: 

The Future of Music Group The Future of Music Group would look at all issues relating to e-Music. A current issue this group would be examining is the fact that we can’t control the flow of music. Ripping a song from a music CD and flinging it onto the Internet is possible with any home computer and 60% of American homes have computers. So we need to think and plan on what to do. In the mix are Napster/Gnutella type streaming solutions, intellectual property security, Internet radio/webcasting, and the distribution and sale of MP3s. Near future issues this laboratory might be looking at include wireless digital music (handhelds, mobile phones), planning for the advent of broadband, streaming for the Next Generation Internet, and technologies for royalty collection (net-based automatic music recognition and their derivatives).

The Content Creation Futures Group: 

The Content Creation Futures Group The Content Creation Futures Group would look at all issues relating to using the latest in computing and networking technology in the production of prototypes of the future of interactive networked entertainment. This group will play a crucial part as the video game industry is now larger than the motion picture industry. Current issues this group would examine are set-top boxes versus game consoles (X-Box, Playstation-2), mobile delivery systems, and the analysis and coherent planning for the utilization of 3D graphics.

The Content Creation Futures Group: 

The Content Creation Futures Group Near-future issues include novel display technologies and their impact on content creation, technologies for human body tracking for interactive TV, full sensory interfaces for LBEs (visual, auditory, olfactory, and haptic), VR Broadband (3D virtual environments and games on the Internet), content creation tools, computer-generated autonomy and computer-generated characters, story line engines for the rapid prototyping of interactive content and motion picture story, technologies for autonomous, real-time story direction and interaction.

The Technology Transition Group: 

The Technology Transition Group The Technology Transition Group would look at all issues relating to taking the center’s developed technologies and content prototypes and hardening them for placement into the field, on the Internet, and into the hands of the entertainment buying public. It is expected that members of this group will come from the other groups as technologies and content delivery systems become ready and that this group will productize those technologies, utilizing members of the original development teams.

For each of these groups: 

For each of these groups For each of these groups, we plan on evaluating the developed technologies using world-renowned social psychologists and technical anthropologists so that we can insure that what we are developing provides a desirable, consumer pre-tested experience of the materials proposed to be offered for our community of members.

Where are these investments going to be made?: 

Where are these investments going to be made?

Entertainment Corporations?: 

Entertainment Corporations? Disney? Just interested in the D of R&D AOL Time Warner? Spending most of its time with its merger & layoffs Internet Startups? Yeah, right - for a couple of years, which now seem a long time ago, we saw this R&D model.

Universities?: 

Universities? Maybe There are a few that have laboratories that are working on many of these issues: The MOVES Institute, Naval Postgraduate School UCF IST USC ICT Of course, others for pieces & parts …

The MOVES Institute: 

The MOVES Institute Research Programs

Mission: 

Mission Research, application and education in the grand challenges of modeling, virtual environments and simulation. The institute operates both independently and in collaboration with the various Navy and Defense centers to: carry out basic and applied research; analyze modeling, virtual environments and simulation programs; create advanced prototypes; and develop technologies and applications for the defense community.

Research Scope: 

Research Scope Our scope encompasses the fundamentals and applications required for the next generation including: 3D Visual Simulation Networked Virtual Environments Computer-Generated Autonomy Human Performance Engineering Technologies for Immersion Defense and Entertainment Collaboration Evolving Operational Modeling

Organizational Structure: 

Organizational Structure Director Michael Zyda Technical Directorate John Hiles - Computer-Generated Autonomy Don Brutzman - 3D Visual Simulation & Networked Virtual Environments Rudy Darken - Human Performance Engineering & Technologies for Immersion Michael Capps - Defense & Entertainment Collaboration Alex Callahan - Evolving Operational Modeling

Organizational Structure: 

Organizational Structure Infrastructure Director John Falby Research Programs Director Open Products Director David Williams Technical Writer & Web Support Margaret Davis Technical Support Meg Boone

Executive Advisory Board: 

Executive Advisory Board Executive Advisory Board provides guidance on funding for research and products. VADM Richard Mayo, USN - N6 RADM Lee Kollmorgen, USN (ret) CAPT Dennis McBride, USN (ret), PhD - VP, Potomac Institute Dr. Harold Hawkins, ONR CAPT Mike Lilienthal, USN - Director of DMSO Dell Lunceford - Director of AMSO Dr. Mike Bailey - Director, USMC Combat Developments Center Michael Kapp - Founder & President Time Warner Special Projects (ret)

Technical Advisory Board: 

Technical Advisory Board Technical Advisory Board provides guidance on technical alternatives to proposed research and products. Dr. Phil Barry, DMSO Chief of S&T Division CAPT Richard Bump, USN - Director, Navy Modeling & Simulation Management Office, N6M Jim Weatherly - Deputy Director, Navy Modeling & Simulation Management Office, N6M LCDR Dylan Schmorrow, USN – ONR VIRTE Program Manager & DARPA Program Manager Dr. Bowen Loftin - Old Dominion University, Director VMASC Brian Goldiez - Director, UCF Institute for Simulation & Training

Projects: 

Projects

3D Visual Simulation: 

3D Visual Simulation Project VAST - Virtual At Sea Training Third Fleet - Tactical Information Visualization Generic Hub: XML-Based Information Interchange for Defense Messaging, Shipboard/Theater Command & Control, and Distributed 3D Battlespace Visualization

Networked Virtual Environments: 

Networked Virtual Environments NPSNET-V - An Architecture for Constructing Scalable, Dynamically Extensible, Networked Virtual Environments Explorations in Dynamic Extensibility & Dynamic Behavior Protocols for Web-Based, Networked Virtual Environments Virtual Reality Transfer Protocol (vrtp) Development

Computer-Generated Autonomy: 

Computer-Generated Autonomy A Symbolic Reactive Agent Architecture for Multi-Agent Systems Interactive Computer Generated Stories DMSO - Self-Learning Autonomous Agents for Distributed Simulations

Human Performance Engineering: 

Human Performance Engineering Virtual Environment Spatial Knowledge Training & Acquisition Virtual Technologies & Environments - Navy & Marine Corp Expeditionary Warfare DARPA Augmented Cognition - The Context Machine: Determining Context from Symbolic Inputs

Technologies for Immersion: 

Technologies for Immersion CDTEMs - A VR CAVE for the MOVES Institute Hybrid Inertial Motion Tracking for Inserting Humans into Networked Synthetic Environments

Modeling & Simulation: 

Modeling & Simulation Modeling Tactical Level Combat Using A Multi-Agent System Design Paradigm (GI-AGENT) Modeling Conventional Land Combat Using Generalization of the Different Combat Entities and Combat Operations in a Multi-Agent System Scenario Authoring and Visualization for Advanced Graphical Environments (SAVAGE): Amphibious Raid at Red Beach, Camp Pendleton California Modeling and Simulation of the Ocean Environment

Evolving Operational Modeling: 

Evolving Operational Modeling Evolving Operational Modeling - Navy Concepts, Research, & Analysis Network (NCRAN) Course Development Hands-on Operational Modeling Current Programs in Operational Modeling

Defense & Entertainment Collaboration: 

Defense & Entertainment Collaboration Audio Engineering and Sound Design Issues in VE: Lessons Learned from the Entertainment Industry The MOVES Institute War Game Laboratory - A Videogame Research & Production Facility SimSecurity - A Distance Learning and Virtual Laboratory for Information Assurance Army Game Project

In closing: 

In closing http://movesinstitute.org