Slide1: Cassini Distributed Instrument Operations – What We’ve Learned Since Saturn Orbit Insertion Jet Propulsion Laboratory California Institute of Technology NASA Sue Linick Carole Boyles Pam Woncik


Instrument Ops Concept: Instrument Ops Concept Distributed Instrument Teams operate from Principal Investigator’s home institution Responsible for their own ground system Operate from two Science Ops Planning Computers (SOPCS) Perform instrument operations in flight Reasons for distributing instrument operations (early 1990s) Cassini mission is long Frequent travel or co-location would increase cost Keep instrument expertise around for the duration of the mission Instrument problems likely in tour Scientists wanted more control


Tour Planning: Tour Planning Implementation of the Science Operations Plan began 3 years before Saturn Orbit Insertion 75 orbits about Saturn 45 Titan flybys 9 Targeted Satellite Flybys Sequences developed early put on the shelf


The 12 Orbiter Instruments : The 12 Orbiter Instruments INMS CDA CA P S RPWS MIMI INCA & LEMMS CIRS F & P Instruments ORS Instruments ISS V I MS MAG Microwave Remote Sensing Radar & Radio Science


Instrument Team Architecture & Responsibilities: Instrument Team Architecture & Responsibilities Design Science Observations Develop Subsequences Monitor Instr Health & Safety Process Data Archive Data Analyze Data & Publish Results Resolve Anomalies Perform GDS Engineering Maintain GDS Maintain Flight S/W Develop Subsequences Design Science Observations Maintain GDS Process Data Perform Data Analysis Archive Data Facility Instrument Teams Principal Investigator Teams Science Ops Team Instrument Engr Team Develop Sub-Sequences Monitor Instrument Health & Safety Perform Level-1 Data Processing Resolve Anomalies Maintain Flight S/W Maintain Testbed & Models Maintain Data Processing S/W RSS & Radar also design Observations RSS & Radar also archive data


Slide6: Instrument Teams CAPS Cassini Plasma Spectrometer, Southwest Research Institute, San Antonio, USA CDA Cosmic Dust Analyzer, Max Planck-Institut fuer Kernphysik, Heidelberg, Germany CIRS Composite Infrared Spectrometer, Goddard Space Flight Center, Greenbelt, USA ISS Imaging Science Subsystem, Space Science Institute, Boulder, USA INMS Ion And Neutral Mass Spectrometer, Southwest Research Institute, San Antonio, USA MAG Dual Technique Magnetometer, Imperial College, London, UK MIMI Magnetospheric Imaging Instrument, John Hopkins University, Baltimore, USA RADAR Radar, Jet Propulsion Laboratory, Pasadena, USA RPWS Radio and Plasma Wave Spectrometer, University of Iowa, USA RSS Radio Science Subsystem, Jet Propulsion Laboratory, Pasadena, USA UVIS Ultraviolet Imaging Spectrograph, University of Colorado, Boulder, USA VIMS Visible And Infrared Mapping Spectrometer, University of Arizona, Tucson, USA * RSS Radar * ISS UVIS * * * * VIMS RPWS INMS * * CAPS CIRS MIMI * MAG CDA * * * * * * CAPS Co-Investigator Support Centre National d’Etudes Spatiales (CNES) Los Alamos National Laboratory Goddard CIRS Co-Investigator Support University of Madone, Paris University of Paris Centre National d’Etudes Spatiales (CNES) Oxford, England ISS Co-Investigator Support Freie Universitat Berlin Queens Mary College University of London


Tour Operations: Tour Operations Seq. N+1 Seq. N+2 Seq. N+3 Workforce (FTE’s) 1 2 3 4 4 3 2 1 Data Processing Science Analysis & Archive Seq. N-1 SOP U/D SSUP EXEC Seq. N Long Range Mission Planning Process Aftermarket Aftermarket Aftermarket Aftermarket Seq. N+4 Health & Safety U P L I N K D O W N L I N K Needed FTE’s Work Force for this Slice: FTE’s Funded SOP U/D SSUP EXEC SOP U/D SSUP EXEC SOP U/D SSUP Health & Safety Data Processing Science Analysis & Archive Seq. N SSUP SSUP


Tour Surprises: Titan’s atmosphere was denser and more uncertain that had been predicted 21 of 28 Titan flyby altitudes had to be raised Playback policy change Transmit data to the ground once Lost science data from some of 13 Titan flyby’s A Titan contingency playback plan is now developed “Live Update” process required to update pointing for close satellite flybys is now almost obsolete Tour Surprises


Distributed Instrument Ops Lessons Learned: . Distributed Instrument Ops Lessons Learned . : . . . . . . . . . . Communications Use all methods of communications – be consistent Meeting Planning Tool used for setting up meetings Meetings attended via meet-me lines (audio) with presentations on the web Web used for schedules, documentation and distribution lists Video conference was tried Three Scheduled Face-to-Face Meetings Yearly Planetary Science Group Two in the U.S., one in Europe


Distributed Instrument Ops Lessons Learned: . . : . . . . . . . . . . Additional personnel at Mission Ops Center needed Payload Engineer/s at the Mission Ops Site Small Team of Ground System Experts SA SOPC support at Mission Ops Center & remote sites Operations Validation Test important science events One good test is worth 1,000 good opinions Also use a 2nd set of eyes Distributed Instrument Ops Lessons Learned


Distributed Instrument Ops Lessons Learned: * . * . . . . . . . . . Ground and Flight Software Keep a budget for flight software upgrades Cassini Instrument Flight S/W is still being changed Don’t freeze your ground /flight software when entering a new phase Updates to S/C flight S/W affect all teams Archive (Cassini archiving began June 2005) Design data products to meet Planetary Data System (PDS) formats Design an operational pipeline to do the archiving Develop tools to verify the archive before delivery Distributed Instrument Ops Lessons Learned


Distributed Instrument Ops Lessons Learned: . . : . . . . . . . . . . Testbeds Instrument Testbeds Must maintain testbed compatibility with flight H/W and S/W CDA Team ported ADA compiler S/W to run on multiple platforms Either purchase extra testbed H/W or port compiler S/W Distributed Testbed Capability Remote Terminal Interface Unit – Use at comfort of home institute Simulates instrument/spacecraft interface (medium fidelity) for each instrument Need SOPC Testbeds Distributed Instrument Ops Lessons Learned * *


Distributed Instrument Ops Lessons Learned: . . : . . . . . . . . . . International Traffic in Arms Regulations (ITAR) Regulate what information can flow between the U.S. organization and the international partner ITAR rules and regulations have become more of a challenge Projects need at least one ITAR/security expert Keep up with with ITAR/IT security changes Ability to communicate technical solutions to non-technical people New Projects - Work the Technical Agreement Authorization early On-going Projects Look at the affect of not sharing information Distributed Instrument Ops Lessons Learned * *


Smart Solutions With Limited Workforce: Instrument Team Workforce Hire high school or college students for the summer. Engage Co-Investigators to design their own observations. Co-Is must be provided with tools and training. Cross-train the entire instrument team – Rotate personnel through jobs to keep the skills sharp and the back-up ready. Provide automation for remote team common processes Synchronizing Input Files Across Distributed Ops Sites To design science observations, many input files are required. Script that pushes the correct input files/sequence to all SOPCS Smart Solutions With Limited Workforce


Smart Solutions with Limited Workforce: Instrument Testbed Operability Instrument Team implemented a way to control their testbed from any where in the world using the internet. Can set up and run tests and check status Additional network security precautions had to be implemented Faster Software without Upgrading Hardware The Cassini Pointing Design Tool (PDT) performance too slow Tour sequence development was “crunched” when Instrument Teams had to work on six sequence loads in parallel Ported PDT code to Linux (done relative quickly) Developed a client server version where the client was a PC and the Server was the SOPC. Increased throughput by 50%. Smart Solutions with Limited Workforce


Smart Solutions with Limited Workforce: Smart Solutions with Limited Workforce Faster Command Generation Process Project developed a web application we call Automated Sequence Processor (ASP) which generates command requests in a matter of minutes and notifies the Flight Controller that a file is ready to be sent. ASP is used only for non-interactive instrument commands ITAR/Security – How to Share Set up a Science Users Net Separate from JPL so restricted JPL resources cannot be accessed for it, but access to Cassini needed resources is available. Allows members of any Cassini Instrument Team to work together on the same network, sharing Cassini data a resources. Cassini Instrument Teams can remotely access this net or can bring their laptops to JPL with their own software installed and connect to this secure network.


Conclusions: Conclusions Distributed Instrument Operations does work, even for a mission as large as Cassini Adds a level of complexity Processes are long, methodical and rigid Flexibility is limited Communication is the biggest challenge Additional cost should be expected for going distributed Co-Investigators are a good source of additional workforce