Real Time Data Strategy and Architecture

Views:
 
     
 

Presentation Description

This describes a generalised and structured approach to defining a strategy for collecting (near or actual) real time, high volume data. The appproach can be applied to areas such as Telemetry, Big Data, Smart Metering and Internet of Things implementations and operations. This proposed structured approach is intended to ensure that complexity is understood and can be appropriately addressed at an early stage before problems become to embedded to be solved. Real time situational data gives rise to situational awareness and understanding which in turn presents opportunities for effective and rapid situational decisions. Real time situational data enables greater situational visibility which means increased operational intelligence.

Comments

Presentation Transcript

Real Time Data Strategy And Architecture:

Real Time Data Strategy And Architecture Alan McSweeney http:// ie.linkedin.com/in/alanmcsweeney

Real Time Data Collection Strategy And Architecture Approach:

Real Time Data Collection Strategy And Architecture Approach These notes are concerned with describing a generalised approach to defining a strategy for collecting (near or actual) real time, high volume data This is data is generated by sensors that is transmitted to a central location for processing, reporting, analysis and ultimately action Sensors can be regarded as logical or physical sources of streams of measurement data (Near) real time data can be termed Telemetry March 8, 2016 2

Real Time Data Collection Strategy And Architecture Approach:

Real Time Data Collection Strategy And Architecture Approach Approach adopted from TMForum Resource Domain Frameworx eTOM (Enhanced Telecoms Operating Model) Business Process Framework Approach has been generalised for real-time data and telemetry March 8, 2016 3

Real Time Data Collection Strategy And Architecture:

Real Time Data Collection Strategy And Architecture Collect data from range of data sources across the organisation’s (internal and external) operating landscape Approach can be applied to collection of measurement data from multiple sources and of multiple types through “sensors”: Different entities interacting with the organisation and the gathering of data on different actions and events Approach can be applied to areas such as Telemetry, Big Data, Smart Metering and Internet of Things implementations and operations March 8, 2016 4

Real Time Data Collection Strategy And Architecture:

Real Time Data Collection Strategy And Architecture March 8, 2016 5 Real Time Data Strategy Telemetry Strategy Big Data Strategy Internet Of Things Strategy SmartX Strategy Digital Strategy

Why Have A Real Time Data Collection Strategy And Architecture?:

Why Have A Real Time Data Collection Strategy And Architecture? Real time situational data gives rise to situational awareness and understanding which in turn presents opportunities for effective and rapid situational decisions What is happening – usage, performance What can be improved What are the optimisation and productivity opportunities Real time situational data enables greater situational visibility which means increased operational intelligence March 8, 2016 6

Organisation Operating Landscape:

Organisation Operating Landscape Business Customer Retail Customer Shareholder Shareholder Partner Outsourcer Competitor Supplier Regulator Contractor Service Provider Distributor Intermediary Collaborator Sub-Contractor Franchisee Counterparty Intermediary Representative Agent Researcher Client Public March 8, 2016 7 Dealer

Organisation Operating Landscape:

Organisation Operating Landscape The operating landscape of the organisation defines the number and type of interactions outside the organisation This operating landscape affects the decision on what and how to measure Not all interactions with all entities are measured Need to be realistic about what can be collected and processed Need to understand the need for sensors to collect data March 8, 2016 8

Data Sensors – Combinations Of Options:

Data Sensors – Combinations Of Options Sensor Type – Physical sensors are actual units such as RTUs (Remote Telemetry Units) that measure and generate data while Logical sensors are representations of data sources Sensor Ownership – Direct sensors are those installed and maintained by the collecting organisation while Indirect sensors are installed by a third-party March 8, 2016 9 Sensor Type Logical Physical Sensor Ownership Direct Web Site and App Activity and Usage Data Internet of Things Devices Remote Real Time Unit Smart Devices Indirect Third-Party Web Site and App Activity and Usage Data Third-Party Devices

Measurement Data Sensors:

Measurement Data Sensors Business Customer Retail Customer Public March 8, 2016 10 Ò Ò Ò Shareholder Shareholder Partner Outsourcer Competitor Supplier Regulator Contractor Service Provider Distributor Intermediary Collaborator Sub-Contractor Franchisee Counterparty Intermediary Representative Agent Researcher Client Dealer

Measurement Data Sensors:

Measurement Data Sensors These can gather data for different measures using multiple measurement techniques These can be regarded as collectors of any data – usage, activity, performance, throughput Each measurement type will have a unit or dimension Logical representation of data collectors Need to decide on what can be measured, what to measure and how to measure it March 8, 2016 11

Measurement Data Sensors – Decide On What And How To Measure:

Measurement Data Sensors – Decide On What And How To Measure Business Customer Retail Customer Shareholder Shareholder Partner Outsourcer Competitor Supplier Regulator Contractor Service Provider Distributor Intermediary Collaborator Sub-Contractor Franchisee Counterparty Intermediary Representative Agent Researcher Client Public March 8, 2016 12 Dealer Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò Ò

Real Time Data Architecture Complexity:

Real Time Data Architecture Complexity March 8, 2016 13 Real Time Data Architectures Tend To Focus On The Simplicity Of A Possible Real Time Data Collection Architecture … … And Ignore The Complexity And Difficulties Of Sensor Installation, Operation, Maintenance, Errors, Rework, Logistics, Service Management, Data Volumes

Real Time Data Architecture Complexity:

Real Time Data Architecture Complexity Structured approach is intended to ensure that complexity is understood and can be appropriately addressed at an early stage before problems become to embedded to be solved March 8, 2016 14

Real Time Data Architecture Complexity:

Real Time Data Architecture Complexity March 8, 2016 15 Don’t Let The Ignored But Knowable And Addressable Complexity Sink Your Real Time Data Programme/Initiative

Real Time Data Strategy And Architecture Issues:

Real Time Data Strategy And Architecture Issues What business benefits will a real time data strategy yield? How can the benefits be realised? What is the business case for investment in a real time data strategy? What infrastructure, communications/connectivity, data and application architectures are needed to support a real time data strategy? What integration is required with existing applications? What skills, capabilities and changes does the organisation need to adopt and exploit a real time data strategy? How is the real time data strategy managed and serviced? What solution and service providers and tools/platforms and sourcing strategy should be selected? What are the privacy and security issues and requirements? March 8, 2016 16

Evolution Of Real Time Data Architecture:

Evolution Of Real Time Data Architecture March 8, 2016 17 Create Awareness f or Real Time Data Investment Undefined/unarticulated/uncertain real time data strategy . Real time data processes are ad hoc, focussed on individual solution and outcomes vary widely. Crawl Walk Run Fly Building Real Time Data Investment Foundation Implement investment controls and develop key foundational capabilities Developing Complete Real Time Data Portfolio Comprehensive selection and control processes with benefit and risk criteria linked to strategy requirements Improving Real Time Data Processes Process evaluation techniques focus on improvement of performance and management Leveraging for Strategic Outcomes Real time data management , reporting and analysis techniques are deployed for strategic mission/business outcomes Admitting There must be a better way Communicating Establishing and communicating business case Governing Making and implementing effective investment decisions Managing Processes, mechanisms and metrics Optimising Sense and respond

Developing A Real Time Data Strategy – Generalised High Level Steps:

Developing A Real Time Data Strategy – Generalised High Level Steps March 8, 2016 18 Real Time Strategy And Planning Real Time Capability Delivery Real Time Development And Retirement Real Time Management and Operations Support And Readiness Real Time Provisioning Real Time Data Collection And Distribution Real Time Problem Management Real Time Performance Management Workforce Management Real Time Data Aggregation and Reporting 4 1 2 3 5 10 9 8 7 6

Developing A Real Time Data Strategy – Generalised High Level Steps:

Developing A Real Time Data Strategy – Generalised High Level Steps Comprehensive set of steps from definition of what is required from real time data to commissioning of real time data collection facilities to effective use of collected data Not all steps are relevant to all real time data initiatives For example, if installation, commissioning, operation and maintenance of physical sensors is not applicable then related steps will not be required Represents an idealised organisation and process breakdown across entire spectrum of real time data from strategy to workforce management to device installation, data collection and data usage and actioning Provides a basis for developing a work breakdown and an implementation plan Represents a comprehensive structure that can be adapt to meet its long-term real time data needs March 8, 2016 19

Developing A Real Time Data Strategy – Generalised High Level Steps 1 – 2:

Developing A Real Time Data Strategy – Generalised High Level Steps 1 – 2 Step Scope 1. Real Time Strategy And Planning Develop real time data strategy, policies and plans for the organisation governed by long-term business, market, product and service needs directions Perform research and analysis to determine real time targets and strategies to reach the defined targets Understand the real time data capabilities of the existing infrastructure Build real time data model Define approaches to real time data quality and real time data governance Define and agree the infrastructure needs based on market, product and service strategies of the organisation Manage the capabilities of the suppliers and partners to develop and deliver new real time data capabilities and detail the approach to the deployment of new and enhanced infrastructure Define the real time data implementation standards sought, key real time data capabilities required, real time data support levels and approaches required, real time data design elements to be developed, and real time data cost parameters and targets. Define the policies relating to technical real time data sensors and their implementation 2. Real Time Capability Delivery Ensure that network, application and computing real time data facilities are deployed Provide the physical real time data capabilities necessary for the ongoing operations and long-term well-being of the organisation and ensure the basis on which all real time data capabilities and services will be constructed Plan real time data resource supply logistics Plan real time data sensor i nstallation Verify the real time data sensor i nstallation Handover real time data capabilities to operations March 8, 2016 20

Developing A Real Time Data Strategy – Generalised High Level Steps 3 – 6:

Developing A Real Time Data Strategy – Generalised High Level Steps 3 – 6 Step Scope 3. Real Time Development And Retirement Develop new or enhance existing technologies and associated real time data types applying the capability definition or requirements defined by the Real Time Strategy And Planning step Decide on acquisition of real time data resources from third parties Retire or remove technology and associated real time data resource types that are no longer needed by the organisation 4. Real Time Management and Operations Support And Readiness Manage the types of real time data resources and ensure that necessary application, computing and network facilities are available and ready to implement and manage resource instances 5. Workforce Management Manage the direct and indirect personnel who perform work assignments or work orders relating to real time data resources installation, commissioning and maintenance as well as managing the actual activity being performed Report and monitor activities Establish, manage and allocate work assignments to direct and indirect personnel Establish and manage priority and urgent assignment capabilities to allow for modification of work assignments as required to meet urgent and high priority conditions 6. Real Time Provisioning Allocate, install, configure, activate and test of real time data resources to meet the defined requirements Resolve real time resource capacity issues, availability issues or failure conditions Configure and activate physical and/or logical real time resources Update of real time resource register database to reflect that the specific real time resource has been allocated, modified or recovered March 8, 2016 21

Developing A Real Time Data Strategy – Generalised High Level Steps 7 – 8:

Developing A Real Time Data Strategy – Generalised High Level Steps 7 – 8 Step Scope 7. Real Time Data Collection And Distribution Collect and distribute management information and real time data between data sources and service instances and other organisation functions and processes Work with the real time data resource and service instances to collect usage, network and technology events and other management information for distribution to other processes within the organisation Handle and process command, query and other management information for distribution to resource and service instances Process the data and management information through actions such as filtering, aggregation, formatting, transformation and correlation of the information before presentation to other processes, real time data instances or service instances Perform usage reporting, fault and performance analysis, service quality management analysis, resource performance analysis of resources and services 8. Real Time Problem Management Manage real time data resource problems including security events Detect, analyse, manage and report on resource alarm event notifications Initiate and manage real time data resource problems reports Perform real time data resource problem localization analysis and resolve problems Reporting progress on resource trouble reports to other processes Assign and track real time data resource problem testing and resolution activities Managing real time data resource problem urgent conditions March 8, 2016 22

Developing A Real Time Data Strategy – Generalised High Level Steps 9 – 10:

Developing A Real Time Data Strategy – Generalised High Level Steps 9 – 10 Step Scope 9. Real Time Performance Management Manage, track, monitor, analyse and report on the performance of real time data resources Identify real time data resource performance disruptions or a service performance disruptions 10. Real Time Data Aggregation and Reporting Manage real time data resource events by correlating and formatting them into a usable format Report of real time data resource data March 8, 2016 23

Sample Expansion – Step 1 – Real Time Strategy And Planning – Activities 1.1 – 1.7:

Sample Expansion – Step 1 – Real Time Strategy And Planning – Activities 1.1 – 1.7 March 8, 2016 24 Step Scope 1.1 Gather And Analyse Real Time Data Information Research and analyse customer, technology, competitor and marketing information to identify new real time data requirements and industry real time data capabilities and availability. 1.2 Manage Real Time Data Research Manage internally driven research investigations and activities which are used to provide detailed technical assessment or investigation of new and emerging real time data capabilities. 1.3 Establish Real Time Data Strategy And Architecture Establish the real time data strategies based on market trends, future needs, technical capabilities and addressing shortcomings in existing real time data support. 1.4 Define Real Time Data Support Strategies Define the principles, policies and performance standards for the operational organisation providing real time data support. 1.5 Produce Real Time Data Business Plans Develop and deliver annual and multi-year real time data plans in support of services, products and offers that include volume forecasts, negotiation for required levels of resources and budgets. Obtain real time data development and management as well as supply chain commitment and executive approval for the plans. Identify the impacts that new or modified real time data infrastructure will cause on the installed infrastructure and workforce and establish the functions and benefits that new or modified real time data will provide to users. 1.6 Develop Real Time Data Partnership Requirements Identify the requirements for real time data capabilities to be sourced from partners or suppliers, and any real time data capabilities to be delivered internally to the organisation. 1.7 Gain Enterprise Commitment To Real Time Data Plans Obtain organisation commitment to the resource strategy and business plans including all aspects of identification of stakeholders and negotiation to gain stakeholder approval.

Sample Expansion – Step 1 - Real Time Strategy And Planning – Activity 1.5 – Tasks 1.5.1 – 1.5.5:

Sample Expansion – Step 1 - Real Time Strategy And Planning – Activity 1.5 – Tasks 1.5.1 – 1.5.5 March 8, 2016 25 Step Scope 1.5.1 Develop And Deliver Annual/Multi Year Real Time Data Business Plans Develop and deliver annual/multi year real time data business plans focus on developing and delivering annual and multi-year real time data in support of services, products and offers that include volume forecasts, negotiation for required levels of resources and budgets, gaining real time data development and management as well as supply chain commitment and executive approval for the plans. 1.5.2 Forecast High Level Real Time Data Demand And Capture New Opportunities Forecast real time data demand and capture new opportunities processes ensures that budgets are assigned which allow the organisation to implement the real time data capabilities and capacity necessary for the future needs of their customers and potential customers. 1.5.3 Assess Impact Of Real Time Data Business Plans Asses impact of real time data business plan processes assess the impacts that new or modified real time data infrastructure will cause on the installed infrastructure and workforce, and establish the functions and benefits that new or modified real time data will provide to users 1.5.4 Identify Timetables For New Real Time Data Capability Introduction Identify timetables for new real time data capability introduction 1.5.5 Identify Logistics For New Real Time Data Capability Introduction Identify logistics for new real time data capability introduction

Real Time Data Quality And Data Governance:

Real Time Data Quality And Data Governance Real-time data is inherently: High Volume – lots of sensors generating lots of data Noisy – lots of variation, statistical noise, inaccuracies, sensor drift, errors, incorrect calibration Changing – data landscape subject to substantial change along the dimensions of data sources, volumes, types Inconsistent – different sensor types measuring different values with different units of measure and at different intervals Heterogeneous – very mixed data sources Non-standardised – multiple, emerging, overlapping standards and approaches Sophisticated approach to data quality and data governance must be embedded in any real-time architecture Traditional approach of data collection storage and analysis may need to change to handle data volumes and quality – data filtering, quality, summarisation and transformation component March 8, 2016 26

Real Time Data Principles:

Real Time Data Principles March 8, 2016 27 To manage and utilise real time information as a strategic asset To implement processes, policies, infrastructure and solutions to govern, protect, maintain and use real time information To make relevant and correct real time information available in all business processes and IT systems for the right people in the right context at the right time with the appropriate security and with the right quality To exploit real time information in business decisions, processes and relations

Real Time Data And Data Governance And Data Quality:

Real Time Data And Data Governance And Data Quality Data Quality - measure, assess, improve, and ensure the fitness of data for use Data Governance - authority and control over the management of data assets March 8, 2016 28

Real Time Data Governance:

Real Time Data Governance Core function of real time data management Interacts with and influences each of the surrounding ten data management functions Data governance is the exercise of authority and control (planning, monitoring, and enforcement) over the management of data assets Data governance function guides how all other data management functions are performed High-level, executive data stewardship Data governance is not the same thing as IT governance Data governance is focused exclusively on the management of data assets March 8, 2016 29

Real Time Data Governance – Definition and Goals:

Real Time Data Governance – Definition and Goals Definition The exercise of authority and control (planning, monitoring, and enforcement) over the management of data assets Goals To define, approve, and communicate data strategies, policies, standards, architecture, procedures, and metrics To track and enforce regulatory compliance and conformance to data policies, standards, architecture, and procedures To sponsor, track, and oversee the delivery of data management projects and services To manage and resolve data related issues To understand and promote the value of data assets March 8, 2016 30

Real Time Data Governance Structure:

Real Time Data Governance Structure March 8, 2016 31 Real Time Data Governance Framework Real Time Data Architecture to Implement Data Governance Real Time Data Infrastructure to Implement Data Architecture Real Time Data Operations to Manage Data Infrastructure

Real Time Data Governance Activities:

Real Time Data Governance Activities March 8, 2016 32

Real Time Data Governance Inputs And Outputs:

Real Time Data Governance Inputs And Outputs March 8, 2016 33 Business Goals Business Strategies IT Objectives IT Strategies Data Needs Data Issues Regulatory Requirements Inputs Business Executives IT Executives Data Stewards Regulatory Bodies Suppliers Intranet Website E-Mail Metadata Tools Metadata Repository Issue Management Tools Data Governance KPI Dashboard Tools Executive Data Stewards Coordinating Data Stewards Business Data Stewards Data Professionals DM Executive CIO Participants Data Policies Data Standards Resolved Issues Data Management Projects and Services Quality Data and Information Recognised Data Value Primary Deliverables Data Producers Knowledge Workers Managers and Executives Data Professionals Customers Consumers Data Value Data Management Cost Achievement of Objectives # of Decisions Made Steward Representation / Coverage Data Professional Headcount Data Management Process Maturity Metrics Real Time Data Governance

Real Time Data Quality Management:

Real Time Data Quality Management Critical support process in organisational change management Data quality is synonymous with information quality since poor data quality results in inaccurate information and poor business performance Data cleansing may result in short-term and costly improvements that do not address the root causes of data defects More rigorous data quality program is necessary to provide an economic solution to improved data quality and integrity Institutionalising and operationalising processes for data quality oversight, management, and improvement hinges on identifying the business needs for quality data and determining the best ways to measure, monitor, control, and report on the quality of data Continuous process for defining the parameters for specifying acceptable levels of data quality to meet business needs, and for ensuring that data quality meets these levels March 8, 2016 34

Real Time Data Quality Management – Definition and Goals:

Real Time Data Quality Management – Definition and Goals Definition Planning, implementation, and control activities that apply quality management techniques to measure, assess, improve, and ensure the fitness of data for use Goals To measurably improve the quality of data in relation to defined business expectations To define requirements and specifications for integrating data quality control into the system development lifecycle To provide defined processes for measuring, monitoring, and reporting conformance to acceptable levels of data quality March 8, 2016 35

Real Time Data Quality Management Inputs And Outputs:

Real Time Data Quality Management Inputs And Outputs March 8, 2016 36 Business Requirements Data Requirements Data Quality Expectations Data Policies and Standards Business metadata Technical metadata Data Sources and Data Stores Inputs External Sources Regulatory Bodies Business Subject Matter Experts Information Consumers Data Producers Data Architects Data Modelers Suppliers Data Profiling Tools Statistical Analysis Tools Data Cleansing Tools Data Integration Tools Issue and Event Management Tools Tools Data Quality Analysts Data Analysts Database Administrators Data Stewards Other Data Professionals DRM Director Data Stewardship Council Participants Improved Quality Data Data Management Operational Analysis Data Profiles Data Quality Certification Reports Data Quality Service Level Agreements Primary Deliverables Data Value Statistics Errors / Requirement Violations Conformance to Expectations Conformance to Service Levels Metrics Real Time Data Quality Management Data Stewards Data Professionals Other IT Professionals Knowledge Workers Managers and Executives Customers Consumers

Real Time Data Quality Plan Definition Activities:

Real Time Data Quality Plan Definition Activities March 8, 2016 37

Developing A Real Time Data Strategy – Generalised High Level Steps:

Developing A Real Time Data Strategy – Generalised High Level Steps 10 high level steps with activities and tasks Over 180 detailed tasks for a complete view of work required Comprehensive set of steps from definition of what is required from real time data to commissioning of real time data collection facilities to effective use of collected data March 8, 2016 38

Real Time Data Strategy and Implementation – Organisation, Function And Process Structure – Steps 1-10:

Real Time Data Strategy and Implementation – Organisation, Function And Process Structure – Steps 1-10 March 8, 2016 39

Step 1 – Real Time Data Strategy And Planning – Processes And Functions Details:

Step 1 – Real Time Data Strategy And Planning – Processes And Functions Details March 8, 2016 40

Step 2 - Real Time Data Capability Delivery – Processes And Functions Details:

Step 2 - Real Time Data Capability Delivery – Processes And Functions Details March 8, 2016 41

Step 3 - Real Time Data Development And Retirement – Processes And Functions Details:

Step 3 - Real Time Data Development And Retirement – Processes And Functions Details March 8, 2016 42

Step 4 - Real Time Data Management and Operations Support And Readiness – Processes And Functions Details:

Step 4 - Real Time Data Management and Operations Support And Readiness – Processes And Functions Details March 8, 2016 43

Step 5 - Workforce Management – Processes And Functions Details:

Step 5 - Workforce Management – Processes And Functions Details March 8, 2016 44

Step 6 - Real Time Data Provisioning – Processes And Functions Details:

Step 6 - Real Time Data Provisioning – Processes And Functions Details March 8, 2016 45

Step 7 - Real Time Data Collection And Distribution – Processes And Functions Details:

Step 7 - Real Time Data Collection And Distribution – Processes And Functions Details March 8, 2016 46

Step 8 - Real Time Data Trouble Management – Processes And Functions Details:

Step 8 - Real Time Data Trouble Management – Processes And Functions Details March 8, 2016 47

Step 9 - Real Time Data Performance Management – Processes And Functions Details:

Step 9 - Real Time Data Performance Management – Processes And Functions Details March 8, 2016 48

Step 10 - Real Time Data Aggregation And Reporting – Processes And Functions Details:

Step 10 - Real Time Data Aggregation And Reporting – Processes And Functions Details March 8, 2016 49

Using Real Time Data Strategy and Implementation Approach:

Using Real Time Data Strategy and Implementation Approach March 8, 2016 50 Activity Timeline Real Time Data Strategy and Implementation Plan                         1 Real Time Data Strategy And Planning                         1.1 Gather And Analyse Real Time Data Information                         1.1.1 Gather Real Time Data Information                         1.1.2 Analyse New Real Time Data Requirements                         1.1.3 Analyse To Develop New/Enhance Real Time Data Requirements                         1.2 Manage Real Time Data Research                         1.2.1 Manage Real Time Data Research Investigations 1.2.2 Manage Administration Of Real Time Data Research                         1.2.3 Define Real Time Data Research Assessment Methodologies                         1.3 Establish Real Time Data Strategy And Architecture                         1.3.1 Establish Real Time Data Strategy                         1.3.2 Develop Real Time Data Strategy                         1.3.3 Establish Real Time Data Delivery Goals                         1.3.4 Establish Real Time Data Implementation Policies                         1.4 Define Real Time Data Support Strategies                         1.4.1 Define Real Time Data Support Principles                         1.4.2 Define Real Time Data Support Policies                         1.4.3 Define Real Time Data Support Performance Standards                         …                        

Using Real Time Data Strategy and Implementation Approach:

Using Real Time Data Strategy and Implementation Approach Use the proposed work breakdown to produce a detailed plan March 8, 2016 51

Real Time Architecture High-Level Components:

R eal Time Architecture High-Level Components March 8, 2016 52 Data Sources Data Collection And Data Source Management Communications And Security Data Integration Data Quality/ Summary/ Filter/ Transformation Data Storage Data Storage Infrastructure Data Reporting and Analysis System Management, Administration and Control External Systems (Asset Management, Workforce Management)

Real Time Architecture High-Level Components:

Real Time Architecture High-Level Components March 8, 2016 53 Component Description Data Sources These are the data collection/generation sources/sensors installed in the data collection landscape and new sensor devices and signal data sources. Over time the approach to data collection may be standardised and old equipment replaced. Data Collection And Data Source Management This logical component acts as a local front-end to existing signal data collection/generation sources/sensors. It eliminates the need to replace existing devices. It offers a standard interface. It manages the sensor infrastructure and landscape Communications And Security This is the communications infrastructure used to securely transmit remotely collected data from local data sources/sensors and data collection units to the central facility. Data Integration This component manages the receipt of multiple data types in multiple formats from multiple data sources. Data Quality/ Summary/ Filter/ Transformation This component applies data quality algorithms to intrinsically noisy real time data to make it more usable. Data may be filtered, summarised and transformed prior to storage Data Storage This is the software component for storing in a structured manner and providing access to real time data. Data Storage Infrastructure This is the underlying data storage infrastructure. The volumes of real time data are potentially very substantial – hundreds of millions of data points per day. The data storage requirements could amount to thousands of Terabytes over several years. This components includes facilities for backup, recovery, archiving and deletion. System Management, Administration and Control This component provides facilities to manage, administer and control the overall real time system. Data Reporting and Analysis This provides reporting and analysis facilities to meet a wide variety of business requirements. External Systems Real time data can be merged with other data such as asset to provide a usage dimension to static asset data and to integrate with workforce management to manage sensor infrastructure.

Possible Approaches To Real Time Data Architecture :

Possible Approaches To Real Time Data Architecture You can expend a great deal of time, resources and money on defining the requirements of an idealised real time data architecture before embarking on any procurement and implementation OR You can research the viable products and their functionality and what other water utilities have implemented and define requirements in terms of what is realistically achievable, reducing costs and time and delivering results and learning more quickly March 8, 2016 54

Getting Real Time Data Right:

Getting Real Time Data Right Avoid false starts. Balance implementation urgency and pace with organisational readiness and maturity. Success requires change management Cannot do it all at once Real time data is an enabler and not an end in itself Embed use of real time data data in the organisation Don't turn it into a "finding the right tool" decision Recognise the interaction of governance, processes and tools that enables the organisation optimise its real time data investment Focus on value, risks and prioritisation with active engagement of key stakeholders Avoid using it solely for one-time annual budget decisions. It is about continuous alignment, tracking and benefits realisation of existing real time data systems and new projects Clarify governance and put a process in place that uses portfolio management approach to consider, make and enforce decisions March 8, 2016 55

Summary:

Summary Real time data includes Telemetry , Big Data, Smart Metering and Internet of Things Represents an idealised organisation and process breakdown across entire spectrum of real time data from strategy to workforce management to device installation, data collection and data usage and actioning Provides a basis for developing a work breakdown and an implementation plan Real time data infrastructure and systems without organisation and processes will yield few benefits Represents a comprehensive structure that an organisation can adapt to meet its long-term real time data needs Enables application and use of real time data to be embedded in the organisation March 8, 2016 56

authorStream Live Help