“World in Figures”: Statistical Data Dissemination via the Net with an Online Data Analyzer : “World in Figures”: Statistical Data Dissemination via the Net with an Online Data Analyzer STATISTICS: INVESTMENT IN THE FUTURE International Statistical Conference Prague, Czech Republic, 6-7 September 2004 Andras Vag – Eva Hideg


Introduction to World in Figures (WiF): Introduction to World in Figures (WiF) WiF is a statistical data integrating and disseminating project initiated by four Hungarian organizations. (see: Participants slide) The project’s basic objective is to provide comparable quantitative information for policymakers, education, research industry and the public sector. (see: Data integration (1/5) database contents slide) A multilingual Internet portal was built to provide access to millions of data records. The data accessing tool is extended with an analyzer and a simple modelling tool. WiF’s next objective is to upgrade the current version to a state-of-the-art online interactive world-model. (see: Developing plans) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Innovations: WiF is a tool to exploit statistical knowledge: Innovations: WiF is a tool to exploit statistical knowledge Access to results - with some mouse-clicks information from all fields of life, from economy to opinions; from traditional to alternative indicators; option for joint analysis of different type of data. Concept of a 21. century world-model theoretical background; first steps towards fulfilment. Data from wide scope of facts The main objective is to accelerate the analyzing process of socio-economic-environmental phenomena. That’s why the project ... processes data records from different sources; combines different contents (variables); integrates millions of statistical data records. Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


WiF’s architecture: WiF’s architecture Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Project philosophy: Project philosophy Data integration principles WiF integrates off-line stored statistical data; Data integration is solved with an intelligent tool; The result of the integration process is a single database, which can be further manipulated; Dissemination practice Direct access to the data with the help of a Java application; Free access to the data and tools; Global, country & sub-country data; Multilingual databases. Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants Project linkages Participating in international projects (e.g. EC COST A22, NewPol) Cooperations in development; Supporting education & research (e.g.BUESPA Futures).


Data integration: database contents (1/5): Data integration: database contents (1/5) About data: CONTENTS: economic, social, environmental, ecological, political, public opinion, policy, behavioral data; TIME INTERVALS: year, (optionally: quarter, month, day); DATA TYPES: frequencies, volumes, values, etc; VARIABLE TYPES: statistical variables, generated variables, and other numerical data. About databases: GLOBAL DATABASE: country level data, multiple-million of records; COUNTRY DATABASES: national, regional and local (city, county) data; SPECIAL DATABASES: content specified individually Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Data integration: about the software (2/5): Data integration: about the software (2/5) Main functions of the Slave (see Architecture): Importing statistical tables from different source files (XLS, CSV, HTML, TXT, PDF) and from the Clipboard; Importing source databases (MDB); Generating Slave databases and sending to the Master; Matching data, editing headers and the fields with a user-friendly tool, merging of source records and the Slave database. Examples for some important sub-functions: > Data Conflict Manager (i.e: Source Priority), > Equivalent Manager (to handle different wordings of the same fields from different sources); Treating “Multiple Variables” to accelerate the merging process of different variable structures. Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Data integration: Slave database feeding (3/5) : Data integration: Slave database feeding (3/5)


Data integration: Master’s functions (4/5): Data integration: Master’s functions (4/5) Main functions of the Master (see Architecture): Managing “Master databases”; Importing databases prepared by Slaves; Merging imported records from “Slave databases” and the “Master database”; Adjusting database structure; Editing the fields of “Master database”; Setting Options: > Data Conflict Manager (i.e: Source Priority), > Equivalent Manager (to handle different wordings of the same fields from different sources); > Slave Administration; Queries; Converting databases (e.g. in case of db import). Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Data integration: Main window of the Master (5/5): Data integration: Main window of the Master (5/5) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Online Statistical Data Browser: Category search tab (1/2): Online Statistical Data Browser: Category search tab (1/2) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Online Statistical Data Browser: results in 3D tables (2/2): Online Statistical Data Browser: results in 3D tables (2/2) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Online Statistical Data Analyzer: results in line chart (1/2): Online Statistical Data Analyzer: results in line chart (1/2) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Online Statistical Data Analyzer: a dialogue-panel (2/2): Online Statistical Data Analyzer: a dialogue-panel (2/2) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Online Modelling Tool: the Design and the Monitor tabs: Online Modelling Tool: the Design and the Monitor tabs Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Benchmarks: statistical data on the web (1/3): Benchmarks: statistical data on the web (1/3) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants Global data sources: e.g. UN, World Bank; European data sources: e.g. Eurostat; National statistical agencies: e.g. 40-50 agencies publish valuable data; Vertical statistical data sources: e.g. UN organizations. Number of free, highly aggregated country level data on the net: ROUGH ESTIMATION: 5-15 x 106 records (overlaps excluded) Number of records in the WiF database: 2 x 106 records


Benchmarks: online analyzers (2/3): Benchmarks: online analyzers (2/3) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants Downloadable desktop analyzers: Dataplot, Instat Plus, Scilab, Vista 5.5, Mx, R. Online running data analyzers: WebStat 3.0, Rweb, Statiscope. Comments: Biometry in the focus; Some of the analyzers are specialized to solve specific types of problems; Some others need programming knowledge; Free access. WiF’s online analyzer: Innovative and user-friendly solutions; Quite a few modules yes.


Benchmarks: online world-models (3/3) : Benchmarks: online world-models (3/3) Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants Downloadable world-models: e.g. International Futures, Globesight, IMAGE 2.2 Online running world-models: e.g. CORMAS, NetLogo Comments: New methods (e.g. multi-agent models) are implemented; Some of them are still in development phase; Widely used in education and by international organizations... But rarely in the business and at country level WiF’s online world-model: Still in embryonic stage; Its online world-model with EXTENDED WITH A STATISTICAL KNOWLEDGE BASE is unique.


Development plans: Development plans WiF’s upgrade to a 21st century world-model Adopting traditional methodological approaches; Implementing new research methods (e.g. MA models; Using extended, multi-level statistical knowledge bases. The ‘floating’ aspect The user has the option > CHOOSE THE SUBJECT OF RESEARCH and > CREATE OWN MODELS; Flexible access to the MODEL LIBRARY and to the STATISTICAL KNOWLEDGE-BASE. Networks and interactivity Data collection and research in network; Interactivity with end-users. Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Participating organizations: Participating organizations Budapest University of Economic Sciences and Public Administration, Futures Studies Centre National Academy of Sciences, Institute of Automation Heuréka Research Ltd. ATLAS Engineering Ltd. Contact: Andras Vag (Heuréka Research Ltd.) avag@worldinfigures.org Introduction Innovations Architecture Project philosophy Data integration (5) Stat. Data Browser (2) Analyzer (2) Modelling tool (2) Benchmarks (3) Development plans Participants


Slide21: Thank you.