SEEGRID2 WP4 TR 021 SDA Demo a 2007 03 12

Kandilli Earthquake Seismic Data Server and Analysis / SDA : 

Kandilli Earthquake Seismic Data Server and Analysis / SDA M. Yılmazer Kandilli Observatory and Earthquake Research Institute, Boğaziçi Üniversity, Istanbul, Turkey D. Unat, A. S. Balkır, E. Şenay, C. Özturan Computer Eng. Dept. Boğaziçi University, Istanbul, Turkey

Some Facts About the Collected Data : 

Some Facts About the Collected Data SDA is developed by Bogazici University and Kandilli Earthquake research center. Generally researchers that work in the field of earthquake research downloads the data from web repositories by selecting a group of sensors and time region. Main motivation of the project for researchers was to run the application near the data, not to bring huge amounts of data from web servers. SDA is a project to enable grid infrastructure for seismic data analysis applications and to gain expertise on this field.

Some Facts About the Collected Data : 

Some Facts About the Collected Data Data is collected from over 100 stations in Turkey Data comes in compressed SAC file format Data is transferred hourly from the stations to Kandilli Data is available on Kandilli web servers Size of compressed SAC file that is generated hourly from each station is roughly 20K-580K Size of compressed data collected monthly is roughly: 32 GB Size of 10 year data is : 3.8 TB

Seismic Data Collection Stations in Turkey: 

Seismic Data Collection Stations in Turkey

Project Motivation: 

Project Motivation It is impractical to download all this data via web in order to process on a local computer Grid can provide both the massive data as well as the computational power to process it Web server download Local computer Grid Approach Data + Computational Power Web approach

Project Components: 

Project Components

1.Tr-Grid Kandilli Seismic Data Storage : 

1.Tr-Grid Kandilli Seismic Data Storage Seismic data at http://barbar.koeri.boun.edu.tr/sismo/zKDRS/zzTRlogin.asp is mirrored to se.ulakbim.gov.tr storage element and registered to lfc.ulakbim.gov.tr file catalog LFC_HOME is : /grid/trgrid/kandilli/barbar.koeri.boun.edu.tr/wData/

1.Tr-Grid Kandilli Seismic Data Repository: 

1.Tr-Grid Kandilli Seismic Data Repository Mirroring Failures Possible Problems on Kandilli side: Some sites do not send its data on time Data has not been uploaded to web site, it is not ready Temporary breakdown in the Kandilli's web page Delay due to the special cases (ex: the weekends and holidays) Problems on the Grid side: The creation of proxy certificate may fail Server may be shutdown Timeout can occur in copy & register due to network congestion Ways to solve problems: Make a list of the files that faced download problems After completing current hour's data, go back to the 5 days before and check the same hour's data Every week on Wednesday after midnight,    Scan the whole week, if missing any, copy & register

2. C++ SAC File Iterators : 

2. C++ SAC File Iterators There are hundred thousands SAC files High level programming interface is needed in order to iterate over these files esenay@levrek:~$ lfc-ls -l /grid/trgridd/kandilli/barbar.koeri.boun.edu.tr/wData/2006/04/11/23 -rw-rw-r-- 1 103 110 256478 Apr 12 2006 2006041123.AKS.SHZ.KO.zip -rw-rw-r-- 1 103 110 158038 Apr 12 2006 2006041123.ALT.SHZ.KO.zip ....... ....... -rw-rw-r-- 1 103 110 17984 Apr 12 2006 2006041123.YOZ.SHZ.KO.zip C++ classes and iterators are defined for accessing and iterating over files In order to speed up file access, pipelining techniques are used while iterating over files

2. C++ SAC File Iterators : 

2. C++ SAC File Iterators Iterators for stations and time intervals are provided Example: #include <kandilli> ..... Kandilli K ; KandilliStation station ; for(station = K.begin() ; station != K.end() : station++ ) { ....... }

2. C++ SAC File Iterators : 

2. C++ SAC File Iterators How to improve file access performance From various tests on se.ulakbim.gov.tr Latency = 0.0181 sec, Bandwidth = 30.864 MB/sec

2. C++ SAC File Iterators : 

2. C++ SAC File Iterators From various tests on se.ulakbim.gov.tr

2. C++ SAC File Iterators : 

2. C++ SAC File Iterators File access performance is improved by using multiple threads arranged in pipeline fashion Original loop Get File Process File Multi-threaded Loop

2. C++ SAC File Iterators : 

2. C++ SAC File Iterators Pipelined file access

3. Example Application : 

3. Example Application There are many seismic data analysis applications that can use TR-Grid Kandilli seismic data repository and benefit from C++ iterators. Application selected to demonstrate was earthquake epicenter locator which runs automatically from hourly data A webpage was created to: To display seismic data stations, To query earthquakes

3. Displaying seismic data collection station information : 

3. Displaying seismic data collection station information

3. Querying earthquakes : 

3. Querying earthquakes

3. Displaying earthquakes that happened in the queried interval : 

3. Displaying earthquakes that happened in the queried interval