Metadata Generation and Retrieval of Geographic Imagery: Metadata Generation and Retrieval of Geographic Imagery Aidong Zhang Director, Multimedia and Database Laboratory Computer Science and Engineering University at Buffalo 05.10. 2001


Introduction: Introduction Observations: USGS, NIMA and NASA provide the archiving of large repositories of remote-sensing data. New Issues: problem of resource selection. Given a query, where should a user start a search? Our Approach: Design a metaserver on top of various visual databases. Given a query, the metaserver first produces a ranking of the databse sites and then distributes the queries to the selected databases.


Distributed System Architecture: Distributed System Architecture GIS Database GIS Database GIS Database GIS Database Client Browser Client Browser Client Browser Client Browser Metasearch Agent Meta Database Metaserver Query Manager Metaserver (Our focus) GIS database at remote sites Client applications for visual display


Slide4: Local Severs/DB Local Severs/DB Ranked DB List 1.GIS-SANF Server/DB 2.GIS-1999 Server/DB …… 7.GIS-FLOR Server/DB Step 1 Step 2 GIS1998 Server/DB GIS1999 Server/DB GIS2000 Server/DB GISWNY Server/DB GIS-SANF Server/DB GIS-FLOR Server/DB GIS-FLOR2 Server/DB Matching Images Users


Global View of Data Sources: Global View of Data Sources METADATABASE Color Texture Shape Feature Classes Templates Database Sites DB1 DB2 DBn


Generating Templates: Generating Templates Images are clustered and the centroids of the clusters are chosen as templates. Environment Residential Water Grass Agriculture


Metadatabase: Templates of local databases are collected in the metadatabase to represent the content of the databases Statistical data: We can measure the similarity of images in the databases to the templates. Using these similarity measurements, statistical data are computed that capture the likelihood of a database containing data that are relevant to a template. The relevant databases for a given query can be selected by determining the similarity of the query with metadatabase templates and ranking the database sites based on the visual relationships recorded between the databases and templates. Metadatabase


Keyblock Approach: Keyblock Approach Generalizing text retrieval techniques to image retrieval Text IR: use keywords to index and retrieve What are the 'keywords' of an image? Region segments of images Features of images Objects of images How to generate 'keywords' of images? Keyblocks: select centroids of clusters


Slide9: Image Database Sampling Training Blocks Feature-based Clustering (GLA,PNNA,etc.) Codebook Query Image Image Encoding Keyblock Generation Content-based Image Retrieval Feature Representation: BM, VM, HM, etc. Query and Retrieval Training Set


Slide10: Training Images Water Codebook Keyblock Generation Training Images Keyblock Generation Forest Codebook (Water) (Forest) Stage I Merge Codebooks Stage II Stage III LVQ-based Fine Tuning Knowledge-based Keyblock Generation


Slide11: …... Codebook ( a list of keyblocks) Original Image Segmentation Table Lookup Segmented Image Encoded Image 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Block Encoding Image Decoding Reconstructed Image 16 18 16 16 15 15 18 18 19 19 19 16 16 16 18 18


A raw image and the reconstructed images with different codebooks: A raw image and the reconstructed images with different codebooks


An Example Query: An Example Query


Conclusion: Conclusion We established a framework for browsing and navigating geographic images We use effective metadata representation and management for integration of multiple data sources and provide efficient access to the data sources. We developed a keyblock-based approach to generalize the text-based IR techniques to geographic image retrieval.