RICH Pilot

Slide1: 

Pilot presentation RICH

Introduction: 

Introduction RICH = Reading Images for the Cultural Heritage Two pilot projects Application for semi-automatic dendrochronology Content-based image retrieval for historical glass Today, focus on the latter project

Introduction: 

Introduction Classification of archaeological artefacts Now performed manually by experts Expert compares artefact with objects from reference collection Reference collections consist of drawings in books Thus: slow, subjective, and error-prone process

Example: 

Example An archaeological artefact:

Example: 

Example And its corresponding drawing: 

The task: 

The task Given an artefact photograph Find the most ‘similar’ drawings Speeds up the classification process Can give archaeological experts new insights

The problem: 

The problem Drawings contain no color information Drawings contain only very abstract texture representation Texture hard to extract from glass photographs Thus: only outer shape information is accurate Shape matching

Shape matching: 

Shape matching Several approaches in literature: Shape contexts (Belongie, 2000) Curvature scale spaces (Mokhtarian, 1996) Turning functions (Tanase, 2003) Dynamic programming (Petrakis, 2002) Moment invariants (Hu, 1962) Hausdorff, Procrustes, etc. MPEG-7 standard: Curvature scale spaces

Shape matching: 

Shape matching For today, focus on: Shape contexts Curvature scale space Desired properties of approach: Invariant to scale, translation, and rotation Robust to distortions due to Broken artefacts 3D rotations Drawing interpretations

Broken artefacts: 

Broken artefacts 

Or worse…: 

Or worse…

3D rotations: 

3D rotations 

Shape contexts: 

Shape contexts Sample points from outer contour For all points: Compute angle (relative to baseline) and distance to all other points in coarsely discretized log-polar space All resulting histograms form the shape representation

Shape contexts: 

Shape contexts Matching using startpoint invariant k-NN classifier (using χ2-distance) Startpoint invariance obtained by circular shifting one of the histograms

Curvature scale space: 

Curvature scale space Determine positions of zero-crossings of curvature for an ‘evolving’ shape contour Curvature is a function that is 0 for a straight line, and 1 / r for a circle with radius r Shape evolution: convolve coordinates with 1D Gaussian kernel with increasing variance

Shape evolution: 

Shape evolution Evolving shape with curvature zero-crossings:

Curvature scale space: 

Curvature scale space CSS image: Align CSS by aligning global maximum Sum distances between main peaks

Results: 

Results Low identification performance (as expected) due to difficult dataset

Results: 

Results We examined various variations, such as quantization in shape context space, etc. Best performance: 33% for hitlist size 10

Results: 

Results However: For ‘good’ artefacts results are encouraging Shape analysis on reference collection allows for making ‘shape maps’ (using MDS) This allows for archaeologists to create new typologies (since typologies are not ‘fixed’) Allows for new methods of presenting collections Good results expected for flint data

Example query: 

Example query 

Applications: 

Applications Matlab application (local)

Applications: 

Applications Navigation structure for collection presentation Precalculated and stored in database http://www.referentiecollectie.nl/richglas

Applications: 

Applications Web-based CBIR tool Servlet running on webserver (using Tomcat) User uploads photograph to webserver Sends query image to UM calculation server (using RMI) RMI server executes original Matlab-scripts (using JMatLink) Results are sent back to servlet Servlet generates result pages Advantages: no local calculations, no porting of code

Conclusions: 

Conclusions Matching glass artefacts with drawings is a difficult problem Shape context matching outperforms (MPEG-7 standard) CSS matching Allows for shape analysis of reference collection Number of (preliminary) applications delivered We expect shape matching to be usefull for flint data Possible improvements: Incorporating texture features Design of shape matching methods for partial shape matching using closed contours

Questions: 

Questions ?