christophe pascal slides

Neuronal basis of natural textures codingin area V4 of the awake monkey: texture analysis: 

Neuronal basis of natural textures coding in area V4 of the awake monkey: texture analysis P.Girard, C. Jouffrais, F. Arcizet, J. Bullier Insight2+ IST–2000-29688 3D shape and material properties for recognition

Aim of the study (WP3): 

Aim of the study (WP3) Coding of material properties In area V4 of awake macaque monkey Performing a visual fixation task Stimuli from the CURET database: 12 textures + 12 scrambled textures Frontal viewing direction 3 illumination directions (22.5, 45 and 67.6 deg.) 72 stimuli

Stimuli: 

Stimuli Terrycloth Sand paper Plaster Aluminum foil Salt crystals Roof shingle Plaster (zoom) Lettuce leaf Linen Concrete White bread Soleirolia plant

Experimental setup: 

Experimental setup Control of the experiment and real time analog and digital acquisition: CORTEX (courtesy of NIH) 5 independent microelectrodes (TREC) Sorting software: MSD (Alpha-Omega) Eye monitoring: IScan eye-tracker (120 Hz, 0.2 DVA)

Protocol: 

Protocol Mapping of the Receptive Field (RF) Hand-moved bars M-sequences of black and white dots Recording of response to the 72 stimuli (10 trials per stimulus) Control: 36 original textures moved 1 deg apart

Recording sites: 

Recording sites .

Database and statistics: 

Database and statistics Database: 167 cells (42 with unshaped stimuli, 98 with shaped stimuli, 27 with new set of textures) Statistics ANOVA 3-factors (Texture, Illum. Dir., Type) Population (Rank analysis, MDS, comparison V4/IT)

V4 neuron sharply selective to textures: 

V4 neuron sharply selective to textures 22.5 deg. 45 deg. 67.6 deg.

neuron selective to illumination direction: 

] ] ] ] ] ] Texture neuron selective to illumination direction Example of a V4 cell whose discharge is systematically increased for a lighting direction of 67.6 deg.

V4 neuron selective to original and “moved” textures : 

V4 neuron selective to original and “moved” textures Example of a V4 cell whose selectivity is the same for ‘original’ and ‘moved’ conditions. No response to scrambled sitmuli.

Statistics: 

Statistics 3 factors ANOVA (main effect + interaction, P<0.05) shows that: 82% of the cells are selective to textures 69% of the cells have a different response to original and random-phase textures 69% of the cells are selective to lighting direction

82% selective to texture: 

82% selective to texture

Multidimensional Scaling (MDS) – originals: 

Multidimensional Scaling (MDS) – originals MDS analysis performed on 68 cells. Original textures only, final configuration, 3 dimensions (Alienation:0.108, Stress: 0.099).

Correlations of neuronal responses with first,second,third and fourth order parameters: 

Correlations of neuronal responses with first,second,third and fourth order parameters Median luminance Rms contrast skewness kurtosis

Texture analysis: 

Texture analysis Is there a match between V4 cell population and a set of filters that could be used to classify the textures? Are there other interesting parameters that characterize the textures and are coded in V4?

Texture analysis: methodology: 

Texture analysis: methodology Sets of 2D GABOR filters (several sizes, spatial frequencies and 8 orientations (0°:22.5:157.5°) 3 different types of quantification of outputs - thresholds -energy -Spectral histograms

Example of filter and computations (thresholds): 

Example of filter and computations (thresholds) Size= 12 pixels, freq: 9.5 c/°, sigma 4 pixels, orientation 0 Size= 12 pixels, freq: 14 c/°, sigma 4 pixels, orientation 0

Example of cluster analysis with filters and neurons: 

Example of cluster analysis with filters and neurons

Example of filter and computations (energy): 

Example of filter and computations (energy)

Cluster analysis based upon energy: 

Cluster analysis based upon energy N=56 filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

MDS based upon energy: 

MDS based upon energy

Spectral Histogram: 

Spectral Histogram N=29

Spectral Histogram vs ENERGY: 

Spectral Histogram vs ENERGY energy Spectral histogram

MDS over different epochs after the stimulus onset: 

MDS over different epochs after the stimulus onset filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

MDS with images (filters/neurons): 

MDS with images (filters/neurons)

New textures: 

New textures

SNR is an important parameter: 

SNR is an important parameter Mean2/std2 (of image, not of filtered image)

Snr : 1 possible dimension: 

Snr : 1 possible dimension N=27 filters: Size 12 pixels, freq: 2 to 28 c/°, sigma 3 pixels, orientations 0:22.5:157.5°

SNR another example: 

SNR another example

Mds with images of the textures: 

Mds with images of the textures

Luminance?: 

Luminance?

Conclusions: 

Conclusions Coding of material properties in V4 and IT Is this indeed texture classification or identification? We need expert advice to use better texture characterization (Spatial frequency…) or classification (Varma and Zisserman, Geusebroek and Smeulders) Do neurons perform such expert classification? Need to use a comparable behavioural task?

Not shown: 

Not shown