Tools to explore dynamics of visual search & biolo

Tools to explore dynamics of visual search & biological behavior. : 

Background reading:Long-range fractal dynamic in visual perception http://aks.rutgers.edu/AksInfo/Papers/Pubs/1_Perceptual_Dynamics/ daks@rci.rutgers.edu 2/26/07-Last update Tools to explore dynamics of visual search & biological behavior. Deborah J. Aks RU-Center for Cognitive Sciences (RuCCs) 4/17/07--Presentation for E. Sontag’s BioMath Seminar:Mathematics as Biology's New Microscope

Scale-free --> : 

Scale-free --> Rethinking what we study & measure Power laws! Size (of an event, object or behavior) Size # Typical scale = Central tendency # Small Large Many Few

Networks & relation to pdf’s : 

Networks & relation to pdf’s

Overview : 

Overview Visual search study---------------------------- Tools to study dynamics Stats, time-series analysis, FFT… Power laws Possible source(s) of (1/f) power laws:SOC, feedback & recurrent models

Visual Search in Medical images : 

Visual Search in Medical images Detecting tumors in: Mammograms x-rays, CT-scans Ultrasound MRI…

Slide 7: 

Edward J. Delp Purdue University School of Electrical and Computer Engineering; Video and Image Processing Laboratory (VIPER) The Analysis of Digital Mammograms: Spiculated Tumor Detectionand Normal Mammogram Characterization West Lafayette, Indiana, ace@ecn.purdue.edu http://bmrc.berkeley.edu/courseware/cs298/fall99/delp/berkeley99.htm http://www.ece.purdue.edu/~ace

Slide 8: 

Abnormal Markings: Spiculation or a stellate appearance Shape & contours: spicules or “arms” (increased vascularization feeding tumors) Center masses of spiculated lesions usually irregular with ill-defined borders Size: vary from a few millimeters to several centimeters Larger the tumor center, the longer its spicules ------------------------------------------------------------------------ Normal Markings Linear & smooth masses Shadow of normal ducts & connective tissue elements Approximately linear (over short segments) but often curved Diagnostic Features Abnormal & Normal Masses, Calcification..

Slide 9: 

Normal Mammograms

Slide 10: 

Human -vs- Computer (-aided) detection Which is better?Both use feature detection & classification detecting one of the three abnormal structures classifying breast lesions as benign or malignant Human advantage: Fewer false positives Superior in pattern recognition Efficient & unsystematic search patterns often are effective Computer advantage: Don’t fatigue Only biases are those built into algorithm Thorough & systematic search

Non-systematic eye-movements especially in unstructured environments : 

Engle, 1977; Ellis & Stark, 1988; Scinto & Pillalamarri, 1986; Krendel & Wodinsky, 1960; Groner & Groner, 1982 Non-systematic eye-movements especially in unstructured environments

Search in a complex environment with minimal structure : 

Search in a complex environment with minimal structure

Without structure, eye-movements appear erratic. : 

Without structure, eye-movements appear erratic.

Search in a structured environment : 

Search in a structured environment

Few saccades are needed to find the bird : 

Few saccades are needed to find the bird

QUESTIONS. : 

QUESTIONS. What guides complicated eye movements? Random or non-random process? Is there memory across fixations? Might neural interactions drive search? METHOD OF TESTING. Challenging visual search task

Visual Search Task : 

Visual Search Task T T T T T T T Find the upright “T” T T T T T T T T T T T T T T T T T T T T T T T T T T T T

Method. : 

Method. Each trial contained 81 Ts. 400 trials lasting 2.5 hours. Eight 20-minute sessions separated by 5-minute rest Generation V dual purkinje-image (DPI) tracker Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across Eye-Movements: 1/f Dynamic in Visual Search. Nonlinear Dynamics, Psychology and Life Sciences, 6 (1), 1-15. (Current Search experiments use IR eyetracker SR Research Eyelink 1000)

Map trajectory of eye scan-paths: : 

Duration & x,y coordinates for each fixation.---------------------------------------------------------- Saccadic eye-movements: Differences between fixations xn – xn+1 & yn – yn+1 Distance = (x2 + y2)1/2 Direction = Arctan (y/x). Map trajectory of eye scan-paths:

Dynamical tools : 

Dynamical tools Probability Distributions (PDFs) Power spectra (FFT)… Descriptive & Correlational Statistics

Additional tools : 

Additional tools Autocorrelation Recurrent maps Relative Dispersion (SD/M) Iterated Functions Systems (IFS) Rescaled range R/S (Hurst exponent)-- running sum of deviations from mean/SD evaluate persistence & anti-persistence

Results (from our preliminary experiment) : 

Results (from our preliminary experiment) 24 fixations per trial (on average) 5.1 seconds (SD =6.9 sec) per trial Mean fixation duration = 212 ms (SD = 89 ms) 10,215 fixations across complete search experiment. Conventional search stats… Focusing on the dynamic… What’s the central tendency?

Series of Fixation Differences : 

Series of Fixation Differences (yn+1- yn)

Scatter plot of 10,215 eye fixations for the entire visual search experiment. : 

Scatter plot of 10,215 eye fixations for the entire visual search experiment. Eye Fixations

Delay Plot of Fixations : 

yn -vs- y n+1 Delay Plot of Fixations

Scaling across 8 sessions: : 

Fixation frequency decreased from 1888 to 657 Fixation duration increased from 206 to 217 ms. Changes in fixation position … xn – xn+1 decreased yn – yn+1 increased No typical scale! Scaling across 8 sessions:

Heavy-tail distributions : 

Heavy-tail distributions Power-laws Small eye-mvmts are (very) common; large ones are rare! xn - x n+1

Heavy-tail distributions : 

Heavy-tail distributions Small eye-mvmts are common; large ones are rare! yn - y n+1

Spectral analysis Fast-Fourier Transform (FFT) : 

Power vs. Frequency Regression slope = power exponent f a f -2 = 1/ f 2 Brown noise Spectral analysis Fast-Fourier Transform (FFT)

Noisy time series : 

Noisy time series White Pink Brown

“Color’ of noise : 

White Noise Pink Noise Brown Noise 1/f 0 noise -- flat spectrum= no correlation across data points Short & Long range = 0 1/f noise --shallow slope = subtle long range correlation 1/f 2 noise-- steep slope = Predictable long-range, ‘undulating’ correlation Short range = 0 (successive events uncorrelated) “Color’ of noise

Power law indicates… : 

Power law indicates… Fractal properties: Scale-free (means ? w/ measuring resolution) Self-similar (statistically) Critical + flexible + self-organizing (1/f) Memory Steepness of the slope (on a log-log scale) reflects.. Correlation across data points = ‘Colored’ noise White Pink Brown

Power Spectra of raw fixations : 

??????? Power Spectra of raw fixations

Power Spectra of first differences across fixations : 

? = -.6 Power Spectra of first differences across fixations

Distance across eye fixations : 

(x2 + y2) 1/2 ? = -.47 Distance across eye fixations

Distance across eye fixations : 

(x2 + y2) 1/2 ? = -.47 ? = -0.3 ? = -1.8 Distance across eye fixations

Preliminary results: : 

Preliminary results: Sequence of… Absolute eye positions --> 1/f brown noise local random walk Differences & distance-across-fixations --> ~1/f pink noise Subtle long-term memory.

Ongoing experiments: : 

Ongoing experiments: Do power laws change under different conditions? Structured vs. unstructured contexts? What conditions produce 1/f pink noise? Do 1/f patterns produce more effective search?

Slide 39: 

Visual search study Power law results & implications* Possible source of 1/f results:SOC, feedback NN models

Power laws are common : 

Newman, M. (2005). Power laws, Pareto distributions and Zipf’s laws. Physics Letters, 2. Power laws are common

Network ---> Distribution : 

Network ---> Distribution Barabasi, A. & Bonabeau, E. (2003). Scale-Free Networks. Scientific American, 288, 60-69.

Networks & relation to pdf’s : 

Networks & relation to pdf’s

Slide 43: 

Scale-free networks in cell biology, Albert R., J Cell Sci. 2005 4947-57. Degree & Cluster distributions Protein interaction network (C. Elegan )

Brain network : 

Edelman, G. & Tononi, G. (2000).A Universe of Consciousness: How Matter Becomes Imagination.. Brain network

Overview of dynamical study : 

Overview of dynamical study Visual search study Dynamical tools Power law results & implications Possible source of 1/f resultsSOC,feedback & recurrent models

What may be producing ~1/f patterns? : 

? = -.6 What may be producing ~1/f patterns?

Source of 1/f dynamic? : 

(Big controversy!) Source of 1/f dynamic?

Models : 

Models ~ Hebb, 1969; Rummelhardt & McClelland, 1985; Grossberg et al, 2003Douglas, Koch, Mahowald, Martin Suarez H (1995) Beggs & Plenz(2003) Neuronal interactions ---> implicit guidance Can eye movements be described by a simple set of neuronal interaction rules (e.g., SOC) to produce 1/f behavior?

Mainzer, K. (1997). Thinking in complexity: The complex dynamics of matter, mind & mankind. Berlin: Springer. Pg. 128 : 

Mainzer, K. (1997). Thinking in complexity: The complex dynamics of matter, mind & mankind. Berlin: Springer. Pg. 128

SOC Network(Adapted from Bak, Tang, & Wiesenfeld, 1987) : 

0 4 Increasing Neural Activation ---> SOC Network(Adapted from Bak, Tang, & Wiesenfeld, 1987)

SOC_2 : 

Stimulate 1 neuron SOC_2

SOC_3 : 

Z(x,y)= initially stimulated site As individual neurons are activated beyond a threshold (of 3), activity (4) is dispersed to surrounding cells. Threshold rule: For Z(x,y) > Zcr =3 SOC_3

SOC_4 : 

Activity in the original site is depleted to zero. Z(x,y) -> Z(x,y) - 4 SOC_4

SOC_5 : 

Surrounding activity increases by 1 Z(x,y)-> Z(x,y) + 1 SOC_5

SOC_6 : 

SOC_6

SOC_7 : 

SOC_7

SOC_2EM : 

Neural SOC w/ eye movements trails SOC_2EM

SOC_3EM : 

Eye movements are pulled to the site(s) of greatest activation SOC_3EM

SOC_4EM : 

SOC_4EM

SOC_5EM : 

SOC_5EM

SOC_6EM : 

SOC_6EM

SOC_7EM : 

SOC_7EM

SOC_8EM : 

SOC_8EM

SOC_9EM : 

SOC_9EM

Simple set of SOC rules.. : 

Complex & effective search For Z(x,y) > Zcr Z(x,y) -> Z(x,y) - 4 Z(x + 1,y)-> Z(x + 1,y) + 1 Z(x,y + 1) -> Z(x,y + 1) + 1 Simple set of SOC rules.. { …can produce:

Preliminary Conclusions : 

Long-range dynamic across eye-movements! Fractal & scale-free 1/f relative eye-movements. Self-organizing & critical system--> effective search Preliminary Conclusions

Palmer, S. (1999). Vision Science: Photons to phenomenology. Boston: MIT. : 

FEF IPS LPN SC RAS Palmer, S. (1999). Vision Science: Photons to phenomenology. Boston: MIT. V1- 4

Future research : 

Future research Neural interactions --> search dynamic Behavioral & modeling approach Evaluate biologically plausible SOC & recurrent mechanisms Evaluate dynamic under different conditions (e.g., structured vs. unstructured) Does the dynamic change? How should we interpret these changes? SOC serves as a memory. SOC as a filter of white-noise.

More sources of 1/f noise.. : 

More sources of 1/f noise.. Add periodic functions Low-pass filter of white noise Combine additive & multiplicative noises Differentiate log-cumulative distributions Modified random walks; Percolation (forest-fire) models Highly optimized tolerance models Coherent noise mechanism Fragmentation Inverses of quantities Yule process See: Newman, M. (2005). Power laws, Pareto distributions & Zipf’s laws. Physics Letters, 2 Ward, L.M. (2001). Dynamical Cognitive Science. MIT Press.

Bluebird contributed by www.Sierra foothill.org : 

Bluebird contributed by www.Sierra foothill.org

http://aks.rutgers.edu/AksInfo/papers/Pubs/1_Perceptual_Dynamics/ : 

http://aks.rutgers.edu/AksInfo/papers/Pubs/1_Perceptual_Dynamics/ Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across Eye-Movements: 1/f Dynamic in Visual Search. Nonlinear Dynamics, Psychology and Life Sciences, 6 (1).