Tools to explore dynamics of visual search & biological behavior. : Background reading:Long-range fractal dynamic in visual perception
http://aks.rutgers.edu/AksInfo/Papers/Pubs/1_Perceptual_Dynamics/
[email protected] 2/26/07-Last update Tools to explore dynamics of visual search & biological behavior. Deborah J. Aks RU-Center for Cognitive Sciences (RuCCs)
4/17/07--Presentation for E. Sontag’s BioMath Seminar:Mathematics as Biology's New Microscope
Scale-free --> : Scale-free --> Rethinking what we study & measure Power laws! Size (of an event, object or behavior) Size # Typical scale =
Central tendency # Small Large Many Few
Networks & relation to pdf’s : Networks & relation to pdf’s
Overview : Overview Visual search study----------------------------
Tools to study dynamics
Stats, time-series analysis, FFT…
Power laws
Possible source(s) of (1/f) power laws:SOC, feedback & recurrent models
Visual Search in Medical images : Visual Search in Medical images Detecting tumors in:
Mammograms x-rays, CT-scans
Ultrasound
MRI… Slide 7: Edward J. Delp Purdue University School of Electrical and Computer Engineering; Video and Image Processing Laboratory (VIPER)
The Analysis of Digital Mammograms: Spiculated Tumor Detectionand Normal Mammogram Characterization
West Lafayette, Indiana,
[email protected] http://bmrc.berkeley.edu/courseware/cs298/fall99/delp/berkeley99.htm
http://www.ece.purdue.edu/~ace
Slide 8: Abnormal Markings:
Spiculation or a stellate appearance
Shape & contours: spicules or “arms” (increased vascularization feeding tumors)
Center masses of spiculated lesions usually irregular
with ill-defined borders
Size: vary from a few millimeters to several centimeters
Larger the tumor center, the longer its spicules
------------------------------------------------------------------------
Normal Markings
Linear & smooth masses
Shadow of normal ducts & connective tissue elements
Approximately linear (over short segments) but often curved
Diagnostic Features Abnormal & Normal Masses, Calcification..
Slide 9: Normal Mammograms
Slide 10: Human -vs- Computer (-aided) detection Which is better?Both use feature detection & classification
detecting one of the three abnormal structures
classifying breast lesions as benign or malignant Human advantage:
Fewer false positives
Superior in pattern recognition
Efficient & unsystematic search patterns often are effective Computer advantage:
Don’t fatigue
Only biases are those built into algorithm
Thorough & systematic search
Non-systematic eye-movements especially in unstructured environments : Engle, 1977; Ellis & Stark, 1988;
Scinto & Pillalamarri, 1986;
Krendel & Wodinsky, 1960; Groner & Groner, 1982 Non-systematic eye-movements especially in unstructured environments
Search in a complex environment with minimal structure : Search in a complex environment with minimal structure
Without structure, eye-movements appear erratic. : Without structure, eye-movements appear erratic.
Search in a structured environment : Search in a structured environment
Few saccades are needed to find the bird : Few saccades are needed to find the bird
QUESTIONS. : QUESTIONS. What guides complicated eye movements?
Random or non-random process?
Is there memory across fixations?
Might neural interactions drive search? METHOD OF TESTING.
Challenging visual search task
Visual Search Task : Visual Search Task T T T T T T T Find the upright “T” T T T T T T T T T T T T T T T T T T T T T T T T T T T T
Method. : Method. Each trial contained 81 Ts.
400 trials lasting 2.5 hours.
Eight 20-minute sessions separated by 5-minute rest
Generation V dual purkinje-image (DPI) tracker Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across Eye-Movements: 1/f Dynamic in Visual Search. Nonlinear Dynamics, Psychology and Life Sciences, 6 (1), 1-15. (Current Search experiments use IR eyetracker SR Research Eyelink 1000)
Map trajectory of eye scan-paths: : Duration & x,y coordinates for each fixation.----------------------------------------------------------
Saccadic eye-movements:
Differences between fixations
xn – xn+1 & yn – yn+1
Distance = (x2 + y2)1/2
Direction = Arctan (y/x). Map trajectory of eye scan-paths:
Dynamical tools : Dynamical tools Probability Distributions (PDFs)
Power spectra (FFT)… Descriptive & Correlational Statistics
Additional tools : Additional tools Autocorrelation
Recurrent maps
Relative Dispersion (SD/M)
Iterated Functions Systems (IFS)
Rescaled range R/S (Hurst exponent)-- running sum of deviations from mean/SD
evaluate persistence & anti-persistence
Results (from our preliminary experiment) : Results (from our preliminary experiment) 24 fixations per trial (on average) 5.1 seconds (SD =6.9 sec) per trial Mean fixation duration = 212 ms (SD = 89 ms) 10,215 fixations across complete search experiment. Conventional search stats… Focusing on the dynamic… What’s the central tendency?
Series of Fixation Differences : Series of Fixation Differences (yn+1- yn)
Scatter plot of 10,215 eye fixations for the entire visual search experiment. : Scatter plot of 10,215 eye fixations for the entire visual search experiment. Eye Fixations
Delay Plot of Fixations : yn -vs- y n+1 Delay Plot of Fixations
Scaling across 8 sessions: : Fixation frequency decreased from 1888 to 657 Fixation duration increased from 206 to 217 ms. Changes in fixation position …
xn – xn+1 decreased
yn – yn+1 increased No typical scale! Scaling across 8 sessions:
Heavy-tail distributions : Heavy-tail distributions Power-laws
Small eye-mvmts are (very) common; large ones are rare! xn - x n+1
Heavy-tail distributions : Heavy-tail distributions Small eye-mvmts are common; large ones are rare! yn - y n+1
Spectral analysis Fast-Fourier Transform (FFT) : Power vs. Frequency
Regression slope = power exponent
f a f -2 = 1/ f 2
Brown noise Spectral analysis Fast-Fourier Transform (FFT)
Noisy time series : Noisy time series White Pink Brown
“Color’ of noise : White Noise Pink Noise Brown Noise 1/f 0 noise -- flat spectrum= no correlation across data points
Short & Long range = 0 1/f noise --shallow slope = subtle long range correlation 1/f 2 noise-- steep slope =
Predictable long-range, ‘undulating’ correlation
Short range = 0 (successive events uncorrelated) “Color’ of noise
Power law indicates… : Power law indicates… Fractal properties:
Scale-free (means ? w/ measuring resolution)
Self-similar (statistically)
Critical + flexible + self-organizing (1/f) Memory
Steepness of the slope (on a log-log scale) reflects..
Correlation across data points = ‘Colored’ noise
White
Pink
Brown
Power Spectra of raw fixations : ??????? Power Spectra of raw fixations
Power Spectra of first differences across fixations : ? = -.6 Power Spectra of first differences across fixations
Distance across eye fixations : (x2 + y2) 1/2 ? = -.47 Distance across eye fixations
Distance across eye fixations : (x2 + y2) 1/2 ? = -.47 ? = -0.3 ? = -1.8 Distance across eye fixations
Preliminary results: : Preliminary results: Sequence of…
Absolute eye positions --> 1/f brown noise
local random walk
Differences & distance-across-fixations --> ~1/f pink noise
Subtle long-term memory.
Ongoing experiments: : Ongoing experiments: Do power laws change under different conditions?
Structured vs. unstructured contexts?
What conditions produce 1/f pink noise?
Do 1/f patterns produce more effective search?
Slide 39: Visual search study
Power law results & implications*
Possible source of 1/f results:SOC, feedback NN models
Power laws are common : Newman, M. (2005). Power laws, Pareto distributions
and Zipf’s laws. Physics Letters, 2. Power laws are common
Network ---> Distribution : Network ---> Distribution Barabasi, A. & Bonabeau, E. (2003). Scale-Free Networks.
Scientific American, 288, 60-69.
Networks & relation to pdf’s : Networks & relation to pdf’s
Slide 43: Scale-free networks in cell biology, Albert R., J Cell Sci. 2005 4947-57. Degree & Cluster distributions Protein interaction network (C. Elegan )
Brain network : Edelman, G. & Tononi, G. (2000).A Universe of Consciousness: How Matter Becomes Imagination.. Brain network
Overview of dynamical study : Overview of dynamical study Visual search study
Dynamical tools
Power law results & implications
Possible source of 1/f resultsSOC,feedback & recurrent models
What may be producing ~1/f patterns? : ? = -.6 What may be producing ~1/f patterns?
Source of 1/f dynamic? : (Big controversy!) Source of 1/f dynamic?
Models : Models ~ Hebb, 1969; Rummelhardt & McClelland, 1985; Grossberg et al, 2003Douglas, Koch, Mahowald, Martin Suarez H (1995) Beggs & Plenz(2003)
Neuronal interactions ---> implicit guidance Can eye movements be described by a simple set of
neuronal interaction rules (e.g., SOC) to produce 1/f behavior?
Mainzer, K. (1997). Thinking in complexity: The complex dynamics of matter, mind & mankind. Berlin: Springer. Pg. 128 : Mainzer, K. (1997). Thinking in complexity: The complex dynamics of matter, mind & mankind. Berlin: Springer. Pg. 128
SOC Network(Adapted from Bak, Tang, & Wiesenfeld, 1987) : 0 4 Increasing Neural Activation ---> SOC Network(Adapted from Bak, Tang, & Wiesenfeld, 1987)
SOC_2 : Stimulate 1 neuron SOC_2
SOC_3 : Z(x,y)= initially stimulated site As individual neurons are activated beyond a threshold (of 3), activity (4) is dispersed to surrounding cells. Threshold rule: For Z(x,y) > Zcr =3 SOC_3
SOC_4 : Activity in the original site is depleted to zero. Z(x,y) -> Z(x,y) - 4 SOC_4
SOC_5 : Surrounding activity increases by 1 Z(x,y)-> Z(x,y) + 1 SOC_5
SOC_6 : SOC_6
SOC_7 : SOC_7
SOC_2EM : Neural SOC w/ eye movements trails SOC_2EM
SOC_3EM : Eye movements are pulled to the site(s) of greatest activation SOC_3EM
SOC_4EM : SOC_4EM
SOC_5EM : SOC_5EM
SOC_6EM : SOC_6EM
SOC_7EM : SOC_7EM
SOC_8EM : SOC_8EM
SOC_9EM : SOC_9EM
Simple set of SOC rules.. : Complex & effective search For Z(x,y) > Zcr
Z(x,y) -> Z(x,y) - 4
Z(x + 1,y)-> Z(x + 1,y) + 1
Z(x,y + 1) -> Z(x,y + 1) + 1 Simple set of SOC rules.. { …can produce:
Preliminary Conclusions : Long-range dynamic across eye-movements!
Fractal & scale-free 1/f relative eye-movements. Self-organizing & critical system--> effective search Preliminary Conclusions
Palmer, S. (1999). Vision Science: Photons to phenomenology. Boston: MIT. : FEF IPS LPN SC RAS Palmer, S. (1999). Vision Science: Photons to phenomenology. Boston: MIT. V1- 4
Future research : Future research Neural interactions --> search dynamic Behavioral & modeling approach
Evaluate biologically plausible SOC & recurrent mechanisms
Evaluate dynamic under different conditions
(e.g., structured vs. unstructured)
Does the dynamic change?
How should we interpret these changes?
SOC serves as a memory.
SOC as a filter of white-noise.
More sources of 1/f noise.. : More sources of 1/f noise.. Add periodic functions
Low-pass filter of white noise
Combine additive & multiplicative noises
Differentiate log-cumulative distributions
Modified random walks; Percolation (forest-fire) models
Highly optimized tolerance models
Coherent noise mechanism
Fragmentation
Inverses of quantities
Yule process
See:
Newman, M. (2005). Power laws, Pareto distributions & Zipf’s laws. Physics Letters, 2
Ward, L.M. (2001). Dynamical Cognitive Science. MIT Press.
Bluebird contributed by www.Sierra foothill.org : Bluebird contributed by www.Sierra foothill.org
http://aks.rutgers.edu/AksInfo/papers/Pubs/1_Perceptual_Dynamics/ : http://aks.rutgers.edu/AksInfo/papers/Pubs/1_Perceptual_Dynamics/ Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across Eye-Movements: 1/f Dynamic in Visual Search. Nonlinear Dynamics, Psychology and Life Sciences, 6 (1).