Tools to explore dynamics of visual search & biological behavior. :Background reading:Long-range fractal dynamic in visual perception http://aks.rutgers.edu/AksInfo/Papers/Pubs/1_Perceptual_Dynamics/ daks@rci.rutgers.edu 2/26/07-Last update Tools to explore dynamics of visual search & biological behavior. Deborah J. Aks RU-Center for Cognitive Sciences (RuCCs) 4/17/07--Presentation for E. Sontag’s BioMath Seminar:Mathematics as Biology's New Microscope


Scale-free --> :Scale-free --> Rethinking what we study & measure Power laws! Size (of an event, object or behavior) Size # Typical scale = Central tendency # Small Large Many Few


Networks & relation to pdf’s :Networks & relation to pdf’s


Overview :Overview Visual search study---------------------------- Tools to study dynamics Stats, time-series analysis, FFT… Power laws Possible source(s) of (1/f) power laws:SOC, feedback & recurrent models


Visual Search in Medical images :Visual Search in Medical images Detecting tumors in: Mammograms x-rays, CT-scans Ultrasound MRI…


Slide 7:Edward J. Delp Purdue University School of Electrical and Computer Engineering; Video and Image Processing Laboratory (VIPER) The Analysis of Digital Mammograms: Spiculated Tumor Detectionand Normal Mammogram Characterization West Lafayette, Indiana, ace@ecn.purdue.edu http://bmrc.berkeley.edu/courseware/cs298/fall99/delp/berkeley99.htm http://www.ece.purdue.edu/~ace


Slide 8:Abnormal Markings: Spiculation or a stellate appearance Shape & contours: spicules or “arms” (increased vascularization feeding tumors) Center masses of spiculated lesions usually irregular with ill-defined borders Size: vary from a few millimeters to several centimeters Larger the tumor center, the longer its spicules ------------------------------------------------------------------------ Normal Markings Linear & smooth masses Shadow of normal ducts & connective tissue elements Approximately linear (over short segments) but often curved Diagnostic Features Abnormal & Normal Masses, Calcification..


Slide 9:Normal Mammograms


Slide 10:Human -vs- Computer (-aided) detection Which is better?Both use feature detection & classification detecting one of the three abnormal structures classifying breast lesions as benign or malignant Human advantage: Fewer false positives Superior in pattern recognition Efficient & unsystematic search patterns often are effective Computer advantage: Don’t fatigue Only biases are those built into algorithm Thorough & systematic search


Non-systematic eye-movements especially in unstructured environments :Engle, 1977; Ellis & Stark, 1988; Scinto & Pillalamarri, 1986; Krendel & Wodinsky, 1960; Groner & Groner, 1982 Non-systematic eye-movements especially in unstructured environments


Search in a complex environment with minimal structure :Search in a complex environment with minimal structure


Without structure, eye-movements appear erratic. :Without structure, eye-movements appear erratic.


Search in a structured environment :Search in a structured environment


Few saccades are needed to find the bird :Few saccades are needed to find the bird


QUESTIONS. :QUESTIONS. What guides complicated eye movements? Random or non-random process? Is there memory across fixations? Might neural interactions drive search? METHOD OF TESTING. Challenging visual search task


Visual Search Task :Visual Search Task T T T T T T T Find the upright “T” T T T T T T T T T T T T T T T T T T T T T T T T T T T T


Method. :Method. Each trial contained 81 Ts. 400 trials lasting 2.5 hours. Eight 20-minute sessions separated by 5-minute rest Generation V dual purkinje-image (DPI) tracker Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across Eye-Movements: 1/f Dynamic in Visual Search. Nonlinear Dynamics, Psychology and Life Sciences, 6 (1), 1-15. (Current Search experiments use IR eyetracker SR Research Eyelink 1000)


Map trajectory of eye scan-paths: :Duration & x,y coordinates for each fixation.---------------------------------------------------------- Saccadic eye-movements: Differences between fixations xn – xn+1 & yn – yn+1 Distance = (x2 + y2)1/2 Direction = Arctan (y/x). Map trajectory of eye scan-paths:


Dynamical tools :Dynamical tools Probability Distributions (PDFs) Power spectra (FFT)… Descriptive & Correlational Statistics


Additional tools :Additional tools Autocorrelation Recurrent maps Relative Dispersion (SD/M) Iterated Functions Systems (IFS) Rescaled range R/S (Hurst exponent)-- running sum of deviations from mean/SD evaluate persistence & anti-persistence


Results (from our preliminary experiment) :Results (from our preliminary experiment) 24 fixations per trial (on average) 5.1 seconds (SD =6.9 sec) per trial Mean fixation duration = 212 ms (SD = 89 ms) 10,215 fixations across complete search experiment. Conventional search stats… Focusing on the dynamic… What’s the central tendency?


Series of Fixation Differences :Series of Fixation Differences (yn+1- yn)


Scatter plot of 10,215 eye fixations for the entire visual search experiment. :Scatter plot of 10,215 eye fixations for the entire visual search experiment. Eye Fixations


Delay Plot of Fixations :yn -vs- y n+1 Delay Plot of Fixations


Scaling across 8 sessions: :Fixation frequency decreased from 1888 to 657 Fixation duration increased from 206 to 217 ms. Changes in fixation position … xn – xn+1 decreased yn – yn+1 increased No typical scale! Scaling across 8 sessions:


Heavy-tail distributions :Heavy-tail distributions Power-laws Small eye-mvmts are (very) common; large ones are rare! xn - x n+1


Heavy-tail distributions :Heavy-tail distributions Small eye-mvmts are common; large ones are rare! yn - y n+1


Spectral analysis Fast-Fourier Transform (FFT) : Power vs. Frequency Regression slope = power exponent f a f -2 = 1/ f 2 Brown noise Spectral analysis Fast-Fourier Transform (FFT)


Noisy time series :Noisy time series White Pink Brown


“Color’ of noise :White Noise Pink Noise Brown Noise 1/f 0 noise -- flat spectrum= no correlation across data points Short & Long range = 0 1/f noise --shallow slope = subtle long range correlation 1/f 2 noise-- steep slope = Predictable long-range, ‘undulating’ correlation Short range = 0 (successive events uncorrelated) “Color’ of noise


Power law indicates… :Power law indicates… Fractal properties: Scale-free (means ? w/ measuring resolution) Self-similar (statistically) Critical + flexible + self-organizing (1/f) Memory Steepness of the slope (on a log-log scale) reflects.. Correlation across data points = ‘Colored’ noise White Pink Brown


Power Spectra of raw fixations :??????? Power Spectra of raw fixations


Power Spectra of first differences across fixations :? = -.6 Power Spectra of first differences across fixations


Distance across eye fixations :(x2 + y2) 1/2 ? = -.47 Distance across eye fixations


Distance across eye fixations :(x2 + y2) 1/2 ? = -.47 ? = -0.3 ? = -1.8 Distance across eye fixations


Preliminary results: :Preliminary results: Sequence of… Absolute eye positions --> 1/f brown noise local random walk Differences & distance-across-fixations --> ~1/f pink noise Subtle long-term memory.


Ongoing experiments: :Ongoing experiments: Do power laws change under different conditions? Structured vs. unstructured contexts? What conditions produce 1/f pink noise? Do 1/f patterns produce more effective search?


Slide 39:Visual search study Power law results & implications* Possible source of 1/f results:SOC, feedback NN models


Power laws are common :Newman, M. (2005). Power laws, Pareto distributions and Zipf’s laws. Physics Letters, 2. Power laws are common


Network ---> Distribution :Network ---> Distribution Barabasi, A. & Bonabeau, E. (2003). Scale-Free Networks. Scientific American, 288, 60-69.


Networks & relation to pdf’s :Networks & relation to pdf’s


Slide 43:Scale-free networks in cell biology, Albert R., J Cell Sci. 2005 4947-57. Degree & Cluster distributions Protein interaction network (C. Elegan )


Brain network :Edelman, G. & Tononi, G. (2000).A Universe of Consciousness: How Matter Becomes Imagination.. Brain network


Overview of dynamical study :Overview of dynamical study Visual search study Dynamical tools Power law results & implications Possible source of 1/f resultsSOC,feedback & recurrent models


What may be producing ~1/f patterns? :? = -.6 What may be producing ~1/f patterns?


Source of 1/f dynamic? : (Big controversy!) Source of 1/f dynamic?


Models :Models ~ Hebb, 1969; Rummelhardt & McClelland, 1985; Grossberg et al, 2003Douglas, Koch, Mahowald, Martin Suarez H (1995) Beggs & Plenz(2003) Neuronal interactions ---> implicit guidance Can eye movements be described by a simple set of neuronal interaction rules (e.g., SOC) to produce 1/f behavior?


Mainzer, K. (1997). Thinking in complexity: The complex dynamics of matter, mind & mankind. Berlin: Springer. Pg. 128 :Mainzer, K. (1997). Thinking in complexity: The complex dynamics of matter, mind & mankind. Berlin: Springer. Pg. 128


SOC Network(Adapted from Bak, Tang, & Wiesenfeld, 1987) :0 4 Increasing Neural Activation ---> SOC Network(Adapted from Bak, Tang, & Wiesenfeld, 1987)


SOC_2 :Stimulate 1 neuron SOC_2


SOC_3 :Z(x,y)= initially stimulated site As individual neurons are activated beyond a threshold (of 3), activity (4) is dispersed to surrounding cells. Threshold rule: For Z(x,y) > Zcr =3 SOC_3


SOC_4 :Activity in the original site is depleted to zero. Z(x,y) -> Z(x,y) - 4 SOC_4


SOC_5 :Surrounding activity increases by 1 Z(x,y)-> Z(x,y) + 1 SOC_5


SOC_6 :SOC_6


SOC_7 :SOC_7


SOC_2EM :Neural SOC w/ eye movements trails SOC_2EM


SOC_3EM :Eye movements are pulled to the site(s) of greatest activation SOC_3EM


SOC_4EM :SOC_4EM


SOC_5EM :SOC_5EM


SOC_6EM :SOC_6EM


SOC_7EM :SOC_7EM


SOC_8EM :SOC_8EM


SOC_9EM :SOC_9EM


Simple set of SOC rules.. :Complex & effective search For Z(x,y) > Zcr Z(x,y) -> Z(x,y) - 4 Z(x + 1,y)-> Z(x + 1,y) + 1 Z(x,y + 1) -> Z(x,y + 1) + 1 Simple set of SOC rules.. { …can produce:


Preliminary Conclusions :Long-range dynamic across eye-movements! Fractal & scale-free 1/f relative eye-movements. Self-organizing & critical system--> effective search Preliminary Conclusions


Palmer, S. (1999). Vision Science: Photons to phenomenology. Boston: MIT. :FEF IPS LPN SC RAS Palmer, S. (1999). Vision Science: Photons to phenomenology. Boston: MIT. V1- 4


Future research :Future research Neural interactions --> search dynamic Behavioral & modeling approach Evaluate biologically plausible SOC & recurrent mechanisms Evaluate dynamic under different conditions (e.g., structured vs. unstructured) Does the dynamic change? How should we interpret these changes? SOC serves as a memory. SOC as a filter of white-noise.


More sources of 1/f noise.. :More sources of 1/f noise.. Add periodic functions Low-pass filter of white noise Combine additive & multiplicative noises Differentiate log-cumulative distributions Modified random walks; Percolation (forest-fire) models Highly optimized tolerance models Coherent noise mechanism Fragmentation Inverses of quantities Yule process See: Newman, M. (2005). Power laws, Pareto distributions & Zipf’s laws. Physics Letters, 2 Ward, L.M. (2001). Dynamical Cognitive Science. MIT Press.


Bluebird contributed by www.Sierra foothill.org :Bluebird contributed by www.Sierra foothill.org


http://aks.rutgers.edu/AksInfo/papers/Pubs/1_Perceptual_Dynamics/ :http://aks.rutgers.edu/AksInfo/papers/Pubs/1_Perceptual_Dynamics/ Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across Eye-Movements: 1/f Dynamic in Visual Search. Nonlinear Dynamics, Psychology and Life Sciences, 6 (1).