ECSE-6963Biological Image Analysis :ECSE-6963Biological Image Analysis Lecture #8: Nuclear Medicine, Image Pre-Processing Badri Roysam Rensselaer Polytechnic Institute, Troy, New York 12180. Center for Sub-Surface Imaging & Sensing


Recap: CT vs. MRI :Recap: CT vs. MRI CT Cheap & Fast Good resolution with bone Hard to distinguish soft tissues without contrast agent Can’t distinguish atoms beyond their X-Ray cross-section X-Rays harmful to body MRI Expensive & Slow Can distinguish bone and various soft tissues Can distinguish specific atoms No known health hazards to MR imaging Avoidable injury hazard if magnetic objects in/around body


Main advantages of MRI :Main advantages of MRI Structural & Functional Imaging Possible Differentiation between various kinds of soft tissue. X-rays pass through soft tissue without much absorption High sensitivity to early pathological changes makes early detection possible. Studies of blood vessels and flow without use of contrast just oxygen level of blood gives contrast 3-D, allowing Multi-planar display i.e. axial, sagittal, coronal, and oblique. Multi-channel output Enables better segmentation No known biological hazards Magnetic fields don’t ionize, unlike X-rays Exceptions: people with pacemakers and/or implanted metallic objects can’t be imaged safely


Nuclear Medicine :Nuclear Medicine Basic Idea: Inject patient with radio-isotope labeled substance (tracer) Chemically the same, but physically different Detect the radioactive emissions (gamma rays) Super-short wavelength But, can’t achieve the implied high resolution Detection technology limitations Not enough photons! Use filtered back-projection to reconstruct the 3-D image Like fluorescence microscopy, except we don’t need excitation


SPECT & PET :SPECT & PET Major Functional imaging tools SPECT: Single-photon Emission Computed Tomography cheap and low-resolution Tells us where blood is flowing PET: Positron Emission Tomography expensive and higher-resolution PET image Showing a tumor


SPECT Instrument :SPECT Instrument The “gamma camera” is a 2-D array of detectors One or more gamma cameras are used to capture 2-D projections at multiple angles Use filtered back-projection to reconstruct 3-D image! Actual sinograms appear “noisy” due to the fact that we don’t have enough photons Quantum-limited imaging 3-camera SPECT instrument


PET Idea :PET Idea Basic Idea: Nucleus emits a positron A short-lived particle Same mass as electron, but opposite charge Positron collides with a nearby electron and annihilates Two 511 keV gamma rays are produced They fly in opposite directions (to conserve momentum) Nucleus (protons+neutrons) electrons Gamma Photon #1 Gamma Photon #2 BANG


Emission Detection :Emission Detection If detectors A & B receive gamma rays at the approx. same time, we have a detection Hard sensor and electronics design problem, expensive Ring of detectors


Image Reconstruction :Image Reconstruction We can organize our set of detections as a set of angular views Use filtered back-projection algorithm!


PET Images :PET Images Single-channel images Noisy, and blurry Not ideal for segmentation Segment MRI/CT for defining anatomy Register the images Measure activity


Better Algorithms :Better Algorithms Filtered back-projection algorithm produces a background artifact, discussed earlier Noisy reconstruction The Maximum Likelihood algorithm produces a better reconstruction for the same data Filtered Back-Projection Maximum Likelihood


Hybrid Imaging Instruments :Hybrid Imaging Instruments Structure imaging: CT & Magnetic Resonance Imaging Ultrasound Imaging Functional Imaging: Nuclear Imaging Positron Emission Tomography Single-Photon Emission Computed Tomography Combined Modalities Functional & structural imaging 1999 image of the year, U. of Pittsburgh


Image Analysis Steps :Image Analysis Steps Segmentation is the hardest & most critical step to image analysis Much easier to eliminate defects in images prior to segmentation Image Acquisition Image Reconstruction & Pre-processing Image Segmentation Morphometry & Higher-Level Analysis


Imaging Defects :Imaging Defects Types of defects Grayscale distortions Noise Saturation and Cutoff Loss of contrast Blur Loss/movement of edges Non-uniformity of imaging Spatial non-uniformity Temporal non-uniformity Geometric distortions Inadequate Sampling


Avoiding Image Defects :Avoiding Image Defects Make sure the instrument (e.g. microscope) is clean, aligned, and properly adjusted Calibrate the instrument Make every effort to eliminate or minimize disturbances using physical instrumentation Mount the microscope on an anti-vibration table Put an optical filter in the illumination path to suppress light in parts of the spectrum that don’t contribute to a better image Cool parts that are affected by heat


Image Defect Removal :Image Defect Removal Best to avoid them in the first place! Adjust the instrument for maximum performance Once the pixels are recorded, we lose a lot of flexibility Next best thing: Use image processing methods to deal with them “after the fact” It is really impossible to “remove” defects. We can at best “suppress” them to a limited extent There is, invariably, a “price” to pay, e.g., artifacts We need to know the cause of the defect to do the best possible job Being able to construct a mathematical model most helpful


Why Correct Image Defects? :Why Correct Image Defects? Better Visualization So as to not “fool” later processing programs Sometimes lead to better measurements Image Acquisition Image Reconstruction & Pre-processing Image Segmentation Morphometry & Higher-Level Analysis


Noise :Noise What is “noise” anyway? The pixel values in an image can be thought of as representing information about an object An “informative pattern” “Noise” is any variations from the informative pattern It is uninteresting It is a kind of nuisance or disturbance Noise is distinct from “texture” It is informative, though “random” Can’t really “remove” it The pixel values include the disturbance, so can’t isolate it


Noise :Noise Many sources, e.g. “Quantum noise” Not enough photons ( < 50 per pixel) “Thermal noise” at CCD camera front end Digitizer noise Errors in converting signal to digital form with fixed bit length Noise due to computing errors Roundoff errors, especially when working with small numbers of bits per pixel Flickering illumination Vibrations in system (often, due to cooling fans) Errors in image compression/transmission Images often compressed for transmission “Lossy” compression algorithms introduce sub-visual noise-like artifacts


Common Image Observation Model :Common Image Observation Model S(x,y) = f(I(x,y)) + N(x,y) S(x,y) = observed by sensor I(x,y) = the true underlying image f(.) is a distortion function N(x,y) = additive noise


Frame Averaging :Frame Averaging Often, N(x,y) is Gaussian distributed, and independent from one pixel to the next (mean = µ, std dev = ) we can subtract the bias (µ) we can frame average N times to reduce std. dev by . When N(x, y) is non-Gaussian Frame averaging can still be done, but a median average may work better In general, we must use mathematical model for the noise and figure out if averaging can be performed, and then the kind of averaging


When we cannot average frames :When we cannot average frames Moving Object(s) Motion-compensated frame averaging possible if the motion vector is known Restrict averaging to the non-moving regions We look for opportunities to perform averaging in space instead Assumptions: Pixels are much smaller than the objects of interest Images are generally “smooth”


Other Complications :Other Complications N(x,y) may not be uniform across the image µ(x,y) and (x,y) Example: The foreground and background regions may be affected differently by the noise process. N(x,y) may also be time varying µ(x,y,t) and (x,y,t) N(x,y) may not be Gaussian


Quantum Noise :Quantum Noise Common in confocal and fluorescence microscopy & Nuclear Medicine Weak fluorophores, tight pinholes, small radiation dosage, detector limitations Observed image is a realization from a Poisson point process in space with intensity (x,y). Mean = Variance = (x,y) Rule of thumb: If (x,y)  50, then a Gaussian noise model suffices Example, photographic film Frame averaging the method of choice


Multiplicative Noise :Multiplicative Noise S(x,y) = g(I(x,y)) * N(x,y) Also called “modulation noise” Generally, harder to deal with... One method: Take logarithms and get an additive description Another: Consider geometric means instead Generally, requires sophisticated statistical analysis Often, this N(x,y) not random Example: A flickering illumination lamp


Quantum Noise in Fluorescence Microscopy :Quantum Noise in Fluorescence Microscopy


Quantization Noise :Quantization Noise Results when an analog quantity is converted to a discrete n-bit number If we’re off by 1 bit, error is If we assume uniformly distributed additive error, Mean = 0 Variance = 2/12


Neighborhood Averaging :Neighborhood Averaging new pixel value = average of its previous neighbors’ values How do we compute the average? mean/median/mode weights How do we pick the neighbors? How we make the above choices can make a big difference in terms of quality of results artifacts introduced


Choosing the Neighbors :Choosing the Neighbors Neighborhood Size Larger neighborhood leads to more extensive averaging Neighborhood shape Should match object shapes 4 nearest neighbors 8 nearest neighbors


Mean Filtering :Mean Filtering 9 × 9 kernel 3 × 3 kernel Moral: Too big a neighborhood leads to blurring Original Image


Treating Neighbors Differently :Treating Neighbors Differently Simplest Idea: Give lesser importance to neighbors that are farther away E.g., Gaussian weights, discretized and scaled 1 4 1 4 12 4 1 4 1 3 × 3,   0.4 0 0 1 1 1 1 1 0 1 2 3 3 3 2 1 2 3 6 7 6 3 1 3 6 9 11 9 6 1 3 7 11 12 11 7 1 3 6 9 11 9 6 1 2 3 6 7 6 3 0 1 2 3 3 3 2 0 0 1 1 1 1 1 0 1 2 3 3 3 2 1 0 0 0 1 1 1 1 1 0 0 9 × 9,   1


Gaussian Smoothing :Gaussian Smoothing 3 × 3 Gaussian, Variance  0.4 Original Image Magnify for a closer look


Gaussian Smoothing :Gaussian Smoothing 9 × 9 Gaussian, Variance  1.0 9 × 9, Uniform Weighting Magnify for a closer look


“Bad” Neighbors :“Bad” Neighbors Some forms of noise produce drastic and highly localized changes in pixel values “Outliers” Can have huge effect on the averages Recognizing and eliminating these outliers is one other way to treat neighbors differently


The Median Filter :The Median Filter The median = The “middle value” of a bunch of numbers Just sort the numbers and extract the middle number Robust to “outliers” Introduces no “new” values e.g., does not make image dimmer Will smooth (in the extreme, “posterize”) upon repeated application Does not shift boundaries There are many variations on the median filter E.g., center weighted median


Median Filtering :Median Filtering 9 × 9 kernel 3 × 3 kernel Magnify for a closer look


Center-weighted Median :Center-weighted Median 5 × 5 center weighted median 5 × 5 median Basic Idea: Just repeat the values that we want to emphasize Magnify for a closer look


Max/Min Filters :Max/Min Filters Basic Idea: Noise has the effect of creating abrupt changes in the grayscale values of adjacent pixels The extreme values in a neighborhood can form the basis for useful operations Min: Can help us ignore bright outliers - “salt” Max: Can help us ignore dim outliers – “pepper” Suppose our objects are bright against a dark background Min: will erode our objects Max: will fatten (dilate) our objects


Example :Example Min Filtering (radius = 1 pixel) Max Filtering (radius = 1 pixel) Original Image


Max/Min Filters :Max/Min Filters Also called grayscale erosion and dilation We can use them together to smooth images: First pass: pixel value replaced by max{neighbors} Second pass: pixel value replaced by min{neighbors} What happens if we reverse the order (i.e., min followed by max)? The size and shape of the neighborhood (“kernel”) makes a big difference.


Grayscale Closing :Grayscale Closing 3 × 3 rectangular kernel 9 × 9 rectangular kernel Magnify for a closer look


Closing with circular kernel :Closing with circular kernel 9 × 9 circular 9 × 9 rectangular kernel Magnify for a closer look


Opening :Opening 3 × 3 rectangular 9 × 9 rectangular Magnify for a closer look


Opening with Circular Kernel :Opening with Circular Kernel 9 × 9 circular 9 × 9 rectangular Magnify for a closer look


Summary :Summary Survey of common imaging defects Find ways to avoid them in the first place by optimizing specimen preparation and image capture Noise: An important type of defect Methods to deal with unavoidable image noise Introduction to adaptive smoothing Next Class Adaptive, image smoothing algorithms Image Acquisition Image Reconstruction & Pre-processing Image Segmentation Morphometry & Higher-Level Analysis


References on MRI :References on MRI Main MRI Reference: http://www.cis.rit.edu/htbooks/mri/inside.htm Other MRI References http://www.spincore.com/nmrinfo/mri_s.html http://dmoz.org/Science/Chemistry/Nuclear_Magnetic_Resonance/Theory_of_NMR_and_MRI/Basic_NMR_and_MRI_Theory/


References on SPECT & PET :References on SPECT & PET PET http://www.crump.ucla.edu/lpp/lpphome.html SPECT Imaging: http://www.physics.ubc.ca/~mirg/intro.html SPECT Image Atlas http://brighamrad.harvard.edu/education/online/BrainSPECT/BrSPECT.html


Reference for Pre-processing :Reference for Pre-processing Chapter 4 of the textbook Most materials are available online from the author’s web page: http://css.engineering.uiowa.edu/~dip/LECTURE/lecture.html Frame averaging example: http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/imageaveraging/


Summary :Summary Discussion of major medical instruments Structure imaging Function imaging Next Class: Image Pre-processing methods


Instructor Contact Information :Instructor Contact Information Badri Roysam Professor of Electrical, Computer, & Systems Engineering Office: JEC 6046 Rensselaer Polytechnic Institute 110, 8th Street, Troy, New York 12180 Phone: (518) 276-8067 Fax: (518) 276-8715/6261/2433 Email: roysam@ecse.rpi.edu Website: http://www.rpi.edu/~roysab Course Material Website: http://www.ecse.rpi.edu/censsis/BioCourse Weekly Teleconference: Wednesdays at 12:00pm beginning on September 11. Dial 1-888-872-2038 and use the guest passcode 0611#. NetMeeting ID (for off-campus students): 128.113.61.80 Assistant: Betty Lawson, JEC 6049, (518) 276 –8525, lawsob@rpi.edu