Sound Detection

Sound Detection: 

Sound Detection Derek Hoiem Rahul Sukthankar (mentor) August 24, 2004

Objective: 

Objective Learn model of sound object from few (10-20) examples and distinguish from all other sounds Examples of sound classes: Gunshots, screams, laughter, car horns, meow, dog bark, etc

Applications: 

Applications “Tell me if you hear a gunshot.” (monitoring) “Get me video clips containing dogs barking.” (search and retrieval) “What’s going on?” (scene understanding)

Why its difficult: 

Why its difficult Sound classes have large variations Sounds are often ambiguous without context Overlaid “noise” obscures sound

Sound or not?: 

Sound or not? Car horn Laser gun Dog bark Which of these sounds are not from their named classes?

Previous work: 

Previous work Sound Classification (Wold 1996, Casey 2001, etc) Categorize short sound clips Reasonable accuracy (5-20% error) Sound Detection (Defaux 2000, Piamsa-nga 1999) Localize and recognize sound objects in long clips Poor performance or assumption of unrealistic conditions (e.g., very quiet background)

Detection via Windowed Search: 

Detection via Windowed Search Long Track Break audio track into short overlapping short clips Clip Classifier Independently classify short clips as object or non-object Return locations of detected sound object

Representation: 

Representation meows phone rings Raw Representation

Classification Features: 

Classification Features Diverse feature set: Different sound classes are distinctive in different ways means and standard deviations of power at different frequencies Band-width, peaks, loudness, etc. 138 features in all

Classification by Decision Trees: 

Classification by Decision Trees Try to find simple rules that discriminate object from non-object Each decision is based on a threshold of a feature value Assign confidence based on likelihood of data for object and non-object classes at each leaf node Decision nodes Leaf Nodes

Boosted Trees: 

Boosted Trees Problem: One decision tree by itself may not be a great classifier Solution: Use several trees, with each one focusing on the mistakes of previously learned trees Adaboost: Weight training data uniformly Learn a decision tree classifier on weighted data Re-weight data giving more weight to incorrectly classified examples Final classification based on linear combination of confidences from all learned decision trees

Examples of Decision Trees: 

Examples of Decision Trees Low percentage of power in low frequencies in mid-time of sound Very high power amplitude range Meow Gunshot High power amplitude range More complex tree that focuses on examples misclassified by tree above Gunshot

Cascade of Classifiers: 

Cascade of Classifiers Goal: eliminate false positives with few false negatives in early stages Advantages: Allows use of large set of negative training examples Improves classification speed Dangers: cannot recover from false negatives Stage 1 Sound Clip Stage 2 Stage 3 Pass Fail Pass (5%) Pass (2%) Pass (0.005%) Fail Fail Fail

Results: Classification Error: 

Results: Classification Error

Results: ROC curves: 

Results: ROC curves Note: to approximate negative error rate divide FP by 25,000

Results: Anecdotal: 

Results: Anecdotal Gunshots Female Laugh Male Laugh Swords Scream