ling411 23

Schedule of Presentations: 

Schedule of Presentations Th Nov 30 Tu Dec 5 Th Dec 7

Coarse Coding and Fine CodingBroad View of Perception: 

Coarse Coding and Fine Coding Broad View of Perception Ling 411 – 23

Coarse and fine coding: 

Coarse and fine coding Coarsely coded node Responds to a relatively large range of values Finely coded node Responds to a narrow range Needed for sharp contrasts Examples Phonology Morphology Mathematics

Uses of coarse and fine coding: 

Uses of coarse and fine coding Fine coding for Sharp contrasts Voiced vs. voiceless stops Edges in vision Coarse coding for Meanings with broad range of semantic properties General visual impressions

Coarse and fine coding:Low-level nodes: 

Coarse and fine coding: Low-level nodes Low-level: near bottom of hierarchy Lowest level: primary areas Lowest level nodes are coarse-coded At other low levels, coarse and fine coding Colors (visual cortex) Fine coding for fine color discrimination Coarse coding for range of color Frequencies (auditory cortex) Fine coding for fine pitch discrimination Coarse coding for range of pitches

Typical Low-level Node: Coarsely Coded: 

Typical Low-level Node: Coarsely Coded Responds to a range of inputs

Coarse coding at higher levels: 

Coarse coding at higher levels A node with a large number of incoming connections and a relatively low threshold This structure allows it to respond to any of a broad range of situations

Coarse and fine coding:High-level nodes: 

Coarse and fine coding: High-level nodes High-level nodes – concepts, meanings Coarse coding More coarse in RH Broad range of semantic properties In RH, not necessarily logical Fine coding Mainly in LH Narrow range of semantic properties

Example of a conceptual category: 

Example of a conceptual category T CUP MADE OF GLASS CERAMIC SHORT HAS HANDLE Properties The head concept node Different strengths of connection shown by different thicknesses of lines

A coarsely-coded category: 

A coarsely-coded category T CUP MADE OF GLASS CERAMIC SHORT HAS HANDLE Properties The head node Threshold is low enough that the node is activated by a small subset of its links

A coarsely-coded category: 

A coarsely-coded category T CUP MADE OF GLASS CERAMIC SHORT HAS HANDLE Properties The head node Therefore, the CUP node is activated by varying combin-ations of a large range of properties

Coarse coding and RH: 

Coarse coding and RH Coarse coding is particularly prominent in RH Beeman: “diffuse activation” in RH (as opposed to “focused activation” in LH)

Possible bases for RH/LH difference: 

Possible bases for RH/LH difference Higher ratio of white to gray matter in RH Therefore, higher degree of connectivity in RH Difference in dendritic branching Different density of interneurons Evoked potentials (EEG) are more diffuse over the RH than over LH Beeman 257

Experiments (described by Beeman): 

Experiments (described by Beeman) Words presented to rvf-LH or lvf-RH RH more active than LH Synonyms Co-members of a category: table, bed Polysemy: FOOT1 – FOOT2 Metaphorically related connotations Sustains multiple interpretations LH about same as RH Other associations: baby-cradle LH more active than RH Choose verb associated with noun

Patients with brain-damage: 

Patients with brain-damage Some patients with LH damage Can’t name fruits but can say that they are fruits Patients with RH damage Impaired comprehension of metaphorical statements More difficulty producing words from a particular semantic category than producing words beginning with a particular letter (258)

Imaging studies: 

Imaging studies When listening to spoken discourse, cerebral blood flow increases in Wernicke’s area Broca’s area RH homologues of Wernicke’s and Broca’s areas More cerebral blood flow in RH when subjects read sentences containing metaphors than literal sentences

Experiments on speech perception: 

Experiments on speech perception Dichotic listening – normal subjects Right ear (i.e. LH) advantage for distinctions of Voicing Place of articulation Left hear (RH) advantage for Emotional tone of short sentences Sentences presented in which only intonation could be heard RH advantage for identifying sentence type – declarative, question , or command

Experiments on speech perception: 

Experiments on speech perception Split brain patients They hear a consonant Then written representations are presented ‘Point to the one you heard’ rvf-LH exhibited strong advantage

Patients with right-brain damage: 

Patients with right-brain damage Posterior RH lesions result in deficits in interpreting emotional tone Anterior RH lesions abolish the ability to control the production of speech intonation

Split-brain studies: 

Split-brain studies Isolated RH has ability to read single words But not as fast nor as accurate as LH Ability declines with increasing word length Lexical context does not assist letter identification In Japanese subjects RH is better at reading kanji than kana LH is better at reading kana

Structures for coarse and fine coding: 

Structures for coarse and fine coding For coarse coding Large number of incoming links Low threshold For fine coding Threshold high in relation to number of incoming links Lateral inhibition

The structure of lateral inhibition: 

The structure of lateral inhibition Inhibitory connections Extend horizontally to other columns in the vicinity Used for enhancement of contrast

Inhibitory connections Based on Mountcastle (1998): 

Inhibitory connections Based on Mountcastle (1998) Columnar specificity is maintained by pericolumnar inhibition (190) Activity in one column can suppress that in its immediate neighbors (191) Inhibitory cells can also inhibit other inhibitory cells (193) Inhibitory cells can connect to axons of other cells (“axoaxonal connections”) Large basket cells send myelinated projections as far as 1-2 mm horizontally (193)

Coarse coding at low levels: 

Coarse coding at low levels Typical situation for sensory neurons Neurons fire.. Occasionally at random even when not receiving activation More strongly when receiving activation More strongly yet when receiving a lot of activation Hence, low level nodes have broad receptive fields Locally, they are coarsely coded

How to get fine coding: 

How to get fine coding Neurons (hence also columns, presumably) are inherently, locally, coarse-coded For linguistic processing we often need much greater precision: fine coding Problem: How to get finely coded nodes if neurons are inherently coarsely coded?

Response curve of a coarsely coded node: 

Response curve of a coarsely coded node Responds to a wide range of inputs

Response curve of node A (coarsely coded): 

Response curve of node A (coarsely coded) Range of colors Node A is coarsely coded for

Response curve of node B (coarsely coded): 

Response curve of node B (coarsely coded) Node B is coarsely coded for Node A is coarsely coded for

Overlapping receptive fields: 

Overlapping receptive fields “…each individual representation (e.g. receptive field) is inexact, or coarse, but … the overall system of overlapping representations can provide precise interpretations. Mark Beeman (1998), 256

Overlapping receptive fields: 

Overlapping receptive fields A B

Higher-level node C – more finely coded: 

Higher-level node C – more finely coded A B C A B Response curve of C Response curve of B Response curve of A

Enhance fine-coding with inhibition: 

Enhance fine-coding with inhibition Node C can be yet more finely coded by receiving inhibitory inputs from nodes for and A B C A B

Further enhancement by raising threshold: 

Further enhancement by raising threshold A B C A B Threshold

Coarse coding at higher levels: 

Coarse coding at higher levels A node with a large number of incoming connections and a relatively low threshold This arrangement allows it to respond to any of a broad range of situations Coarse coding is the usual situation at the conceptual level A concept node generally represents a category, not just a single thing Different members of the category, with differing features, activate the category node

Coarsely coded concept nodes: 

Coarsely coded concept nodes Cups A great variety of cups activate the ‘CUP’ node To different degrees Properties of prototypical cups activate the node more strongly Your grandmother Wearing different clothes Doing different things Seen live or in a picture At different ages Etc.

Perception – Refining a simple-minded view: 

Perception – Refining a simple-minded view Not just bottom-up Top-down processing fills in unsensed details Not confined to a single perceptual modality The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception Not even confined to posterior cortex Can also use motor neurons Experiment: left hand or right hand? Mirror neurons

Top-down processing in perception: 

Top-down processing in perception T CUP MADE OF GLASS CERAMIC SHORT HAS HANDLE Properties Conceptual and perceptual information Node for CUP in conceptual area for drinking vessels Visual properties in occipital and lower temporal areas

Bidirectional processing and inference: 

Bidirectional processing and inference T CUP MADE OF GLASS CERAMIC SHORT HANDLE These connections are bidirectional

Bidirectional processing and inference: 

Bidirectional processing and inference T CUP SHORT HANDLE Thought process: 1. The head concept node is activated by a subset of its property nodes 2. Feed-backward processing activates other property nodes Consequence: We “apprehend” properties that are not actually perceived

Multi-Modal Perception: 

Multi-Modal Perception Perception is not just bottom-up Top-down processing fills in unsensed details It is not confined to a single perceptual modality The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception It is not even confined to posterior cortex Can also use motor neurons Motor activation in speech perception Mirror neurons

The McGurk Effect: 

The McGurk Effect Acoustic syllable [pa] presented to subjects with visual presentation of articulatory gestures for [ka] Subjects typically heard [ta] or [ka] “Evidence has accumulated that visual speech modifies activity in the auditory cortex, even in the primary auditory cortex.” Mikko Sams (2006)

A terminological problem: 

A terminological problem We need to distinguish Perception narrowly conceived The basic process of recognition Single perceptual modality Bottom-up processing No motor involvement Perception broadly conceived Two different terms needed Recognition (a.k.a. ‘microperception’) Perception (the broad conception) (a.k.a. ‘macroperception’)

Microperception and macroperception: 

Microperception and macroperception Microperception A.k.a. recognition The local process of integrating features Performed in one perceptual modality Bottom-up Macroperception The overall process of perception Uses multiple modalities Uses top-down processing

Perception – Refining a simple-minded view: 

Perception – Refining a simple-minded view Not just bottom-up Top-down processing fills in unsensed details Not confined to a single perceptual modality The McGurk effect Visual input affects auditory perception Conceptual structure affects auditory perception Not even confined to posterior cortex Can also use motor neurons Experiment: left hand or right hand? Mirror neurons

Motor structures in perception: 

Motor structures in perception The left-hand vs. right-hand experiment ‘Mirror neurons’ in motor cortex Articulation as aid to phonological perception Articulation in reading Motor activity in listening to music Watching an athletic event

Mirror Neurons: 

Mirror Neurons NY Times: “One mystery remains: What makes them so smart?” (Jan. 10, 2006) Answer: They are not smart in themselves Their apparent smartness is a result of their position: at top of a hierarchy Compare: The general of an army The head of a business Similarly, high-level conceptual nodes The “grandmother node”

Mirror Neurons: 

Mirror Neurons What makes mirror neurons appear to be special? Ans.: They receive input from visual perception The superior longitudinal fasciculus Connects visual perception to motor areas How can a motor neuron receive perceptual input? Motor neurons are supposed to operate top-down Answer: bidirectional processing

Superior Longitudinal Fasciculus: 

Superior Longitudinal Fasciculus From O. D. Creutzfeldt, Cortex Cerebri (1995)

Are some neurons “smarter” than others?: 

Are some neurons “smarter” than others? Claim: A grandmother node would have to be very smart Identifies very complex object Even in many varieties Alternative: the head of a hierarchy It is the hierarchy as a whole that has those ‘smarts’ Similarly, mirror neurons They get visual input since they are connected to visual areas Superior longitudinal fasciculus

Implications of hierarchical organization: 

Implications of hierarchical organization Nodes at a high level in a hierarchy may give the appearance of being very “smart” This appearance is a consequence of their position — at top of hierarchy As the top node in a hierarchy, a node has the processing power of the whole hierarchy Grandmother nodes Mirror neurons Compare: The general of an army The head of a business organization

Perceptual structures in motor production: 

Perceptual structures in motor production Perceptual structure is used in two ways Planning (e.g. visualizing while painting) Monitoring Examples Phonological recognition in speech production Cf. Wernicke’s aphasia Painting Musical production Baseball, soccer, tennis, etc.

More on the Proximity Principle: 

More on the Proximity Principle Start with the observation: Related areas tend to be adjacent to each other Primary auditory and Wernicke’s area V1 and V2, etc. Wernicke’s area and lexical-conceptual information – angular gyrus, SMG, MTG Hence, we have the ‘proximity principle’ Leads to a question: Why – How to explain? Genetic factors Experience – provides details of localization within the limits imposed by genetic factors Hypothesis based on the learning process Evidence: plasticity

Two varieties of the proximity principle: 

Two varieties of the proximity principle A node that integrates a combination of properties of the same subsystem should be within the same subsystem, and maximally close to the properties it integrates A node that integrates a combination of properties of different subsystems can be expected to lie in a location intermediate between those subsystems

Two Factors in Localization: 

Two Factors in Localization Genetic factors determine general area for a particular type of knowledge Within this general area the learning-based proximity factors select a more narrowly defined location When part of the system is damaged, learning-based factors can take over and result in an abnormal location for a function (“plasticity”)

Genetically determined proximity: 

Genetically determined proximity Presumably, the genetic factors developed over a very long period of evolution Many are shared by most (all?) other mammals This process may be called ‘evolutionary learning’ According to standard evolutionary theory.. A process of trial-and-error: Produce varieties Select the best among them Other genetic factors supplement proximity Long-distance fiber bundles

Some innate factors relating to localization: 

Some innate factors relating to localization The primary areas Genetically determined locations But there are exceptions Genetically determined structures adapted to sensory modality (they have to be where they are) Heterotypical structures Long-distance fiber bundles Interhemispheric – via corpus callosum Longitudinal – from front to back Arcuate fasciculus is part of the superior longitudinal fasciculus

Applying the proximity principle: 

Applying the proximity principle For both types (genetic and experience-based) we can make predictions of where various functions are most likely to be located, based on the proximity principle Broca’s area near the inferior precentral gyrus Wernicke’s area near the primary auditory area Such predictions are also possible in cases where we don’t know whether genetics or learning is responsible (maybe both)

Implications of the proximity principle: 

Implications of the proximity principle Nodes for similar functions should be physically close to one another Nodes that are physically close to one another probably have similar functions Neighboring nodes are likely to be competitors They need to have mutually inhibitory connections Functionally related subsystems will tend to be close to one another Neighboring subsystems will probably have related functions

Exercise: Location of Wernicke’s area: 

Exercise: Location of Wernicke’s area Why is phonological recognition in the posterior superior temporal gyrus? Alternatives to consider: Anterior to primary auditory cortex Advantage: would be close to phonological production Inferior to primary auditory cortex (There are two reasons)

Another exercise: 

Another exercise Locations of the subareas of Wernicke’s area It should have separate layers for (e.g.) Demisyllables Syllables Phonological words Soon we can hope to get confirmation or falsification of such predictions from sophisticated imaging experiments

Multiple phonological levels: 

Multiple phonological levels Phonological words Syllables C- / -V[C] (?) Articulatory Features(?) Phonological phrases Phonological words Syllables CV- / -V[C] (?) Auditory Features(?) Phonological Production Phonological Recognition

Multiple phonological levels: 

Multiple phonological levels Phon Wrds Syl C- / -V[C] Art Feat Phon phr Phon wrds Syl CV-/-V[C] Aud Feat Phonological Production Phonological Recognition

Multiple phonological levels: 

Multiple phonological levels Phon Wrds Syl C- / -V[C] Art Feat Phon phr Phon wrds Syl CV-/-V[C] Aud Feat Phonological Production Phonological Recognition Arcuate fasciculus

Limits on localization by learning: 

Limits on localization by learning Depends on availability of latent connections Limits on intercolumn connectivity Number of cortical minicolumns: If 27 billion neurons If 90 neurons per minicolumn Then 300 million minicolumns Extent of genetically determined interconnection Perhaps 30,000 to 300,000 (latent) connections to other columns Hence, a given column is connected to one-thousandth or one-ten-thousandth of the other columns in the cortex

Locations of available latent connections: 

Locations of available latent connections Local Surrounding area Horizontal connections (not white matter) Intermediate Short-distance fibers in white matter For example from one gyrus to neighboring gyrus Long-distance Long-distance fiber bundles At ends, considerable branching

More innate factors: 

More innate factors High-level vision The ventral pathway (temporal): What The dorsal pathway (parietal): Where Phonological perception Wernicke’s area pretty much has to be where it is to take advantage of the arcuate fasciculus

The role of long-distance fibers: 

The role of long-distance fibers Arcuate fasciculus Genetically determined Limits location of phonological recognition area Interhemispheric fibers Also genetically determined Wernicke’s area – RH homolog of W’s area Broca’s area – RH homolog of B’s area Etc.

Experience-based proximity: 

Experience-based proximity Can be expected to operate more at higher (more abstract) levels, less at lower levels Can be expected to be operative for areas of knowledge that have developed too recently for evolution to have played a role Reading Writing Higher mathematics Physics, chemistry, etc.

Innate features that support language: 

Innate features that support language Columnar structure Coding of frequencies in Heschl’s gyrus Arcuate fasciculus Interhemispheric connections (via corpus callosum) – e.g., connect Wernicke’s area with RH homolog Spread of myelination from primary areas to successively higher levels Left-hemisphere dominance for grammar etc.

Review – Processing in the cortex: 

Review – Processing in the cortex Parallel (distributed) and serial Hierarchical Bidirectional Variable Varying strengths of connections Varying degrees of activation Variation over time Adaptability Learning Plasticity

Properties of connectionist representation: 

Properties of connectionist representation All the information is in the connectivity Information is distributed Yet every node has a specific local function Access without addressing or searching How? – By following connections Low-level information in dedicated locations Low-level: sensory or motor Dedicated locations: primary cortical areas Higher-level information is opportunistic Located in accordance with experience Plasticity reigns

Uniformity of structure and function: 

Uniformity of structure and function Locally, All cognitive and perceptual information, of any kind, is represented as nodes and their interconnections All cognitive processing, of any kind, consists of broadcasting and integration The structure that subserves language understanding is the same as perceptual structure Understanding language is the same process as perception

Complexity from simplicity: 

Complexity from simplicity Complexity: what the brain can do Simplicity: every node is a simple processor Integration Broadcasting Changes in connection strengths and thresholds Problem: how can such simplicity produce such complexity? Answer: Huge quantity of nodes and connections Parallel distributed processing Hierarchical organization

Slide74: 

end

Schedule of Presentations: 

Schedule of Presentations Th Nov 30 Tu Dec 5 Th Dec 7