Lecture 20 -- Ch. 9.1 and 9.2

Explain how Rumelhart and McClelland's neural network worked

- 560 verbs divided into High, Medium, and Low frequency (regular and irregular) 1) train on 10 high frequency verbs for 10 epochs 2) 410 medium frequency verbs added, trained for 190 or more epochs -- net showed dip in performance, making errors like children "come-comed" 3) tested on 86 low frequency verbs not used for the training -- got 92% regular verbs rights, 84% irregular right - required input early on were mostly irregular (few regular verbs) - required input later was some irregular some regular

dual theory interpretation of neurological impairments

- Grammar areas handle regular processing -lexical semantics areas handle irregular processing - agrammatism vs anomia

Connectionist models interpretation of neurological impairments

- Irregulars depend more on semantics than phonology - where as regulars depend more on phonology; so more damage to phonological representation will cause affect regulars more

Alternative dual interpretation (Ullman, 2001) of neurological impairments

- Regular processing on procedural memory, irregular on declarative memory

Critiques of connectionist model by Pinker and Prince

- U-shaped learning occurs as a result of implausible and carefully engineered training regime -- careful set of training procedures that they went through - network can only produce past-tense forms, cannot recognize them - model present pairs of verb+past tense, but children don't normally pair them together - model only looks at past-tense, not the rest of language

broca's aphasia

- agrammatism -- problems with grammar of language (rules and combinatorial processes) - prediction = these patients do worse on regular+ed rule than irregulars - results: poor performance comparatively, but much worse on producing the correct past tense form for regular verbs (and no overregularizations for irregular verbs)...worse at rules and combinatorial processes

wernicke's aphasia

- anomia -- problems with remembering words (lexical access in associative memory) - prediction = these patients do worse on irregulars than +ed rule - results: not so bad comparatively (over 80% production), but better at regular verbs than irregular verbs...also, produced many over-regularizations (19%)...good at rules, not good at irregulars

Neurological evidence of declarative/procedural hypothesis

- aphasia -- studies on patients with brain lesions - broca's aphasia (in frontal lobe) - wernicke's aphasia (cerebral cortex)

What is the neurological basis of the words and rules theory?

- declarative/procedural hypothesis by Pinker and Ullman - declarative memory -- memory of facts and events (knowing "what")...lexical/irregular...hippocampus and medial lobe structures - procedural memory -- knowing how to do things...grammatical/regular...basal ganglia and frontal cortex

Rule based theory (dual route -- words and rules theory)

- different forms of words in all human languages is computed by a dual route mechanism consisting of pattern associator type of memory module (for irregulars and irregular sounding regulars) and a rule (for defaults) which is unblocked only when pattern associator fails - there is a list of exceptions (the pattern associator, certain phonological patterns that we are using to store the irregular verbs( that can block out the application of the verb rule - otherwise, it will go through the regular route and apply the past tense verb rule - propose by Pinker and Prince

What is Fodor's view (language learning and the LOT)

- he thinks of it as defining the rules - he thinks of language learning in terms of truth rules

What are the strengths of connectionism?

- help in understanding how a mind, and thought, emerges from the brain - is it a better account of how we learn something like past-tense, than explicit programming of a rule?? -- this is the big question

What are truth rules?

- how do words contribute to determining if their sentence is true? ex. Tyler wears glasses. - sentence made of proper name and predicate - it's true if the name in sentence does actually have that property...the truth condition is: "Tyler wears glasses" is true just if Tyler wears glasses

double dissociation for regular and irregular verbs

- if we find double dissociation behavior, it means there must be a separate process responsible for producing each verb type - anomic patient -- damage to lexicon retrieval and associative memory; regular verbs and grammar are okay - agrammatic patient -- damage to grammatical and combinatorial processes; irregular verbs and lexicon retrieval are okay

The rule based (dual route) theory explaining children's verb development

- if we suppose over-regularization happens because children's memory is weaker than adult's memory, it should be case that the more often children hear a word, the easier it is to retrieve from memory - implication: the more often children hear irregular past tense forms like "went", the easier it will become to retrieve those irregular past tense forms even when children already have a regular rule (+ed) they use for many other verbs - experimental support for this: Children make more errors on words parents don't use as frequently (presumably this is because their memory for these verbs is weaker)

How does the LOT come into play in language learning?

- in mentalese language we have basic concepts that are being built up to make more complex concepts and then we are using truth tules on these concepts we have built up - innate basic concepts are already represented in our language of thought

Diagram and explanation of Rumelhart and McClelland's neural network

- learning happens in the middle - start with input = phonological representation (sound) of root form (i.e. present tense verb) - output = phonological representation (sound) of past tense - pattern associator with modifiable connections is trained by perceptron convergence rule - wickelfeatures -- represent features of phonemes - certain types of wickelfeatures/sounds/phonemes are going to be more likely to be grouped together

Which theory goes more with connectionists and which goes more with PSSH?

- mastering goes with connectionist because they didn't believe in explicit rules - defining goes with PSSH because gives symbols and rules to manipulate them

Connectionist view explaining children's verb development

- pattern: hold-held, walk-walked, go-went - they found that a neural network could indeed produce U-shaped behavior - caveat: the models received different proportions or irregular verbs in their input at different points in time (high proportion early on, lower proportion later on) - model showed that the neural network discovered the relationship between inputs and outputs, it wasn't programmed --so possible to capture rule-governed behavior in a neural network

What are the stages of acquisition of children's learning pattern of past-tense verbs?

- stage 1 = past tense of a few specific verbs, some regular (ex. looked, needed), but most are irregular (ex. came, got, went, etc.)...assumption that learned by rote (memorized) - stage 2 = use much greater number of verbs in past tense, most of which use the general rule of adding "ed" to stem of verb (ex. camed or comed)...step backwards because have over-regularization errors...also generate past tense for made up words - stage 3 = correct forms for both regular and irregular verbs - u-shaped development

What are neural networks good at modeling?

- the fact that cognition changes as we get older

Connectionist view (words, no rules theory -- single route theory -- neural network theory)

- there is a single mechanism for handling both regular and exceptional forms - wanted to build a neural network to replicate rule-governed linguistic behavior without the rules being explicitly taught - Is it really necessary to have learned rules, or could children (and adults) simply be learning (and using) associative memory to retrieve both regular and irregular patterns - this type of associative memory can be represented in artificial neural networks - Rumelhart and McClelland proposed this

How do you learn a language according to Fodor then?

- therefore, learning a language involves learning truth rules - first form a hypothesis about what "Tyler" and "wears glasses" mean - then test it against more linguistic data - revise if necessary (ex. find out Luke has no brothers)

Plunkett and Marchman

- took connectionist model of network and reproduced over-regularization error - how has hidden layers - more regular than irregular verbs, just like real children - characteristic mistakes (ex. adding "ed' to every verb) - gradual change in performance, just like real children - so, even though had more general inputs, still worked and modeled children well (less biased, and got rid of their first complaint)

What is the default hypothesis of language learning?

- understanding a language is fundamentally a matter of mastering rules (both for vocabulary and grammar)

Because of this problem with truth rules, what does Fodor think we need?

- we need rules to tell us who "Tyler" is and what "wears glasses" means - ex. Tyler would need to be represented by something the user understands like "Luke's brother" -- this is an example of a truth rule for Fodor

What is the problem with truth rules?

- you can't understand what the truth rule means unless you already understand the sentence (aka it doesn't make sense to define the truth of "Tyler wears glasses" as Tyler wears glasses

What are the two schools of thought to understanding a language?

1) mastering the rule -- being able to use it correctly (practical ability)...ex. learning how to ski 2) defining the rule -- must be able to explain the rule, must be able to be represented in the mind (learning how to follow explicit rules)...ex. learning arithmetic

Understanding a language requires what two things?

1) understanding what words mean 2) understanding the grammatical rules of how to put words together coherently