Cognitive Architecture
Kintsch:
“Comprehension: A paradigm for cognition” (1998)
Chapter 1,2 & 4
by Frank Schumann, David Trill, Sebastian Zösch
Two ways to look at Kintsch’s Construction-Integration model.
As language comprehension model As cognitive architecture today
Cognitive Architectures (I): what and why?
- Why?
- explain many different local phenomena in similar ways - You would explain different local phenomena by
different implementations within one framework.
- Render some possible explanations plausible, but not all.
- What?
- “recipes” for developing theories of cognition.
Newell&Simon
Cognition as Problem Solving
Two things to do now:
1) Work out Kintsch’s construction-integration
theory of comprehension
2) Motivate it as cognitive architecture (in the background of current psychology)
-> embodiment, situatedness <-
Comprehension (I): what is meant?
C. contrasted with perception
z comprehension (or understanding) (1) “relationship between some object
and its context is at issue (2) or when action is required”;
(3) when a perception involves language.
perception:
(1) “simple or isolated instances of perception, (2) especially when no action is involved”
Comprehension (II): what is meant?
• C. conceptualized as constraint-solving
• The constraints
:-- Direct environmental settings
-- and organism state (e.g. knowledge, goals)
-- Task: respond. Either action, or mental event related to action.
The constraint-satisfaction process:
typically:
-- parallel processed,
-- autonomous (stimulus driven)
-- having few demands on resources
Contrasted with problem solving:
typically:
-- processing is conscious (to some degree) -- sequential
-- resource demanding (e.g. attention)
-- constraints have to be generated! (both external and internal), are not directly given.
Comprehension is what we normally use
Problem solving more a
“Repair process”
Comprehension (III):
as construction & integration
Construction:
roughly: associations in neural networks
- related knowledge - Bottom-up process (unguided by larger
(top-down) discourse process) - will also lead to irrelevant items - will be incomplete!
Integration:
roughly, spreading activation leading to stable states
- activate meaningful events, - inhibit inconsistent ones
Comprehension (III):
as construction & integration
Construction:
roughly: associations in neural networks
1) What the elements are:
-- in common sense: perceptions, concepts, ideas, images,emotions -- in the theory: ??
2) How elements are represented:
roughly: propositional networks, not logic
focus on preceptual grounding,
not well-structured logical format 3) Where elements come from:
from external: perceptions
from internal: background knowledge,
Integration:
roughly, spreading activation leading to stable states
1) the processes leading to stable configuration
(Preview) Kintsch has to specify:
Two things to do :
1) Work out Kintsch’s construction-integration
theory of comprehension
2) Motivate it as cognitive architecture (in the background of current psychology)
-> embodiment, situatedness <-
- a hierarchy of representations
- formalisms for know. Representation - propositional representation
How we can view representation in Cognitive Science
• Cognitive Science is more than just looking at problem solving
• There is a huge range of psychological
phenomena which makes up representation
• Examining representation is vital for Kintsch to study comprehension
How do we use representation in Cognitive Science?
• We can using the following example of geometry
and algebra to understand our perspective.
Perspective on Representation
• There is an equivalence between Geometric figures and Algebraic expressions
• Point: the alegabraic representation is not always superior to the geometric representation.
• Now, suppose we want to find a square centred at the origin that has the same area as the circle.
• We find that „some computations are easy to perform in one domain but difficult to perform in another.“
• Perception, comprehension and problem solving generate mental
models of the environmental objects and events, and operate on
these models.
The Mental Representational Hierarchy (I)
(in cognitive science)
• Kintsch describes them „as forming a hierarchy of abstractness and increasing independence from the environment.“
• This does not stipulate a hierarchy of complexity
• „The defining feature is that it changes from direct
representations of the environment to ever more indirect, flexible ones that permit more and more arbitrary,
unconstrained computations“
Important:
• It is a bottom-up hierarchy.
• Lower layers are embedded within the higher layers.
The Mental Representational Hierarchy (II)
(in cognitive science)
5) Abstract representation
4) Narrative Oral representation
3) Nonverbal, imagery and action representation 2) Episodic represenation
1) Direct Procedural and Perceptual representation Main Points to the changes in the Hierarchy
• the degree of environmental control lessens
• representations change from sensorimotor and analog in the lower layers to symbolic and arbitrary in the upper layers
• The degree of consicousness and intentionality increases
Environment Abstraction
Levels of the Hierarchy
(in cognitive science)
1) Direct procedural and perceptual representation
• The most environmentally connected and involve innate systems
• Behaviour is primarily contingent on environmental constraints.
• Procedural learning tends to be unconscious 2) Episodic represenation
• based on episodic memory representation
• key difference is it ultilizes recall and reflection
• it is analytic and reflective but it is still largely environmentally bound
• Start of conscious representation
• Water maze task is evidence for this in animals
Levels of the Hierarchy
(in cognitive science)
3) Nonverbal, imagery and action representation
• these representations are sensorimotor in character but are used intentionally.
• Not totally driven by the environment
• Social communication
4) Narrative oral representation
• Verbal behaviour begins but it is not yet at the abstract level
• Story telling and class presentations are examples
• very socially oriented.
Levels of the Hierarchy
(in cognitive science)
5) Abstract representation
• Independent from the environment
• abstract representations are required for categories, logical thought, formal argument, deduction, quantification and formal measurement
• abstract symbols are dependent on visiougraphic invention such as written langugage, calendars or maps
• example: a child using a banana to represent a telephone while playing.
• Examples at the narrative level can be metaphors.
Evidence for the levels of representation
We recognize two different types of representations 1 - Habit System
2 - Cognitive System
Evidence comes from several different areas:
1) Behaviourism 2) Animal Learning 3) Perception
4) Situated Cognition
Evidence for the levels of representation
5) Cognitive Neuroscience
• Theory of two systems in cognition and learning
• Direct action & Represenation
• Interesting evidence in brain damaged patients
• Findings supports: Lower habit system is encapsulated within the higher cognitive system.
• Ability for language-coding in lower levels.
What we see of Representation
(in cognitive science)
• Embedding is a fact at every level of cognition
• Lower levels are not linguistically bound
• But can be linguistically encapsulated
• Nonnarrative mental representations subject to
redescription – linguistical coding
Two things to do :
1) Work out Kintsch’s construction-integration
theory of comprehension
2) Motivate it as cognitive architecture (in the background of current psychology)
-> embodiment, situatedness <-
- a hierarchy of representations
- formalisms for know. Representation
- propositional representations
Propositional representations
How can we represent layers of embedded representations in our theories?
• Comprehension is complex
• Solution: use language
Problems1) Nonnarrative representations can distort (lower layers) 2) Narrative-langauge level does not stand on its own
• built from lower nonnarrative layers
So, we need a format suitable for the expression of meaning
Knowledge Representations
(in cognitive psychology)
Representation system must fulfill multiple functions
Fourkinds of systems for representation of meaning 1) Feature systems
• semantic features semantic composition rules semantic concepts
• „bachelor“ = +HUMAN, +MALE, +HAS-NEVER-MARRIED
• Simple
but cannot capture higher representations 2) Associative Networks
• concepts as nodes and unlabeled links
• Limitation: all knowledge cannot be represented by unlabeled links Free association
lexical decision tasks
3) Semantic Networks
• Concepts as nodes and labeled links - forming relations
• IS-A or PART-OF
•
A canary has skin animals have skin• Advantage: accounts for the inheritance of properties
4) Schemas, frames and scripts
• are structures used to coordinate concepts that are part of the same superstructure, or event.
• Inferring components that are incomplete or missing
• recipes for generating organizational structures in a particular task
Knowledge Representations
(in cognitive psychology)
Knowledge Representations
(let`s talk Kintsch)
Networks of Propositions
• The Predicate-Argument Schema
– basic unit of language– It is a proposition
• GIVE [
agent: MARY,
object: BOOK,
goal: FRED]
• We can start to embed propositions
• GIVE [agent: MARY, object: OLD[BOOK], goal: FRED]
• Ranges from Atomic to Complex propositions
• Key idea is embedding
Predicate-Argument Schema as mother
General and Flexible enough to subsume other systems 1) Features
• RED [ROSE, COLOUR]
• meaning unit: value-object-attribute relation 2) Associative
• meaning units may be linked by unlabeled links in varying strengths
• nodes consist of predicate-argument units
3) Semantic
Predicate-Argument Schema as mother
4) Schemas, frames, scripts
• nodes linked to other nodes
• ROBIN node connected to all knowledge of „robin“
Overall: representing meaning
• Predicate-Argument notation is flexible to cover other forms of representation
• These systems are inadequate on their own
• We need only use one uniform format: Propositional network
Encapsulating Meaning
• Problems
• higher layers of representation encapsulate lower levels
• extracting information causes distortion
The human mind can use all levels simultaneously
• Point: meaning is rooted in perception, action and emotion
• Redescription of words and meaning towards higher levels
• Cognition is both symbolic and action related
• However: language is used at all levels of representations
Two things to do :
1) Work out Kintsch’s construction-integration
theory of comprehension
2) Motivate it as cognitive architecture (in the background of current psychology)
-> embodiment, situatedness <-
What you have seen:
--- MR from concrete to abstract
--- Traditional Concepts: features, schemas, … --- propositional networks can encode
other formalisms. Discussable!
Kintsch‘s comprehension theory: What do we want?
• A psychological theory of understanding
(not just formal analysis of the text to be understood)To understand we have to construct a mental model of the situation that connects
ideas of the text with relevant prior knowledge
similar to gestalt principles:
out of perceptual units coherent coherent wholes wholes emerge emerge
The Traditional View
Process of understanding is under control of Schemas which guide it
Schema is a control structure that regulates the comprehension processes in a top-down fashion Two mechanisms:
perceptual filter (blocks irrelevant material) inference machine (fills the gaps)
What‘s wrong with that?
Guidance
Guidance of of ComprehensComprehensionion seems
seems to to bebe a morea more looselyloosely structured
structured bottombottom-up-up processprocess Comprehension
Comprehensionand and contextcontext--sensitivesensitiveisis flexible flexible
The Construction-Integration model
• Integration:
roughly, spreading activation
leading to stable states - activate meaningful events,
- inhibit inconsistent ones
- global context
• Construction:
roughly: associations in neural networks
- related knowledge - Bottom-up process - will also lead to
irrelevant items - local context
Cake Pacman
Construction
1. Rules to construct the propositions
parsing the text (details missing in the model)
2. Rules for interconnecting the propositions in a network 3. Rules for the activation of knowledge
association ???
4. Rules for construction inferences
These rules are simple, robust and work only on local Context (for more sophisticated Schemas rules are
needed)
- - Integration Model Integration Model
Rules for interconnecting the propositions in a network
4 types of connections:
direct, indirect, subordinate, negative (contradiction)
e.g. they are flying planes
FLY[THE, PLANES] ---ISA[THEY, FLYING[PLANE]]
4. Rules for constructing inferences
If A above B and B above C, then A above C
A rule for constructing a macroproposition
„Franz drove to Aldi, picked up some fresh fruit, some
noodles and Italian cheese for dessert, and paid with his credit card.“
Franz went grocery shopping .
Construction- Model
D B
A C
E
Initially activation value A(1) = (1, 1, 1, 1, 1)
node A B C D E
A(2) = (1.00, .75, .75, .50, .25)
A(9) = (1.00, .85, .46, .85, .00)
So the inappropriate node is surpressed.
Integration Integration
Leading to a stable state
= Constraint satisfaction process
CI happens in Processing Cycles
While reading a network is constructed and each node is integrated immediately (so disambiguity is solved as soon as possible, not only at the end of the sentence)
Working memory (network) is limited and needs to be cleared (transferred to long-term memory).
After that a new cycle begins
1 or 2 central propositions are carried over
Episodic text memory
Winner of the cylce
Episodic text memory
• Textbase
Propositional net purely of the text
Often incoherent, impoverished
Can have a surface structure (rhymes, rhythms etc.)
Reader independent
• Situation model
Reader must add nodes, establish links to make it coherent, complete.
Various sources needed:
language, world etc.
Different readers might
create different models
Unitary structure which contains two components
From textbase to situation model
• Add causal links where missing
• Elaborative inferences
• Inferences for creating an image
• Create overall organization of text
• ...
Add causal links where missing
Jane could not find the apple she was looking for. She become upset.
not finding what you are looking for
makes you upset. So therefore she
became upset (causal link)
Elaborative inferences
Jack missed the class because he played golf. He told the teacher that he was sick.
Jack lied
Inferences for creating an image
The turtle sat on a log. A fish swam under the log.
Fish is in the water,
Turtle above the fish
Create overall organization of text
• Readers generate a macrostructure:
„Jane goes grocery shopping“
Script:
Relation btw. textbase and situation model
Subjects remember the situation but not the text itself
Text was remembered
Getting concrete!
John traveled by car form the bridge to the house.
A train passed under the bridge.
TRAVEL
agent: JOHN instrument: CAR source: BRIDGE goal: HOUSE PASS
object: TRAIN
location: UNDER_BRIDGE
John traveled by car form the bridge to the house .
TRAVEL[JOHN, CAR, BRIDGE, HOUSE]
ON[CAR,ROAD] BETWEEN[HOUSE,ROAD,BRIDGE]
.47
UNDER[RIVER, BRIDGE]
1.00
UNDER[TRAIN, BRIDGE]
A train passed under the bridge
.00
“A context-insensitive construction process is followed by a constraint-satisfaction process (or integration) process that yields, if all goes well, an orderly mental structure out of initial chaos.”
Kintsch, 1998
