Productions(Basal Ganglia)

(1)

Cognitive Architectures Cognitive Architectures

ACT ACT - - R R

(2)

Outline Outline

Short glance on the history of ACT

Short glance on the history of ACT - - R R What is ACT

What is ACT - - R ? R ? Mapping ACT

Mapping ACT - - R onto the brain R onto the brain ACT ACT - - R 5.0 Architecture R 5.0 Architecture

Components of ACT

Components of ACT - - R R What is ACT

What is ACT - - R used for? R used for?

General discussion

(3)

History of the ACT

History of the ACT - - framework framework

1976: first ACT theory came out 1976: first ACT theory came out

1982: first ACT implementation appeared 1982: first ACT implementation appeared

Since then both, the

Since then both, the Theory Theory and the and the Implementations

Implementations were further developed were further developed (ACT, ACT

(ACT, ACT - - R, ACT R, ACT - - R2.0, ACT R2.0, ACT - - R3.0, R3.0, ACT ACT - - R4.0) R4.0)

2001: release of ACT

2001: release of ACT - - R5.0 (Theory and R5.0 (Theory and Implementation) which is since then the Implementation) which is since then the

state of the art ACT

state of the art ACT - - R R

(4)

What is ACT

What is ACT - - R R

ACT ACT - - R is a cognitive architecture R is a cognitive architecture Researchers working on ACT

Researchers working on ACT - - R strive to R strive to understand how people organize

understand how people organize knowledge and produce intelligent knowledge and produce intelligent

behaviour

behaviour . .

(5)

What is ACT

What is ACT - - R R

ACT ACT - - R is a programming language R is a programming language Models are written in ACT

Models are written in ACT - - R R

During runtime of a model, ACT

During runtime of a model, ACT - - R R provides the runtime environment.

provides the runtime environment.

Due to it’s special design as a cognitive Due to it’s special design as a cognitive

architecture, models in ACT

architecture, models in ACT - - R can mirror R can mirror

human behavior on a cognitive psychology

task task

(6)

What is ACT

What is ACT - - R R

Based on facts Based on facts

derived from derived from

psychology psychology

experiments, experiments, ACT ACT - - R is a R is a

framework framework

Models in ACT

Models in ACT - - R R reflect a certain reflect a certain

aspect of cognition

(7)

Framework ? Framework ?

Environment

ACT-R

(8)

ACT ACT - - R Architecture R Architecture

ART ART - - R claim: cognition as the interaction R claim: cognition as the interaction between specific units of knowledge:

between specific units of knowledge:

Declarative knowledge Declarative knowledge Unit:

Unit: Chunks Chunks

E.g. facts, goals, … E.g. facts, goals, …

Procedural knowledge Procedural knowledge Unit:

Unit: Production rules Production rules

E.g. action rules, behavior E.g. action rules, behavior

rules, …

(9)

ACT ACT - - R Architecture R Architecture

Environment

Production rules

Chunk

Chunk Chunk

Chunk

(10)

A C T

A C T - - R Architecture R Architecture

Chunks are created by specific

Chunks are created by specific modules modules

visual module produces chunk “Christian is in visual visual module produces chunk “Christian is in visual

field”

Motor module produces “pressure on left hand”

Chunks set modules to action Chunks set modules to action

“search Christian” said to visual module

Modules transmit and retrieve information Modules transmit and retrieve information

only out of

only out of buffers buffers

Each module has a specific buffer for his Each module has a specific buffer for his

chunks

(11)

ACT ACT - - R Architecture R Architecture

Environment

Modules Modules

Buffers Buffers

Production rules

Chunk

Chunk Chunk

Chunk

(12)

Mapping ACT

Mapping ACT - - R onto the brain R onto the brain

Question: How is ACT

Question: How is ACT - - R related to newest R related to newest studies in neurobiology and

studies in neurobiology and neuroimaging

neuroimaging ? ?

Answer: all parts of ACT

Answer: all parts of ACT - - R are designed R are designed to reflect certain brain areas!

to reflect certain brain areas!

(13)

Mapping ACT

Mapping ACT - - R onto the brain R onto the brain Modules

Modules

In a few examples we will try to give you a In a few examples we will try to give you a

scratch of how ACT

scratch of how ACT - - R is designed. R is designed.

Visual system: there are two build in visual Visual system: there are two build in visual

modules in ACT

modules in ACT - - R referring to: R referring to:

The dorsal “where” pathway (locations) The dorsal “where” pathway (locations)

The ventral “what” pathway The ventral “what” pathway

(14)

Mapping ACT

Mapping ACT - - R onto the brain R onto the brain

Environment

Visual Buffer (Parietal) Visual Module (Occipital/etc)

Modules Buffers

Production

rules

(15)

Mapping ACT

Mapping ACT - - R onto the brain R onto the brain

As for the visual system, other modules As for the visual system, other modules

have been designed to match specific have been designed to match specific

brain areas:

Manual buffer Manual buffer = motor and = motor and somatosensory somatosensory cortical areas

cortical areas

Goal buffer Goal buffer = = dorsolateral dorsolateral prefrontal cortex prefrontal cortex DLPFC

DLPFC

Retrieval buffer Retrieval buffer = = ventrolateral ventrolateral prefrontal prefrontal cortex VLPFC (long

cortex VLPFC (long - - term declarative memory) term declarative memory)

(16)

Mapping ACT

Mapping ACT - - R onto the brain R onto the brain

Environment

Visual Buffer (Parietal) Visual Module (Occipital/etc)

Modules Buffers

Production rules

Manual Buffer (Motor)

Manual Module (Motor/Cerebellum) Retrieval Buffer

(VLPFC) Goal Buffer

(DLPFC) Intentional Module

(not identified) Declarative Module (Temporal/Hippocampus)

(17)

Mapping ACT

Mapping ACT - - R onto the brain R onto the brain Production rules

Production rules

The basal ganglia are thought to The basal ganglia are thought to

implement production rules in ACT

implement production rules in ACT - - R: R:

Striatum: corresponding with cortical areas, Striatum: corresponding with cortical areas, responsible for

responsible for patter recognition patter recognition

Palladium: inhibitory component, performs Palladium: inhibitory component, performs conflict

conflict - - resolution function resolution function

Thalamus: projects to all major cortical areas, Thalamus: projects to all major cortical areas, controls

controls execution of production actions execution of production actions

(18)

Mapping ACT

Mapping ACT - - R onto the brain R onto the brain Production rules

Production rules

Environment

Productions (Basal Ganglia)

Retrieval Buffer (VLPFC) Matching (Striatum)

Selection (Pallidum) Execution (Thalamus) Goal Buffer

(DLPFC)

Visual Buffer

(Parietal) Manual Buffer (Motor)

Manual Module (Motor/Cerebellum) Visual Module

(Occipital/etc) Intentional Module

(not identified) Declarative Module (Temporal/Hippocampus)

Production

rules

(19)

ACT ACT - - R Architecture R Architecture

Environment

Retrieval Buffer (VLPFC) Matching (Striatum)

Selection (Pallidum) Execution (Thalamus) Goal Buffer

(DLPFC)

Visual Buffer

(Parietal) Manual Buffer (Motor)

(Occipital/etc) Intentional Module

(not identified) Declarative Module

(Temporal/Hippocampus)

Declative

memory procedural

procedural memory

“pattern-

matcher” modules

buffers

Chunk

Chunk Chunk

Chunk

(20)

The modules The modules

There are two types of modules:

memory modules memory modules . .

declarative memory declarative memory

procedural memory procedural memory

perceptual

perceptual - - motor modules motor modules

take care of the interface with the simulation of the take care of the interface with the simulation of the real world (visual and the manual modules).

real world (visual and the manual modules).

(21)

chunks chunks

chunks = chunks =

units of declarative knowledge units of declarative knowledge

represent things remembered or perceived represent things remembered or perceived

example:

2+3=5 2+3=5

Boston is the capital of Massachusetts Boston is the capital of Massachusetts

there is an attended object in the visual field there is an attended object in the visual field

... ...

(22)

Chunk

chunks: examples chunks: examples

(CHUNK-TYPE integer value)

(CHUNK-TYPE addition-fact addend1 addend2 sum)

{ATTRIBUTES}

DEFINITION

INSTANCE

TYPE

one way to model the fact:

one way to model the fact: 2+3=5 2+3=5

(CHUNK-TYPE integer value)

(CHUNK-TYPE addition-fact addend1 addend2 sum) (CHUNK-TYPE integer value)

(CHUNK-TYPE addition-fact

addend1 addend2 sum)

(three

isa

integer

value 3)

^NAME

(CHUNK-TYPE integer value)

(CHUNK-TYPE addition-fact addend1 addend2 sum)

(three

isa

integer

value 3)

(23)

(CHUNK-TYPE integer value)

(CHUNK-TYPE addition-fact addend1 addend2 sum) (three

isa integer value 3) (four

isa integer value 4) (seven

isa integer value 7) (fact3+4

isa addition-fact addend1 three addend2 four sum seven)

chunks: examples chunks: examples

reference to other chunks

(24)

ADDITION-FACT

FACT3+4

ADDEND1 SUM

ADDEND2 THREE

FOUR

SEVEN

isa

INTEGER

isa

VALUE VALUE

3 7

isa

VALUE

4

chunks: examples

(25)

Fact:

Encoding:

(Chunk-Type proposition agent action object)

The cat sits on the mat.

proposition

action cat007

sits_on

mat isa

fact007

agent object

(Add-DM (fact007

isa proposition agent cat007 action sits_on object mat) )

chunks: examples

(26)

Fact:

The black cat with 5 legs sits on the mat.

Chunks

(Chunk-Type proposition agent action object) (Chunk-Type cat legs color)

(Add-DM

(fact007 isa proposition agent cat007 action sits_on object mat) (cat007 isa cat

legs 5

color black) )

proposition

action cat007

sits_on

mat isa

fact007

agent object

cat

isa

color 5

black legs

chunks: examples

(27)

chunks: examples chunks: examples

animal

fish

shark salmon canary ostrich

bird moves skin

gills swims

swims

dangerous edible

swims

yellow sings

can‘t fly tall

flies wings

(28)

productions productions

Procedural knowledge Procedural knowledge

to achieve a given goal:

processes processes

skills skills

production = production =

unit of procedural knowledge unit of procedural knowledge

condition- condition - action rule that “fire” when the action rule that “fire” when the conditions are satisfied and execute the conditions are satisfied and execute the

specified actions.

Environment Productions (Basal Ganglia)

Retrieval Buffer (VLPFC)

Matching (Striatum) Selection (Pallidum) Execution (Thalamus) Goal Buffer (DLPFC)

Visual Buffer (Parietal) Manual Buffer (Motor)

Manual Module (Motor/Cerebellum) Visual Module (Occipital/etc)

Intentional Module

(not identified) Declarative Module

(Temporal/Hippocampus)

(29)

productions productions

conditions can depend on conditions can depend on

the current goal to be achieved, the current goal to be achieved,

the state of declarative knowledge (i.e. recall of a the state of declarative knowledge (i.e. recall of a chunk)

chunk)

the current sensory input from the external the current sensory input from the external environment.

environment.

actions can:

alter the state of declarative memory alter the state of declarative memory

change goals change goals

initiate motor actions in the external environment initiate motor actions in the external environment

(30)

Structure of Productions Structure of Productions

( ( ^P ^P ^name ^name

condition part

condition part Specification of Buffer Tests Specification of Buffer Tests . .

. . delimiter

delimiter ==> ==>

action part

action part Specification of Buffer Specification of Buffer Transformations

Transformations . .

. .

) )

(31)

Example of productions

(P increment

=goal>

ISA count-from number =num1

=retrieval>

ISA count-order first =num1

second =num2

==>

=goal>

number =num2 +retrieval>

ISA count-order first =num2

)

Å If the goal is

to count from

=num1

Å and a chunk has been retrieved of type count-order

where the first number is =num1 and it is followed by =num2

Å Then

Å change the goal

to continue counting from =num2 Å and request a retrieval

of a count-order fact

for the number that follows =num2 operation & buffer

(32)

Example of productions

(P find-next-word

=goal>

ISA comprehend-sentence word nil

==>

+visual-location>

ISA visual-location screen-x lowest

attended nil

=goal>

word looking )

Å no word currently being processed.

Å find left-most unattended location

Å update state

(33)

Example of productions

(P attend-next-word

=goal>

ISA comprehend-sentence word looking

=visual-location>

ISA visual-location

==>

=goal>

word attending +visual>

ISA visual-object

screen-pos =visual-location )

Å looking for a word

Å visual location has been identified

Å update state

Å attend to object in that location

(34)

Discussion Discussion

The atomic components of thought?

a Is declarative knowledge (=chunk) available in every cognitive module?

a semantic be modeled arbitrary a chunks be of any granularity

•

“pixel in visual field” vs. “Chris is standing in front of me”

Î Can timing of the computation be compared with humans?

Î Does the division of symbolic and subsymbolic processing make sense?

Î Is ACT-R just a strange kind of programming language?

(35)

Environment

Matching (Striatum) Selection (Pallidum) Execution (Thalamus)

Goal Buffer (DLPFC)

Visual Buffer

(Parietal) Manual Buffer

(Motor)

(Occipital/etc)

Declarative Module (Temporal/Hippocampus) Intentional Module

(not identified)

(36)

the perceptual

the perceptual - - motor modules motor modules

no real sensors and effectors no real sensors and effectors

the output of the visual and the input to the the output of the visual and the input to the

motor system are just modeled motor system are just modeled

the visual and manual module are most the visual and manual module are most

important ( because of many computer important ( because of many computer

tasks, involving scanning the screen, tasks, involving scanning the screen,

typing, moving the mouse...)

(37)

the Act

the Act - - R visual system R visual system

visual location module

“where”

-> dorsal stream

visual object module

“what”

->ventral stream

visual system

Production P Request:

•constraint a

•constraint b

•...

Response:

location meeting those constraints

e.g. - screen x lowest - color: red

- screen-y-greater-than 153 ....

Æ leftmost word

Æ red object ( among green ones; supports experimental data from visual “pop-out- effects”

chunks

(38)

the Act

the Act - - R visual system R visual system

visual location module

“where”

-> dorsal stream

visual object module

“what”

->ventral stream

visual system

Production P chunk:

representation of the visual location

attention-shift to that location

(39)

Environment

Goal Buffer (DLPFC)

Visual Buffer

(Motor)

(Occipital/etc)

(not identified)

(40)

the goal module the goal module

In humans: Suppose the goal is to add 64 + 36 In humans: Suppose the goal is to add 64 + 36 Assumption: the sum is not already stored, but Assumption: the sum is not already stored, but

one has to go through a series of

one has to go through a series of substeps substeps to to come up with the answer and keep track of the come up with the answer and keep track of the

various partial results ( e.g. sum of the ten digits) various partial results ( e.g. sum of the ten digits)

The goal module has this responsibility of The goal module has this responsibility of

keeping track of what these intentions are so keeping track of what these intentions are so

that behavior will serve that goal

(41)

How could the goal buffer be How could the goal buffer be

organized ? organized ?

good example for goal

good example for goal - - subgoal subgoal structures structures in problem solving: Tower of Hanoi

in problem solving: Tower of Hanoi Problem

Problem

naive human response: move the disks to naive human response: move the disks to

their ultimate location ( greedy) their ultimate location ( greedy)

but: goal

but: goal - - subgoal subgoal strategy is often strategy is often discovered during practice

discovered during practice

(42)

Tower of Hanoi

Tower of Hanoi - - problem problem

Interest of Anderson et al. (Tower of Hanoi:

Evidence for the Cost of Goal Retrieval1, Evidence for the Cost of Goal Retrieval1,

2002):

not which strategy is adopted, but what not which strategy is adopted, but what

does it tell about goal

does it tell about goal - - subgoal subgoal interaction interaction what are the “cognitive costs” for

what are the “cognitive costs” for implementing a

implementing a subgoaling subgoaling strategy strategy

(43)

Experimental setup:

the strategy to solve the problem was given and the strategy to solve the problem was given and

trained trained

task: solve the problem as fast as possible task: solve the problem as fast as possible

formulate a goal by clicking the disk on the source formulate a goal by clicking the disk on the source peg , then the destination peg

peg , then the destination peg

do action or post goal do action or post goal

time between actions and accuracy of moves time between actions and accuracy of moves

were measured; also eye movements were were measured; also eye movements were

recorded

(44)

user interface

(45)

Strategy

Strategy - - Algorithm Algorithm

1. formulate a goal 1. formulate a goal

2. decision 2. decision

if a legal move to achieve your goal is possible, do it and skip if a legal move to achieve your goal is possible, do it and skip next step (3), otherwise post it on the

next step (3), otherwise post it on the goal stack

goal stack

3. formulate a prerequisite goal 3. formulate a prerequisite goal

if you cannot move a disk D, find the if you cannot move a disk D, find the largest disk

largest disk

that’s blocking that’s blocking the move and move it to a peg which is neither the source, nor the move and move it to a peg which is neither the source, nor

the destination peg of D the destination peg of D

4.Try again 4.Try again

go back to #2 to see whether you can achieve your last goal go back to #2 to see whether you can achieve your last goal posted

posted

5. Repeat the process 5. Repeat the process

go back to step one, until all disks are at their final position go back to step one, until all disks are at their final position

(46)

example demo example demo

A B C

Do it ! Post it !

GOAL STACK

C B C

(47)

results:

(48)

results results

participants are slower at those points where participants are slower at those points where

they must retrieve a goal, and are more slower they must retrieve a goal, and are more slower

the longer ago it was posted the longer ago it was posted

the accuracy data suggests that participants are the accuracy data suggests that participants are

forgetting their goals forgetting their goals

The tendency to inspect the goal stack increases The tendency to inspect the goal stack increases

dramatically at those retrieval points dramatically at those retrieval points

Æ Æ goal retrieval seems to be the major factor goal retrieval seems to be the major factor limiting performance in this task

limiting performance in this task

(49)

But what has Act-R to do

with this

experiment?

(50)

Act Act - - R R

Act Act - - R was used to model this task R was used to model this task

Act Act - - R 4.0 had a “perfect memory” goal R 4.0 had a “perfect memory” goal stack on which all goals can be stored stack on which all goals can be stored

perfectly and accessed without any perfectly and accessed without any

retrieval time costs retrieval time costs

BUT: data shows clear goal limitations!

Altman &

Altman & Trafton Trafton : memory for goals might : memory for goals might behave like any other memory and be

behave like any other memory and be subject to forgetting

subject to forgetting

(51)

new Act

new Act - - R model R model

getting rid of the goal stack ! getting rid of the goal stack !

relies on

relies on ACt ACt - - Rs Rs general declarative general declarative memory to store goals

memory to store goals

in Act in Act - - R each chunk has a base R each chunk has a base - - level level

activation that increases each time the chunk activation that increases each time the chunk

is used and decreases with lack of use is used and decreases with lack of use

Gaussian retrieval Gaussian retrieval - - probability function over probability function over the base

the base -level activation - level activation

(52)

results results

nice fit !

nice fit ! nice fit !

nice fit !

(53)

conclusions of this research:

cognitive architectures like Act

cognitive architectures like Act - - R(4.0) or SOAR R(4.0) or SOAR are wrong in their assumption of a special goal are wrong in their assumption of a special goal

stack stack

goals in the

goals in the subgoaling subgoaling task are probably no task are probably no different than other sort of intentions people set different than other sort of intentions people set

goals appear to behave like other more common goals appear to behave like other more common

kinds of declarative memory and shows the kinds of declarative memory and shows the

same effects in practice and retention interval

(54)

Environment

Goal Buffer (DLPFC)

Visual Buffer

(Motor)

(Occipital/etc)

(not identified)

(55)

the buffers the buffers

ACT ACT - - R accesses its modules (except for R accesses its modules (except for the procedural

the procedural - - memory module) through memory module) through buffers

buffers

For each module, a dedicated buffer For each module, a dedicated buffer

serves as the interface with that module serves as the interface with that module

The contents of the buffers at a given

moment in time represents the state of

ACT ACT - - R at that moment. R at that moment.

(56)

the buffers the buffers

each buffer can hold a relatively small amount of each buffer can hold a relatively small amount of

information (

information ( Î Î chunk) chunk)

chunks that were former buffer contents are now chunks that were former buffer contents are now

stored in the declarative memory module stored in the declarative memory module

buffers are conceptual similar to

buffers are conceptual similar to Baddley’s Baddley’s working memory “slave systems”

working memory “slave systems”

the central cognitive system can only sense the the central cognitive system can only sense the content of the buffers

content of the buffers

the content of the chunks can only be accessed by the content of the chunks can only be accessed by the highly specialized modules

the highly specialized modules

(57)

the buffers the buffers

the most important buffers in Act

the most important buffers in Act- -R are: R are:

Goal Buffer

Goal Buffer

keeps track of one’s internal state in solving a problem keeps track of one’s internal state in solving a problem preserves information across production cycles

preserves information across production cycles

Retrieval Buffer

Retrieval Buffer

holds information retrieved from long

holds information retrieved from long--term declarative memoryterm declarative memory seat of chunk activation calculations

seat of chunk activation calculations

Manual Buffer

Manual Buffer

responsible for control of hands responsible for control of hands

Visual “where” Buffer

Visual “where” Buffer location

location

Visual “what” Buffer

Visual “what” Buffer visual objects

visual objects

attention shifts correspond to buffer transformations attention shifts correspond to buffer transformations

(58)

Environment

Goal Buffer (DLPFC)

Visual Buffer

(Motor)

(Occipital/etc)

(not identified)

(59)

Pattern matcher Pattern matcher

The pattern matcher searches for a The pattern matcher searches for a

production that matches the current state production that matches the current state

of the buffers of the buffers

Only one such production can be Only one such production can be executed at a given moment

executed at a given moment

That production, when executed, can That production, when executed, can

modify the buffers and thus change the modify the buffers and thus change the

state of the system state of the system

Thus, in ACT

Thus, in ACT - - R cognition unfolds as a R cognition unfolds as a succession of production firings

succession of production firings

(60)

Making Choices: Conflict Resolution Expected Gain = E = PG-C

Probability of choosing i =

∑

j

j t E

i t E

e e

/

P =

Successes = α + m Failures = β + n

Successes

Successes + Failures

P is expected probability of success G is value of goal

C is expected cost

t reflects noise in evaluation and is like temperature in the Boltzman equation

α is prior successes

m is experienced successes β is prior failures

n is experienced failures

Production selection

(61)

Outlook Outlook

What is ACT

What is ACT - - R used for? R used for?

(62)

What is ACT

What is ACT - - R used for? R used for?

ACT ACT - - R has been used successfully to R has been used successfully to create models in domains such as:

create models in domains such as:

learning and memory, learning and memory,

problem solving and decision making, problem solving and decision making,

language and communication, language and communication,

perception and attention, perception and attention,

cognitive development, or cognitive development, or

individual differences individual differences

…but not only in tasks of cognitive

…but not only in tasks of cognitive psychology ACT

psychology ACT - - R has applications… R has applications…

(63)

What is ACT

What is ACT - - R used for? R used for?

(64)

General Discussion General Discussion

Modularity Modularity

Fodor: “higher

Fodor: “higher- -level cognition is impossible to level cognition is impossible to encapsulated into separate components”

encapsulated into separate components”

General doubts about success of function General doubts about success of function localization in brain imaging research

localization in brain imaging research

(65)