Appraisal Models of Others

The Merriam-Webster dictionary defines a generic appraisal as being an evaluation of worth, significance, or status; an estimate.  An appraisal, as such, is part of a process that assesses the meaningfulness and relevance of ongoing and previously experienced perceptual (sensory-related) and/or emotional (reward or aversive-related) schematic experience.  Accordingly, a social appraisal is an appraisal that assesses the personal meaningfulness of a social transaction and interaction between others and between oneself with another/others.

There are many other appraisal theorists that have basically supported Scherer’s approach and conceptualization of the appraisal process.  For instance, Brian Parkinson (2001) suggested that emotions impart a stimulus’s importance and relevance.  An appraisal functions somewhere on either side of emotion; it has causative components that first underlie the later expression of emotion and descriptive components that serve to describe an emotional state by its core relational theme (CRT) (p. 180).  According to Parkinson’s conceptualization, statements like “I feel loss”  and “I feel helpless” serve to measure and conceptualize an earlier expression of emotion (like sadness).  This is in contrast to this author’s prior section’s conceptualization of a  primary appraisal as being a reactive conceptualization of the impact of environmental outcome on the self, a descriptor of meaningfulness of an outcome, and an earlier precursor of later emotional expression, like sadness.  Parkinson conceptualized the nature between emotion and appraisal.  He recognized the interdependence between  both emotion and appraisal within context of a temporal model.

Smith & Kirby (2001) also saw appraisals as being elicitors of emotions (p. 121) and as inherently relational, i.e. relating to one’s individualized needs, goals, beliefs and values (p. 124).  Accordingly they concluded that there are two distinct modes underlying appraisal processes.  The first is associative.  The associative mode is fast, automatic, outside focal awareness and requires minimal attentional resources for processing sensory input.  These descriptive qualities characterize primary appraisals .  The second mode is reasoning.  It is slow, deliberate, requires considerable attention and focal awareness, and employs constructive cognitive processes evidenced in mature and complex problem-solving.  These qualities describe secondary appraisals, which have a role in shaping and organizing emotionally-related physiological activity and serve to intensify any emotional reaction.

They also identified seven appraisal components.  Two of seven appraisal components relate to motivation, i.e. motivational relevance (or the rated importance of an object or situation) and  motivational congruence (or the nature of the relevance or consistency of a situation to one’s existing goals, e.g. one’s affiliative needs can be addressed by satisfying outcomes relating to affiliation not achievement). The other four components related to problem-focused coping potential (or an assessment of one’s ability for assessing conditions and for one’s ability for reaction), emotion focused potential (or one’s ability and one’s adjustment to a situation), self-accountability (or one’s assessed responsibility in a situation’s outcomes), and future expectancy (or one’s prognosis of events).  Please reference the depiction below, which summarizes the above.

The first two components in the above diagram relate to primary appraisals and the second five components relate to secondary appraisals.  Each secondary appraisal component relates to cognitive activity, which had originally been associated with certain emotions.  For example, fear serves to motivate one’s coping potential for engaging in planning (i.e. appraising and problem-solving) to protect oneself and support and foster the development of an action plan to escape and protect from threat.  One adjusts accordingly to changing environmental conditions as they offer safety or continued threat, i.e. to discontinue escape in the former or continue or intensify escape response in the latter.  Sadness motivates one’s coping potential for engaging in planning that will separate oneself from thwarted goals for wanting something and not getting it as well as for developing an action plan that will seek support for dealing with loss.  One adjusts to changing environmental conditions as they offer or fail to offer support in dealing with sadness.  In these situations emotions, like fear and sadness, served to stimulate cognitive activity for later preparatory coping action.

Smith & Kirby offered an appraisal conceptualization that is simple, clearly delineated, and carefully conceived of appraisal processes in a temporal, step-wise manner.  Reference the primary appraisal and secondary appraisal sections of this website.

Marnier & Laird (2006) indicated that the appraisal process was derived from a series of occurrences and interactions.  As Lazarus, Marnier & Laird saw a relationship between appraisals and perception and appraisals, emotion, and coping.  In their PEACTIM model they identified three types of appraisals, which develop in response to different environmental occurrences.  The first appraisal was automatic in nature (like the primary appraisal) and was generated in response to the nature of a stimulus’s novelty, intrinsic pleasantness, and causality.  The second type of appraisal was deliberate and required (like the primary appraisal).  It was goal/need relevant, monitored discrepancy, and supported outcome probability.  The third type of appraisal was deliberate and optional (like the secondary appraisal).  This third type of appraisal was goal and need-conductive and facilitated coping potential.  Appraisals, as such, were “inherent” for understanding and comprehending one’s experience and surroundings at many points during the emotion process. 

According to Marnier & Laird’s (2006) appraisal model comprehension was seen as critical for integrating cognition and emotion.  The ability for comprehension of an event(s) was not limited to one context, by working memory, to one input (but many different input), and to one point in time (but to many different cumulatively experienced periods of time).  Comprehension was evidenced spontaneously at the onset of each event and could link through an associative mechanism many different events together, and supported and was fundamental for predictive analysis.  In response to ambiguity the preferred and selected representation was based on the best goodness-to-fit model.  Two levels of comprehension error recovery existed; one was characterized by mental processes (or schema structures), which were accurate and necessitated simple and uncomplicated updating and the second was associated with schema structures, which were disparate and required complex (schematic) reorganization.  This model supported the development of many different diverse appraisals spanning many different contexts. 

In 2007 Marnier & Laird differentiated between perceived and active appraisal frames.  Perceived appraisals, like primary appraisals, were linked with feelings and active appraisals, like secondary appraisals, were linked with emotions.  Both supported the later expression of mood.  Their model differentiated between inputs, input range, linearity, symmetry of mood and non-mood, symmetry of opposite values of mood (or opposite moods like happy versus sad mood), the symmetry of all values of mood and emotion that impact on feeling, and the impact of intensity on feelings, emotions, and moods.   

Marnier & Laird (2008) and Marnier, Laird, & Lewis (2008) later updated PEACTIM model.  They assimilated and integrated Scherer’s characterization of appraisal variables of suddenness, unpredictability, intrinsic, pleasantness, and relevance into their PEACTIM model.  In the PEACTIM model “stimuli are perceived and encoded so cognition can work with them”.   Behaviors and processes relating to attending to stimuli, comprehending their meaningfulness, and intending and decoding motor regimes supported later physical action production (2008, p. 115) mediating later environmental change.  The PEACTIM model’s approach was sequentially linear, i.e. each variable was mediated by a previously occurring dependency.  The subject could derive meaning from environmental occurrences in response to relevant perceptual information. Analysis of perceptual information helped to enchance comprehension of an ongoing event.  With comprehension, a probability of future occurrences could later be speculated and predicted.  When a prediction was assessed as correct, i.e. when events matched to expectation, the current line of reason and analysis was judged as correct.  The match between expectation and event outcome signaled little or no need for modifying a current strategy.  When a prediction was judged incorrect, percepts and subsequent events needed reinterpretation and, when appropriate, strategies needed modification. 

Appraisals, as such, served to function in a critic role (Marnier & Laird, 2008); the subsequent reactive appraisal-induced valenced feeling or sensation that was generated provided a reward signal.  (This approach had been previously conceptualized by Sutton & Barto (1999) in Chapter 6.6, The Actor Selects Actions and the Critic Evaluates the Effectiveness of those Actions at Achieving One’s Goals.  According to both authors this reward signal provided immediate feedback on one’s progress towards reaching a reward outcome.  Because the critic resided in the organism’s internal environment, each organism was considered to be capable of generating his/her own reward or intrinsic reward.  

The relationship between appraisals and the environment was thus judged to be interdependently dynamic, i.e. in response to changing dynamic conditions (i.e. acquiring or not obtaining reward) appraisal values changed and, in response to changing appraisal processes, perceptions of the environment also changed.  The agent typically made predictions about future reward outcomes based on probability of occurrence derived from accumulated prior occurrences and outcomes.  It was assumed that an agent with mood learned faster than an agent without mood.  An agent with mood carries a summary of all recent emotions and establishes value estimates of the external and internal environments.  

Marnier, Laird, & Lewis (2008) also supported their PEACTIDM model with the Soar model.  The Soar model suggested that the environment provided associated sensory stimuli, which inevitably lead to a subject’s later perceptions.   Event-related perceptual components were thus encoded and placed in short term memory awaiting additional percepts, experiences, and input.  As noted in earlier publications (e.g. Squire, 2004) procedural, semantic, and episodic components, were eventually transferred to long term memory from short term memory processes.  According to Marnier, Laird, & Lewis (2008) the Soar model primarily measured the procedural memory component of short and long term memory.  The Soar decision cycle progressed sequentially and accordingly and was based on the following progression.  Environmental input served to direct an organism’s procedural learning and behavior via attention processes.  An organism typically elaborated, proposed, compared this input with that which had already been known, later decided and applied this information during planning, and acted on subsequent decision-making. 

Marnier & Laird’s appraisal model drew from many prior authors, i.e. Lazarus and Scherer.  They conceptually elaborated on these models in their development of their PEACTIDM and Soar models and offered a window into the components underlying decision-making processes.

Also like Lazarus, Hudlicka (2004) evaluated the interactions between appraisals and emotion.  In her cognitive architecture the earliest processing level occurred at the sensory pre-processing or automatic level.  At this level attending processes analyze incoming sensory data, perform situational analyses, generate expectation (or a rudimentary model of the meaning of this data), and generate affect appraisal, goal selection, action selection, the development of mental constructs, etc (Scherer, 2001).   This multi-level model also had many different stages.  The earliest construct, the automatic appraisal (like the primary appraisal) processed unexpectedness, novelty, pleasantness, and calculated valence value for the later expression of emotion.  The later construct, the expanded appraisal, considered the congruence of current situations with one’s goals for well-being and coping.  Like secondary appraisals, the expanded appraisal was a response to given emotions and produced a “vector of intensities for each of the four (basic) emotions.”  Hudlicka evaluated these variables in her MAMID architecture and found that affective states were indeed dynamically generated by an affect appraisal.

Also like Lazarus and others, Gratch & Marsella (2004) suggested that appraisals embodied and characterized certain aspects of the person-environment relationship. They saw cognitions as being interpretations of  environmental events within context of one’s goals and desires and appraisals as being abstractions of this information, which guide later behavior.  Both delineations of cognitions and appraisals were suggestive of early and later primary appraisal processes, as suggested in the primary appraisal section, and were capable of eliciting later emotion and coping cognitions , suggesting a causative relationship between the former and the latter (Marsella & Gratch, 2009).   Accordingly, emotion-eliciting coping cognition later served to repair and sustain the person-environment relationship by problem-focused coping, i.e. implementing an action plan to either alter environmental events or to seek the advice or support of another.  Emotion-focused coping sought to change and modulate how one has been coping with conflicting person-environment relationships through emotional and behavioral restraint, denial, mental and behavioral disengagement, and other means of disengagement (Gratch & Marsella, 2004-p. 7).

They (Gratch & Marsella, 2004; Marsella & Gratch, 2009) also identified and integrated many appraisal variables (also referred to as appraisal components below) into their model, such as relevance (Scherer, 1984), desirability (Leventhal & Scherer, 1987), causal attribution (Scherer, 1984), likelihood (or probability) (Scherer, 1984), unexpectedness (novelty) (Leventhal & Scherer, 1987), urgency, ego involvement (Lazarus, 1999) and coping (Lazarus, 1999) (which was composed of controllability (Scherer, 1984, Leventhal & Scherer, 1987) and power, as well as changeability and adaptability) (Lazarus, 1999)) (Gratch & Marsella, 2004-p. 6).  They delineated between different types or manners of coping, such as attentional (perceptual), belief (cognitive), desire (motivation), and intentional (cognitive-action) related coping methods (Marsella & Gratch, 2009).  They noted that people tended to appraise how evolving events could impact (themselves and) others.   Appraised cognitions, as such, were based on the impact of and one’s responses to prior experiences and tended to guide the selection of later social (interaction) behaviors.

In developing their model, Gratch & Marsella (2004) conceptualized different model parameters.  They reasoned that perceptual (input) and inferential (output) processes have the ability for later altering the nature of an actor’s or subjects interpretation of a situation (Marsella & Gratch, 2009).  They believed that any model should be composed of cognition, appraisals, and coping; all needed to be tightly coupled and monitored.  Relevance and desirability needed representation in order to monitor preferences.  Causal attribution or inferences (imputed onto the environment) (Marsella & Gratch, 2009) needed monitoring for causality and agency. Likelihood, unexpectedness and changeability needed representation in order to monitor causal factors that had influenced events, possible outcomes and interactions underlying their expression.

Like Lazarus and others, Marsella & Gratch (2009)) suggested that the environmental response to one’s earlier actions generated a causal interpretative response, which soon thereafter spawned appraisal variables (appraisal frames and affective states).  Appraisal variables served to direct an organism’s responses to environmental stimuli, as opposed to associating to perceptual aspects to environmental stimuli.  After this point coping responses were allowed expression and were evidenced in control signals of explanation, belief formation, and planning.  All the above were integrated into their EMotion & Adaptation (EMA) model of appraisal and coping, which constructed and maintained a causative interpretation of an internal relationship model.  This model had been composed of one’s responses to one’s environment and was embodied in one’s cognitions, beliefs, biases, desires, plans, etc. The EMA model began with an agent’s interpretation of the “agent-environment relationship” as a (current) causal interpretation of the agent and the state of the world as a conjunction of propositions. Underlying the expression of states and actions was an agent’s beliefs, desires and intentions in a given situation, which emanated from acquired knowledge of one’s world during interpersonal social interactions.  The EMA model was composed of different appraisal frames, which related to one’s causative interpretation of events.  It mapped individual appraisal frames onto expressions of emotion, cumulatively collected and compiled emotion into a mood, and adopted a coping strategy in response to the above. According to Marsella & Gratch (2009) the EMA model provided an efficient and rather quick reference for assessing the nature of perceptual and inferential output, irrespective of the output’s temporal delays with deliberate processing as well as temporal brevity with automatic processing.  They believed any good model should capture and model the unfolding of different processes, elicit preferences, make causal attributions, supply information on predictability, account for urgency and accountability, acknowledge social power, examine external and internal conditions for facilitating adaptability, and the nature of ego involvement (which will impact urgency).

Gratch & Marsella model helped to conceptualize the role of inital causal interpretation (meaningfulness) in producing appraisal and the role of this appraisal in both triggering emotion and generation of appraisal frames.  Both were deemed important for the later generation of coping behaviors.  Though explanation, belief formation and planning also emerged from causal interpretation and concurrently occurred with coping behaviors, the relationship between and among all is unclear and seemingly parallel (Gratch & Marsella, 2004-p. 277).  This model may be helpful in understanding the impact and emergent responses to obtaining and not acquiring social reward, for an appraisal (describing meaningfulness (desireablility, probability, causal attribution, etc. ) is often assessed and generated by any organism.  Areas of this nature will be further discussed in the development of this site’s model.

Reisenzein (2001) organized and conceptualized a structural appraisal theory of emotion. It was composed of a set of appraisal processes  {Α1, Α2, Α 3, …, Αm} and event-related dimensions of valence, probability, agency, and legitimacy, a set of objects or  {o 1, o 2, o3, …, om }, which were sources of the appraisal development, a set of emotions or {Ε1, Ε2, Ε 3,…, Εm} and a relation, (R), which mapped each appraisal and its dimensions onto objects or emotions.

Reisenzein’s work presumed that salient aspects of emotional-eliciting events included appraisals and event-related dimensions of valence, causal locus, and probability (Reisenzein & Hofmann, 1993). Accordingly, dimensions of valence, probability, agency or legitimacy differentiated between different appraisals.  For instance, an appraisal’s VALENCE could be further delineated by {positive, negative or neutral}. Because an appraisal was considered to be an evaluation of some source, object, or outcome, this relationship might be reflected as VALENCE (o) = {positive, negative, …, neutral}. If the selected valence was negative, “an event, o , might be appraised by the person as negative”, VALENCE (o) = negative or VALENCEneg . According to Reisenzein, a relation, R , would map all process sequences (from primary appraisal through to secondary appraisal) onto one another to form a coherent memory.

Please note that Reisenzein’s temporal ordering of concepts, i.e. appraisals and other dimensions of outcomes of emotional-eliciting events, could be contrasted with Lazarus’s appraisal theory ,which cites the following order, (primary appraisal), outcome, primary goal-oriented appraisal, emotion, and secondary coping appraisal.

Staller & Petta (1998) developed the TABASCO model, a 3T mantis architecture (simulated predator-and-prey scenario) and appraisal model for the situated agent in a virtual Artificial Intelligence (AI) environment.

As noted in the primary appraisal section,  the earliest components of the primary appraisal process center on an appraisal of an event (i.e. an evaluation of sensory, perceptual input from the environment). In the TABASCO model perception and appraisal components were defined as initiating situations that were either goal-relevant events, actions of agents, or attractive or unattractive objects, whose meaningfulness was afforded (i.e. affordances) and whose benefit was conferred by the situated agent.  Accordingly, the direct perception of social affordances involved the process of cognizing, a process whereby information had been derived from sources outside a situated agent’s consciousness.  Perceptual processing occurred at three levels, a sensory level (or sensorimotor level) for sensory feature detection, a schematic level for representing semantic networks in memory, and a conceptual level for abstract reasoning and inference about self and other (please reference: Leventhal, 1984; Scherer & Leventhal, 1987; Scherer, 2001).  Perceptual and appraisal components gave rise to perceptual and action components, which lead to the spontaneous (development of and) execution of flexible alternative coping motor programs at the conceptual level and the later expression of emotions or pre-emotions. An action monitoring component monitored “planning and execution processes.” This component forwarded updated findings to both the perceptual and appraisal components, where they were reintegrated within the whole appraisal process to modify its structure.  Therefore initiating situations could be understood as triggering sensory evaluations as well as later needed action tendencies for state readiness.  State readiness was needed for ensuring that resources met task-related requirements and supported the later expression of certain emotions or pre-emotions, respectively. This simulated world was represented by a triggering event’s emotional processing flows, called an “Emotion Eliciting Condition”, which was later mapped onto a specific emotion.  Staller & Petta indicated that these aspects were at the heart of TABASCO’s emotional processing.  The TABASCO model recognized that appraisals emanated from both perceptual and cognitive-motor sources.  As emotion eliciting internal conditions, they could have the ability for later impacting emotional and related emotional behavioral expression.

Pezzula & Calvi’s (2007) model was developed around specifically targeted behaviors.  It too utilized a robotic mantis architecture, AKIRA, which implemented a simulated predator-and-prey scenario.  It was composed of two systems, a motivational system (drives) and sensorimotor system (perceptual-motor schemas).  Accordingly, the motivational system’s (primary) drives were generated from states like hunger, desire for sex, fatigue and fear. Each drive was allowed temporal expression in a predetermined manner and was supported with the expression of cognitive activity of schemas.  Motivations as such had a modulatory impact on specific behavioral responses by influencing the sensorimotor system’s future resource allocation. Numerous, dynamically-associated schemas formed an internal predictive world model, which was required for evaluating possibilities for meeting behavioral goals and desired states (Butz, Herbort, & Pezzulo, 2008). One type of schema, a perceptual schema, was noted as fundamental for realizing avoidance behaviors and evidenced in sensory detection of prey, predators, mates, hiding places, and obstacles and escape strategies from a predator.  On the other hand, the sensorimotor system’s motor schemas were essential for realizing active approach behaviors and evidenced in motor-generated searching for prey, predators, potential mates, hiding places, and in detecting obstacles.  Perceptual and motor schemas together served to guide behavioral responses in the same way that scripts or action schemas guided behavioral selection (Horowitz, 1988; Nelson, 1986; Piaget, 1981; Steiner, 1974).

According to Pezzula & Calvi (2007), the initial detector of perceptual schemas and affordances of an estimated position were represented by < xe, ye, ze >. The next estimated position in the temporal sequence, or fixation point, was added into perceptual schemas and affordances and represented accordingly, < xnƒ, yn ƒ, zn ƒ >.   The fixation point was a function of xe, suggesting that,  xnƒ, the fixation point, was a function of xe, the prior estimated position. Based on information previously gathered by the agent, the next estimated position was represented by < xnρ, yn ρ, zn ρ > and so on.

The abilities for being able to predict the reliability of future agent positions (or the error in prediction) as well as select certain schemas were represented by  1 − | | < xe, ye, ze > − < xρ , yρ, zρ > | |  or the earlier estimated position less the predicted estimated position.  In addition, predicting a stimulus’s future position required sensory and state anticipation and prior experience with a stimulus (Butz, Sigaud, Pezzulo & Baldassarre, 2007).  Sensory anticipation supported goal-related perceptual processing and schematic development; state anticipation supported behavioral decision making and related schema development.  Payoff anticipation provided an incentive for completing the required task.  These aspects of the model are reminiscent of Sutton & Barto’s discounting and outcome representing temporal difference.

According to the AKIRA model, a forward model,  <xp, yp, zp>3, received all prior relevant or irrelevant sensory input for developing a later working model capable of anticipating outcomes, and for actuating and predicting the impact of an agent’s behavior on self (i.e. sensations) and others (Pezzulo & Calvi 2007; Butz, Herbort, & Pezzulo, 2008; Pezzulo & Castelfranchi, 2009).  An inverse model , < xnƒ, yn ƒ, zn ƒ >,  reflected the schematic perceptual representation of a fixation point.  < xnρ, yn ρ, zn ρ > represented the next and predicted bodily position and later actions.   For instance, with regards to the eight perceptual schemas noted above, the inverse model would support the eliciting of needed motor commands for minimizing the distance between occurrence and predicted, i.e. < xe, ye, ze > and < xp, yp, zp >.

In earlier phases, the agent used both inverse and forward models to emerging schemas, which served to screen out irrelevant information; in later phases all schemas worked together as an integrated whole, suggesting the effects of successful learning.  During later phases the forward model received the terminal motor command that had been implemented and elicited a reactive prediction < xnp,  ynρ,  znρ >3 .  Like the TABASCO’s action model, the AKIRA model integrated monitoring activity and previously selected schemas into a unique internal architecture.  It guided goal-directed behavior through the interplay of monitoring, predicting, and controllability (Pezzulo & Calvi 2007; Pezzulo & Castelfranchi, 2009).

In later conceptualizations Pezzulo and colleagues (2008) sought to evaluate and elaborate on prediction and anticipation.  Prediction was viewed as a "representation of a particular future event" (p. 25).  Accordingly prediction served to represent, enhance decision-making, focus the organism, had different time scales, and was generalizable.  The authors recognized that different types of predictions of the environment or properties of objects and of internal states had different meanings for the organism.  Predictions as such were assessments of current conditions and imputed with inferences and probabilities of future environmental events.  Anticipation, like prediction, was seen as assessments of current conditions, which were imputed with inferences and probabilities; however, anticipation was an organism's internal response to the goal-outcome experience.  These concepts are important for understanding the emergence and development of appraisals and later generation of emotion.

These are very important preconditions for the later expression of primary appraisals.  However, because this  model tracked very specific responses (stimulus-expectation-action) to states like hunger, desire for sex, fatigue and fear, their algorithms are not easily generalizable to secondary drives, such as motivations for affiliation, achievement, validation, control over outcomes, and sense of well-being.

Ahn & Picard’s (2006) appraisal model’s approach dynamically examined decision- making, i.e. selecting a decision from many alternatives, where d ∈D.    Their transition model was represented by some key components,  where a was the current affective state, c was the current cognitive state, d was the current decision state, c was the next cognitive state, and a was the next affective state.

Ahn & Picard's decision-making model, dt = Pr (d |c, a ) examined the probabilities of selecting a decision, d, within context of both cognitive, c, and affective, a, states.  Their cognitive model, ct+1 = Pr (c | c, d), identified how the next cognition, c, would be influenced by the current state, c, and decision, d.  Their affect model was represented by context-specific affect, a, which was one of many emotions, θ, in an affect system, ( θ Α = {0,1, … , | Α | } ), having  two probabilities of feeling bad and feeling good , aθ = (aθ,1, aθ,2): aθ,1 and aθ,2 , respectively.     Their affect model, a′ θ = Pr (v′ θ | c ', c, aθ, d), was also influenced by the valuation of reward, R0 , and the outcome response of not receiving it,  i.e. the feeling bad state with breached expectation, (vθ = 1 ) and the outcome response of receiving it, i.e. the feeling good state with obtaining it, (vθ = 2 ). Accordingly, affective probabilities were represented by a = {aθ} = (a0, a1, ∈ a | Α |) .

In their affective anticipatory reward model, reward planning was impacted by certainty/uncertainty, depending upon the sufficiency of  information available during the decision-making state, d. The nature of certainty and uncertainty during decision-making was reflected as  σ0(ct, dt),  where the cognitive state was one of many, ct ∈ C = {1,…, | C | }, and any decision was one of many available decisions, dt ∈ D = {1, ..., | D | }.   The affective anticipatory reward model was composed of the sum of all possible cognitions and their associated valuation and probabilities for developing the next cognition, i.e. the current valuation of reward as well as the current cognitive, decision, and affective states.

  

Furthermore, according to this model, the probability of selecting any decision and associated cognitive and affective state might be derived from the decision's quality, QDM, ( reference Sutton & Barto's value of a policy algorithm, Vπ) within context of all prior values of all decisions,  associated current decisions, as well as cognitive and affective states.  This was reflected by Ahn & Picard accordingly.

According to Ahn & Picard's model, a decision's value was derived from its extrinsic value and the intrinsic value inherent in affect states, when d ∈ D and a ∈A.   This is elaborated in the following equation.

The model suggests that when a decision has been implemented behaviorally, dt, the decision activates a new cognitive state, ct+1. This new cognitive state monitors the effect of decision-making on behavioral implementation and the effects of that behavioral response on the following or subsequent cognitive state.

After a decision has been behaviorally implemented, the next new cognitive state is represented by ct+1. When ct+1 ∈ C = {1, …,| C | }, it is also associated with extrinsic reward, rext .

The decision model is integrated with models relating to probability, extrinsic reward, extrinsic state and associated cognition and decision-making,  Pr, Rext ,Sext(c' |c, d). The average reward, Rmean( c ), is updated with < ct, dt, rext, ct+1>, which represents all current cognitive and decision-making components, the impact of extrinsic reward on these processes, and the next cognitive state.  The best or preferred choices, d1 and d2 , typically underlie later cognitive-induced action, which had been influenced by an earlier cognitive state, c, and decision, d. This may be formalized in ƒ(c, d, c). 

The average reward (Rmean) for either or both decisions, d1and/or d2, has a generalizing effect on all cognitive and affective states and decisions when accounted for across trials.  This generalized reward state for each decision made may be interpreted as a sense of well-being at any point in time. 

The Ahn & Picard model attempts to capture the many different components underlying appraisal expression.  It accounts for stimulus-associated cognition and later decision-guided action, affect, and the role of reward expectation in facilitating and in determining the nature of emotional expression and task dimensions, such as probability.  Their model accounts for the impact of new information (e.g. responses to prior cognition, decision-making and related actions, affect, stimulus reactions, etc.) on an existing cognitive, emotional, and behavioral structure.

References

Ahn, H. & Picard, R. W. (2006). Affective-cognitive learning and decision making: The role of emotions. The 18th European Meeting on Cybernetics and Systems Research (EMCSR 2006). April 18-19, 2006, Vienna, Austria.

Butz, M.V., Sigaud, O., Pezzulo, G. & Baldassarre, G. (2007). Anticipations, brains, individual and social behavior: An introduction to anticipatory systems. In: M.V. Butz, O. Sigaud, G. Pezzulo, & G. Baldassarre (Eds.), Anticipatory Behavior in Adaptive Learning Systems: Advances in Anticipatory Processing. Springer LNAI 4520.

Butz, M.V., Herbort, O., & Pezzulo, G. (2008). Anticipatory, Goal-Directed Behavior. In: Pezzulo, G., Butz, M.V., Castelfranchi, C. & Falcone, R. (Eds.), The Challenge of Anticipation: A Unifying Framework for the Analysis and Design of Artificial Cognitive Systems (pp. 85-113). Springer LNAI 5225.

Gratch, J. & Marsella, S. (2004). A domain-independent framework for modeling emotion, Journal of Cognitive Systems Research, 5(4), 269-306.

Lazarus, R.S. (1999). Stress and emotion: A New Synthesis-2nd Edition (2006). New York: Springer Publishing.

Leventhal, H., & Scherer, K (1987).  The relationship of emotion to cognition: a functional approach to semantic controversy.  Cognition & Emotion, 1(1), 3-28.

Marinier, R., & Laird, J. (2006) A cognitive architecture theory of Comprehension and Appraisal Agent Construction and Emotions. (hosted at the 18th European meeting on Cybernetics and System Research), Vienna, Austria. 

Marinier, R., & Laird, J. (2007). Computational modeling of mood and feeling from emotion.  Cognitive Science Conference in Nashville, TN.

Marinier, R., & Laird, J. (2008). Emotion-driven reinforcement learning. Cognitive Sciences, Washington D.C. Paper Presentation.  Marinier, R., Laird, J., & Lewis, R. (2008) A computational unification of cognitive behavior and emotion. Cognitive Systems Research, 10(1), 48-69.

Marsella, S. & Gratch, J. (2009). EMA: A Model of Emotional Dynamics. Journal of Cognitive Systems Research, 10(1), 70-90.

Nelson, K. (1986). Event knowledge and cognitive development.  Ed.: K. Nelson. In:  Event knowledge: structure and function in development, pp. 1-19.  Hillsdale, New Jersey: Lawrence Erlbaum Assoc., Publishers.

Parkinson, B. (2001). Putting appraisal in context.  In: K.R. Scherer, A. Schorr, & T. Johnstone (Eds.). Appraisal processes in emotion: theory, methods, and research (pp. 173-186), New York: Oxford University Press.

Pezzulo, G., Butz, M.V., Castefranchi, C. (2008).  The anticipatory approach: definitions and taxonomies.  In: G. Pezzulo, M.V. Butz, C. Castelfranchi, & R. Falcone (Eds.). The challenge of anticipation: a unifying framework for the analysis and design of artificial cognitive systems (pp. 23-43).  Springer, LNAI 5225.

Pezzulo, G. & Calvi, G. (2007). Schema-based design and the AKIRA Schema Language: An Overview. In: M.V. Butz & O. Sigaud & G. Pezzulo & G. Baldassarre (Eds.), Anticipatory behavior in adaptive learning systems: Advances in anticipatory processing. Springer LNAI 4520.

Pezzulo, G. & Castelfranchi, C. (2009). Thinking as the Control of Imagination: a Conceptual Framework for Goal-Directed Systems. Psychological Research, 73, 559-577.

Piaget, J. (1981). Intelligence and Affectivity. Palo Alto, California: Annual Reviews, Inc

Scherer, K.R. (2001). Appraisal considered as a process of multi-level sequential checking.  In: K.R. Scherer, A. Schorr, & T. Johnstone (Eds.). Appraisal processes in emotion: theory, methods, and research (pp. 92-102), New York: Oxford University Press.

Reisenzein, R., & Hofmann, T. (1993). Discriminating emotions from appraisal-relevant situational information: Baseline data for structural models of cognitive appraisals. Cognition & Emotion, 7, 271-293.

Scherer, K.R. (1984). On the nature and function of emotion: a component process approach.  In K.R. Scherer & P. Ekman (Eds.), Approaches to emotion, (pp. 293-317). Hillsdale, NJ: Erlbaum.

Scherer, K.R. (2001). Appraisal considered as a process of multi-level sequential checking.  In: K.R. Scherer, A. Schorr, & T. Johnstone (Eds.). Appraisal processes in emotion: theory, methods, and research (pp. 92-102), New York: Oxford University Press.

Smith, C.A. & Kirby, L.D. (2001).  Toward delivering on the promise of appraisal theory.  In: K.R. Scherer, A. Schorr, & T. Johnstone (Eds.). Appraisal processes in emotion: theory, methods, and research (pp. 121-138), New York: Oxford University Press.

Squire, L.R. (2004). Memory systems of the brain: a brief history and current perspective. Neurobiology of Learning and Memory, 82(3), 171-7.

Staller, A. & Petta, P. (1998). Towards a tractable appraisal-based architecture for situated cognizers. In: Canamero, D., Numaoka, C., & Petta, P. (Eds.) Workshop: Grounding emotions in adaptive systems (56-61). Vienna.

Steiner, C. M. (1974). Scripts people live. New York: Grove Press.

Sutton, R. S. & Barto, A. G. (1998).  Reinforcement learning: an introduction.  Cambridge, MA: MIT Press-Chapter 6.6 Actor-Critic Methods.