A Model of Early Motivational States and Their Uses

A model of early development of motivational states is proposed. The states are modeled in terms of modern concepts of state space and are physically realized by long-term-potentiation (LTP)-based neural circuits. The basic idea is to assume existence of libido and aggression instincts that would receive single sensory stimulus and induce capabilities for fight-or-flight, freeze or run, etc. The libido state may lead to happiness, contentment, or activities such as dance or play, imitation of observed behavior and action of others, and engagement in learning by trial and error. Enhancement of two motor skills are presented: responding more quickly in time and delivering a larger force of contact. This is a simple example of how the perception system, the motor system and the motivation system interact. A one-degree-of-freedom second-order mechanical system is modified by a first-order neural facilitator or compensator. The tit-for-tat phenomenon in force escalation is also modeled. The model includes tactile sensors for the measurement of a known force applied to a human finger, afferent transmission of the sensed force to the brain, storage of the perceived force, recovery of the stored force from memory, and efferent transmission of the force to the finger. The situation may change based on perception of more adversaries discouraging retaliation and encouraging resort to withdrawal and / or retreat.


Introduction
Three elementary systems are involved in the execution of motivationally based action and behavior: the perception system, the motor coordination system and the motivational system by Kelly and Dodd (1991).Examples include faster response, delivering a larger force than received, and ballistic movements (Hemami & Dariush, 2016) as in baseball.The faster human response is commonly referred to as fast reaction (Murphy & Murphy, 1962) or fast reflex (Laver & Pollard, 1965).The issue of physiological reflexes in general are treated in reference (McMahon, 1984) and higher reflexes such as in fighting or competition in reference (Ischlondsky, 1952).Concentration, physical fitness, and strength (Shaffer, 1928;Murphy & Murphy, 1962;Merrick et al., 2013) are attributes of human performance (Rasch & Burke, 1978).Response to pain (Eichenbaum, 2002, Chapters 1 and 11) and to flight or fight are other examples.
Motor skill enhancement is brought about by neural processing resulting in a compensator or, alternately called, a temporal facilitator (Kuffler & John, 1976;Brooks, 1986).The compensator (Shepherd, 1979) may reside in the CNS and may involve spinal interneuronal circuits (Burke, McKeon, & Skuse, 1981), synaptic processes (Kaczmarek & Levitan, 1987), or electro-chemical (Schwarz & Tomlinson, 1994) processes.The faster reflex may also be desirable when there is sudden exposure to noxious stimuli such as heat, fire, or sharp objects.The limb must be quickly withdrawn (Gandevia & Burke, 1992;Haines, 2002).The normal time of the withdrawal, based on spinal reflexes, can be much shorter if such a compensating neural element is included in the path of the spinal reflex.However, the overall role and function of the spinal reflexes are not well understood, and the interneuronal circuits are complex (Burke, McKeon, & Skuse, 1981;Jankowska & Lundberg, 1981).
The enhancement of the efferent central signals to the muscles by reflex signals takes place in the spinal cord.This may involve speeding up the central signals relative to the reflex signals or vice versa.The process may involve synchronization of neural signals for better motor skills.
Two simple physiological phenomena are involved in the modeling here: an olfactory receptor as a first-order system and a second-order system as an olfactory receptor model or a very simple mechanical system (Rumble & Hemami, 2007) that regulates fine control of finger pressure (Kuffler & John, 1976, Chapter 12).This system has one afferent and one efferent path.
The efferent path starts from the CNS and reaches to the tip of the finger or arm where a force of a prescribed magnitude is exerted by the subject's stationary arm to a stationary external surface (Kandel, Schwarrtz, & Jessell, 1991, page 618).A simple graphical diagram of the trace of the sensory tactile signal to the efferent signal to motor neurons of a muscle is shown on Fig. 19.5 (Kelly & Dodd, 1991, page 280).Somewhere in this path, the pain (or hurting) motivational state, with the help of perhaps visual information, guides the motor cortex strategy of fightor-flight.Stimulation of the different parts of the hypothalamus produces characteristics that correlate with somatic criteria for anger (Kupfermann, 1991).This means there are motivational states in the hypothalamus.Further, hypothalamic neurons that control the release of endocrine hormones are affected by the same neurons.Hormones that are released from the adrenaline glands in stressful situations facilitate the release of other hormones in case of higher stress.Complex neuro-chemical processes appear to take place that may not be adequately addressed by electrical processing alone.It also has been verified that the complex neural programs that lead either to rest and body renewal or to "fight-or-flight"depend on the hypothalamus (Eichenbaum, 2002, page 36).
If the input is enjoyable, the corresponding motivational state commands a relaxation of the motor system and perhaps a concentration on the dimensions of enjoyment.Other inputs may arouse awareness and challenge the self in a motivational state of performance (Kupfermann, 1991), as in simple athletic physical activity.This state induces the performance of the individual to either of two outcomes: a satisfactory performance leading to a state of happiness or an inferior performance leading to a motivational stress-state leading to frustration.These states sustain their physical measure of frustration in time and theoretically cannot be emptied or have their value reduced unless other proper inputs are applied.Of course, audience applause or disapproval add other sensory inputs to the state equations of the current activity.
In natural systems, the facilitatory mechanism may be located at the thalamus level or deeper in the CNS, where it is in the path of all sensory afferent information and thus contributes to an overall enhancement of perceptual skills.The function of the motivational state at this high level of insertion could apply to speed up all sensory stimuli and could also potentially cause undesirable involuntary responses, such as distractive effects.The tennis player may respond equally well to the proper sensory input, e.g.visual cues, as to airplane noise and spectator clamor and excitement.
The role of amygdala to situations of alarm (Eichenbaum, 2002, pages 67-68) is evidence of motivational states in amygdala for sensory processing and action.One may conjecture that early development of the sense of smell and its rapid processing is important for survival of newborn animals.The fawn detects the mother's location by smell and heads for shelter and protection.The fawn may flee after smelling a threat and danger from a predator.This may require motivational neurons that are physical and be involved in the ensuing LTP processing.Escalation has different behavioral aspects: psychological, emotional, and physical.Our formulation and model here are based on reported physically based experiments by Shergill et al. (2003).In one experiment, six pairs of participants were involved.A motor applied a known measurable force to the index finger of the right hand of the first participant.This participant applied an equal force, as he had perceived, to the left hand of the second participant.The application of the perceived force to the other participant continued for a number of cycles.The sequence of reciprocal alternate force applications by the six pairs of participants resulted in the escalation of the force as they proceeded through more cycles.
In another experiment, the perception of force was examined in a subject.A force was applied to the index finger of participants, and the subjects were asked to apply the same force by their left hand on an external object.The participants consistently overestimated the applied force as measured by the force they exerted.It is interpreted that the self-generated force was perceived as weaker than the one generated externally.The authors conclude that force escalation is the result of predictive sensory attenuation (Blakemore, Wolpert, & Frith, 1998).
It is also possible that a considerable number of motivational states may be a result of concentrations of many chemicals in different parts of the body or in the environment.Therefore, such states of motivation must be somewhat abstractly defined and entered as states of motivation extracted from the sensory input channels.Someday, science may have the knowledge and tools to combine all electrical, chemical, and environmentally based states and processes.The most promising tool at this time seems to be state-space analysis (Mayhan, 1984;Bay, 1999).Actually, the state of "self" can be defined as the assembling of all physical, neural, and chemical processes that define the person.Not all of this information is accessible to the individual.Mind and soul may be referring to part of this assembly that cannot be located, accessed, processed, or observed in the current state of evolution.
In what follows, first a simple action path through the sensory-motivational-motor system is outlined in Section II.Motivational case for survival is discussed in Section III.Preliminary computer simulations, as illustrations and examples, are presented in Section IV. Discussion and conclusions are presented in Section V.

Simple Action Path
Suppose a constant tactile force signal is applied to a finger.The intention of the subject is to transmit the same force signal via the finger of the second hand to a stationary object starting at time zero: t = 0.The path considered here is illustrated in (Kelly & Dodd, 1991, Fig. 5.19).The dynamics of the skin sensor at the contact point are approximated by zero mass but with a nonzero viscous constant f .The applied force to the skin is F. The force F w , due to linear tissue compression, is sensed by the mechanoreceptors and transmitted to the spinal cord.Let Ḟw be the derivative of F w with respect to time.The sensor dynamics is described by a first-order differential equation: Equation 1.
The same first-order differential equation can be used to model an olfactory receptor's response.
Neural facilitators are needed to speed the arrival of the olfactory signal to the conscious mind so that fast action allows escape from a predator that has been detected by its smell.Neural facilitators are traditionally thought of as neural or synaptic structures that amplify signals (Kandel, 1991).Here, the facilitator's definition is extended to compensation for transmission delays and neural mechanisms that speed up and accelerate the arrival of danger signals to consciousness where danger is perceived and appropriate action is taken (Smith, 1958;Merrick et al., 2013).

Transmission Delays
The neural transmission delays (Smith, 1958) can be included in the afferent path and the efferent path.In the computational model, the delays can be ignored and/or included with or without neural facilitators and predictors possibly located in the reticular formation.The general form of these predictors, with input x and output y, is given by Eq. 2.
where α and β are amplifier gains and the predictor is made of an LTP (as a memory element or state of the predictor), two gains and two summing device.There could be more poles and zeros in the facilitator (Smith, 1958, Chapter 2).The predictor above has two states, two summing devices and two amplifiers.In a biological system, the states can be realized by LTPs.Therefore, the identification of such a compensator (facilitator) should not be difficult.Also, invariably, all such compensators have feedforward (sensory to cortex direction) and feedback ( cortex to periphery direction) paths.The identification of such facilitators in the olfactory system may be difficult in adult humans and animals.However, in newborn animals, such mechanisms are very important, crucial for survival, and possibly easier to detect.

Second-Order Dynamics
The second-order system model could arise from locomotory system or from receptor dynamics.The neuromuscu-lar portion of the path can be represented by a one-link arm in the frontal plane where the gravity vector is assumed to be negligible or non-existent.This resembles a human's forearm or index finger with the elbows anchored on the table and the hands immobilized.The equation of motion is derived from newtonian dynamics (Laroussi, Hemami, & Goddard, 1988;Jalics et al., 1999).
The moment of force (u) acts at the base of the finger.The finger is a sagittal inverted pendulum with the point of contact of the force (F) at a distance of k from the base.The arm position (angle Θ from the vertical) and its derivative with respect to time, i.e. angular velocity, are used to stabilize the finger at the equilibrium point (Θ = 0 ).The finger applies a force to a vertical surface.No slippage takes place, i.e., only the force control loop is active.
Let the moment of inertia of the finger relative to the base be J.Let α and β be the gains of the position and velocity feedback terms.The equation of the finger, near the equilibrium point, is given in Eq. 3.

Force Control
Brooks (Kandel, Schwarrtz, & Jessell, 1991, pages 248-249) has discussed the cutaneous feedback and the central efferent inputs needed for precise control of the pressure the fingers apply.More information about the dynamics of the force control system can be found in Rumble and Hemami (2007).

Motivational Case For Survival
An olfactory-triggered example of freeze or flight effort is formulated here that may apply to young animals and has survival benefits.Other benefits could be raising alarm, alerting attention, pointing toward food supplies, seeking shelter, suppressing fear, etc.

Olfactory Bulb
The olfactory model presented here is a very basic and holistic one composed of three components in cascade: the bulb, the connector, and the olfactory cortex.For the time being we ignore other interconnections, feedbacks, and other relationships between the three components (Shepherd, 1979, fig. 13.1, page 290).The bulb components are the glomeruli, the mitral cells, and the granule cells that control the gain and amplitude of mitral cells.There may be additional control (feedback) and command signals ( inhibitory or facilitatory) from the CNS and higher olfactory centers (Dodd & Castellucci, 1991, fig. 34.5, page 516).
The connector section, after the bulb, is primarily for distribution of sensory inputs to the olfactory cortex and could be modeled as a tapped delay line, or a first-in-first-out (FIFO) stack.
The olfactory signals, at this stage, i.e., outputs of the FIFO, are delivered to one or more of the five components (signal processor systems) of the olfactory cortex (Shepherd, 1979, fig. 13.1, page 290).

Olfactory Cortex
The five components are: • Anterior olfactory nucleus (AON) • Olfactory tubercle (OT) • Prepyriform cortex (PP) • Amygdaloid complex (AM), and As stated in the literature, the main functions of the above five components are, crudely, as follows.The AON regulates information flow and signal routing information.The OT is involved in multi-sensor processing and concerned with arousal and reward states.The PP is part of the structures involved in autonomic and endocrine functions.The AM is involved in emotional arousal, increased vigilance and freezing.The TER is involved in learning and processing of olfactory information in navigation and memory systems.Of course, eventually all inputs and outputs and state space processing of each component have to be worked out.In turn , the outputs of the five components are directed to limbic structures: the septum and diagonal band (SEP-DB), hypothalamus (HYP), and hippocampus (HIP).The structure, input-outputs and functions of the limbic system is a challenging and complex research effort beyond the modest goals here.
The present goal is to identify the motivational states of freezing, running (for shelter, escape from predators or following safety, i.e., the mother) or, as stated before, fighting or retreat.These states, once activated, usher in the locomotion scenario needed for the task.The freezing state is probably already programmed in the newborn animal or human.The newborn remains stationary in the den.The running motivational state and the quadruped locomotory CNS programs seem to be active and in effect for animals such as the deer, which have to follow the mother, by the sense of smell first and vision next.A escape strategy in rodents propels them, after smelling or seeing danger, to run back to the burrow or tunnel, where the tactile and smell guide and control the locomotory system.
The fight or withdraw (retreat) seems to be learned in young animals for communication, skill development, building strength.It is put to use later for mating, defending territory, and other survival necessities.

Amigdala and Thalamus
The sensed force signal or olfactory receptor signal is transmitted to the amygdala where a physically implemented motivational neuron allows the transmission of the signal to propagate further in the limbic system ( Green & Silver, 1981, page 398) to the thalamus, or be inhibited if the expected adversarial retaliatory forces are deemed excessive.The signal will be enhanced in the thalamus by other motivationally oriented neurons, to become larger in magnitude or be delivered faster.Different facilitators can be envisioned, and they could be located farther up in the CNS in the motor cortex.The motor cortex now delivers a signal through the lower brain to the motor neurons.That signal implements the desired output force to the external object.
An important distinction and observation is warranted here.Sensory smell and olfactory signals are essential in newborn and young animals to sense the proximity of safety (presence of mother) or danger (proximity of predators).This motivational state, which exists in the environment, can trigger contentment, safety, calm, and happiness.It can trigger freezing or flight.According to Beidler (Beidler, 1980, page 599), there is a latency of about 100 millisecond in the response of electro-olfactograms (EOGs) when odor is applied.The neural transmission speed to the CNS (Martin & Jessell, 1991, page 351) is around 0.5 m/s.Therefore, if the signal needs to move about 10 centimeters, it requires about 200 milliseconds.There is need for compensators to predict and compensate for these time delays if a young animal is to detect threatening situations and take quick action.Several such compensators are synthesized and their behavior is demonstrated with computer simulations.
LTPs, i.e. integrators (1/s in system theory terminology) are needed as memory elements for the feedback and feed-forward path of the compensators.
A different class of inputs, such as physical pain by an adversary or the environment, may accumulate in a LTP and, based on the circumstances lead to fight (strike back) or restraint and retreat (withdrawal).Such motivational states, in the amygdala or the thalamus, require a physical, rather than environmental, state to trigger action and response.The processing also may be more complex because the response may need quicker or slower response with higher or lower force or intensity and may be multi-dimensional.Such motivational states, by necessity, must be neurally available and cannot be all chemical or abstract.

Preliminary Computer Simulations
Several computer simulations are presented here for the purposes of illustration and demonstration of the strategies articulated.The simulations are part of the dynamics of Figure 1.

Sensor Compensation
In the first simulation, a sensor with a first order transfer function is modified by a compensator.The physical parameters are not physiologically based.The behavior of the sensor in tandem with three different compensators is simulated.
The first-order olfactory receptor is specified by its transfer function: Four neural facilitators are considered.In the following equations, s is the differentiator operator (Mayhan, 1984), strategy is chosen.In our example, CNS inputs to the control loop of the second arm are issued 1/s is the integrator operator that can always be approximated by an LTP.
The behavior of the first order receptor with and without C 1 is shown in Figure 2. It can be seen that, without the facilitator, the output reaches the threshold of unity in 20 milliseconds.With the compensator, the threshold is reached much faster.Now consider facilitator C 2 in tandem with the receptor.The response of the receptor with and without the facilitator is shown in Figure 3.
The response is slower in comparison with the the case with C 1 , i.e., the output reaches the threshold (unity in the graph) later in time.

Trajectory Compensation
In a second simulation, a second-order system is assumed that could describe the motion of a one-link sagittal limb; and where the output reaches the input threshold of unity in about two seconds.Therefore, it could represent thought processes or limb motion.The transfer function of this system is given by H 2 (s) = (1)/((τ 1 s + 1)(τ 2 s + 1).τ 1 = 1sec.and τ 2 = 1/3sec.The system's time constants are τ 1 and τ 2 and it is in tandem with C 3 As shown in figure 4, the difference between the outputs is negligible for the system with and without the facilitator.
In a second simulation, the compensator is C 4 .The performance of the system, as shown in Figure 5, is much better.This means the system is conscious of the olfactory input much earlier.

Force Compensation
The system is made of two independent fingers belonging to two individuals.A constant force of known magnitude is applied to the first finger.This force is recorded in short-term memory and retrieved for input to the control machinery of a second finger for applying an equal force.We assume that neither finger moves significantly, and the dynamics of these motions are ignored.The force received by the first finger is transmitted to the CNS with some delay.The physical parameters of the control system are given in the Appendix.These parameters correspond to the exertion of a force of one Newton by a finger with a length of five centimeters.
Two computer simulations are presented here.In the first simulation, the received force is constant, and equal to one newton.The received and exerted forces are plotted as functions of time in Figure 6.
The second simulation shows the same result, but for a sinusoidally-shaped pulse of force as shown in Figure 7.
In another simulation, the loop delay was reduced to 20 milli-seconds.The reduction reduces the escalation factor from 2 to 1.25.The results of this simulation for five cycles of two seconds each are shown in Figure 8.The figure clearly shows an escalation of force as the number of cycles increase.

Discussion and Conclusions
The presence of a mother near a child, detected by the olfactory receptors, is a motivational state that brings the child feelings of rest, relaxation, happiness, and well-being.Thus the child is responding to states in his environment.Similarly the presence of a predator, detected by a fawn, may induce flight or freeze options.
Complex mental, emotional, creative, and psychological processes take place inside the human nervous system, all motivated by internal and external motivational states that are conveyed to the individual by the sensory modalities.Understanding, quantifying, and verifying such processes are and will be on the forefront of challenges for science for a long time to come.Chemical processes for the dynamics and physical growth are additional parts of the evolution, well-being and survival of all living species.
Two very simple situations involving one or two scalar states in force enhancement and quicker awareness of danger were sketched and the physical attributes were demonstrated by computer simulation.Learning what predictive electrical and chemical structures are needed for facilitation was postponed for future efforts where models can be verified and confirmed by observation and measurement.
The formulation involved a one-or two-degree-of-freedom sensor or mechanical system to execute a skill.The neural facilitators are built with neurons involving time integration, amplification and summing.Feedback mechanisms are implemented with interneurons or compensators to implement the enhancement.
The more difficult challenge for computational models is that the relevant parts of the environment have to be modeled with five dimensions of the sensory modalities, and also accounted for internally in the CNS.Further, such internal models have to be experimentally or computationally verifiable.
In application to living and natural systems, many interesting questions arise.Where do these neuronal facilitators reside in the CNS?How they are trained?What experiments can be performed to detect and identify such CNS processing?The short-and long-range CNS mechanisms and procedures involved in this kind of learning merit further studies and research (Kandel, Schwarrtz, & Jessell, 1991, chapter 64) as well.
The model of force escalation was based on experimental human data (Shergill, Bays, Frith, & Wolpert, 2003).The proposed model here mimics this human behavior.Our simulations show that the source of this kind of escalation is the anticipatory predictors in the human sensory apparatus, i.e., the feedback part of the system.Only when there are no predictors in the feedback path, is the direct current (DC) gain of the control loop equal to identity, and the force sequence remains stable and does not escalate.
The DC gain of the feedback path, i.e., the important parameter here, depends on the duration of the delay.The longer the delay, the lower the predictor's DC gain and, consequently, the higher the gain of the control loop, resulting in more escalation.Physically, this means thatwith more transmission delay, there is more escalation as long as there are predictors in the feedback path.There is, relatively speaking, less delay from the lips to the brain, more delay from the fingers to the brain, and much more delay from the toes to the brain.Therefore, there will be less escalation when the lips may be engaged, more escalation when the fingers are involved, and still more escalation when toes and the feet are involved.Perhaps this is why kicking seems to be a mechanism of self defense and effective violence!There are other applications of the work here, such as in synchronization of response (Singer, 2000) or in arousal (Foldibk & Young, 1995;Beeman, Ortony, & Monti, 1998;Sejnowski, McCormick, & Steriade, 1998;Reese, 1999).The exploration of these issues merits further research and may pave the way for a better understanding of the dynamics of alertness, arousal, and attention.
Finally, and philosophically speaking, the same facilitatory mechanism could be involved in the CNS in neural circuits where higher levels of mental, emotional and psychological behavior and function are mediated.The compensatory mechanism can facilitate collaboration and speed up favorable responses.It can offer graded responses in robots and facilitate human-robot or robot-robot collaboration, cooperation, and task sharing.

Appendix
In the simulations, the fingers are assumed to be 5 centimeters long, and the estimate of this length is accurate and exact.The predictors are simple first-order lead compensator (s + 20)/(s + 40) for a 25 millisecond delay in the forward or feedback paths, and (s + 20/(s + 25) for a 10 millisecond delay in each path.The muscular amplification is a linear gain of 10.The tissue dynamics are assumed to be a first order system: e/(s + e) with e= 20.The mechanoreceptors are assumed to be ideal with gain of identity.Any movement of the fingers is assumed, as stated before, to be negligible.

Figure 1 .
Figure 1.A force is received by the first arm.It is delayed due to signal transmission, identified by comparison with stored signals in memory, and, based on what is perceived and the chosen motivational state, a movement strategy is chosen.In our example, CNS inputs to the control loop of the second arm are issued

Figure 2 .
Figure 2. The neural facilitator speeds up the reaching to the threshold of consciousness by almost a factor of 10

Figure 3 .
Figure 3.The second facilitator speeds up the response much more slowly

Figure 4 .Figure 5 .Figure 6 .Figure 7 .Figure 8 .
Figure 4.The performance of a second-order system with and without the facilitator C 3 .The difference in speed of reaching the threshold is negligible