Did you see this good article, Positive Reinforcement: Where Does It Fit in Equine Training? about learning theory and horsemanship (with great comments on our own Shawna)?
I particularly like the graphic that Henderson used to illustrate the four quadrants of operant conditioning:
The Original Learning Quadrant
This is often known as the “learning quadrant” and I’m sure you’ll be familiar with it by now as you explore better ways to train your horse. But the question is, how useful is this approach when we get down to the nitty gritty of understanding learning for the purpose of training your horse? How easy is it to tell if you are using positive or negative reinforcement?
Have a look at this video of a very cute dog training her human to pet her and we’ll discuss this to help dissect the learning quadrant:
If we regard the dog as the trainer, then the human is the operator of the system, which is why we have the term “operant” learning (or conditioning). When you watch this video, you might want to ask: what kind of reinforcement is going on here? Is the dog using negative reinforcement to get the human to stroke…i.e. annoying with the pawing until the human does the right thing? Or is it reward-based training the dog is using? The pawing is a cue the human has learned and is rewarded for the behaviour by feeling happy.
But hang on…isn’t there some punishment going on too? Surely the human has to experience some positive punishment with the pawing in order to receive the reinforcement of the pawing stopping when she does the right thing?
The question is, is the dog using a positively learned cue, the pawing, to which the human responds with the wanted behaviour, because the human finds it rewarding. Or is the dog using pressure, an unwanted stimulus, and so the human strokes the dog to avoid the unpleasantness of the pawing?
Emotions and Relationship
As you can see, it all depends on how the human feels about the pawing. This is plainly an
interaction between two bonded animals. This is not a sole rat in an experimental box, with a non-emotional detached scientist standing by to take notes. And so we have to be careful when we use scientific principles which were developed in one scenario, such as with those poor lab rats, and then apply them to more complex situations, such as the interactions between bonded mammals.
Use Your Common-Sense!
From a common sense view of this video above rather than learning theory, we can say that the dog and her human are engaged in a mutually enjoyable interaction. We can tell that from observing their body language. We can also tell that because we are empathic mammals and we have a smile on our faces as we watch. (This will be culturally determined and individual too. If you come from a culture which views dogs differently, or personally dislike dogs and experience them as dirty, you will have an expression of dislike or disgust on your face.) Modern science completely supports this common sense view. For example, if we were to take blood samples from the pair, we would see rising oxytocin chemicals in both of them. They are “feeling the love”!
This viewpoint would not have been accepted by B.F. Skinner, from whose pioneering work models such as the learning quadrant have been derived. He said that “the ’emotions’ are excellent examples of the fictional causes to which we commonly attribute behavior”.
Modern neuroscience is supporting our more common-sense view and is clearly demonstrating that behaviour is caused by the emotional state of the brain. Behaviour does not happen in an unemotional vacuum at all.
So is it time to ditch such models as the learning quadrant? I don’t think so because people find it useful in thinking about training, especially when you begin to understand reward-based training and the fact that all the horsemanship you were previously taught was probably based on punishment. But we do need to understand how emotions work alongside behavioural contingencies so that we can be happy with how we work with our horses and be confident that our approach is both fair and effective.
Delving Deeper into the Emotional Brain
From my studies into the neuroscience of emotions, I have developed the learning quadrant model to take account of mammalian emotional brain systems. I have been studying the work of neuroscientist Jaak Panksepp for the last few years and his brilliant work on the 7 emotional systems in the brain that all mammals share. I find this a really useful way to understand what is happening in our relationships with our horses and to help us to train most effectively. The seven emotional systems he has painstakingly uncovered are SEEKING, FEAR, RAGE, PANIC/GRIEF, LUST, CARE and PLAY.
For more info on these emotional brain systems, check out this blog, “Phew, it’s ok to be anthropomorphic!“
The basic fact of how the brain learns is through punishment and reward. These emotional systems create the feeling of an event being good or bad, (or appetitive or aversive, to give it the technical terms). If a situation lights up the FEAR system, for example, it will feel aversive to the animal and it will learn to avoid that situation in the future to prevent feeling the fear again. If a situation lights up the PLAY system, by contrast, it feels appetitive and the animal will learn to recreate the situation in the future. The point of Panksepp and the many other neuroscientists work on emotions, is that our animals are feeling the pain or joy due to their brain emotional systems, which are the same as ours.
Putting the two systems of thought together, I have created a new model of the behavioural contingencies (“learning quadrant”) with the emotional systems overlaid:
The Emotional Learning Quadrant
This model clearly shows that in training our horses using rewards, we are operating within the good feeling emotions of SEEKING, CARE and PLAY (I have omitted LUST from the diagram as we don’t want that emotional system in a horse-human interaction!). The SEEKING system is the default for the horse to even engage with the process. It is the emotional system which is characterised by enthusiasm and exploration. CARE is the bonding system, which is very strong in horses as herd-based mammals. We need to be in CARE mode with scared horses and help them to bond with us in their CARE systems and to feel trust in us. Once a good calm CARE state is the default, along with the SEEKING, we can move into PLAY and really help these anxious horses to enjoy their training. On the other hand, you may start with a very confident horse where you build your interactions through PLAY and develop the CARE system bond more slowly, possibly needing to teach the horse to be calm and relaxed at a distance before you invite him closer into your space in a trusting relationship. It will depend on your horse but the important point is that it is all rewarding.
However, it’s important to know that all systems will probably be in play at some time, even if we aim to stay in the green zone. The main reasons for this are that the horse gets frustrated simply by the learning process; where we are counter-conditioning situations where there is existing FEAR or PANIC ; or because something happens in the environment to cause these negative emotions. However, if you work to make the training stay in the green zone, these emotions will be fleeting.
The Important Question: How Does My Horse Feel About This?
The learning quadrant model has been really useful to help us to think about our training and to ask the question “Am I using reward or punishment to teach here?” But that can be a difficult question to answer and can cause us to feel anxious and inhibited if we feel we shouldn’t use punishment and maybe that it should all be positive. Adding in the emotional systems helps us to ask a different question “How does my horse feel about this?” Does my horse look enthusiastic, eager to try and so is in the SEEKING system? Or maybe he swished his tail as he performed the behaviour, so is in his RAGE system, but at a low level of stimulation. Perhaps she’s feeling hesitant and has dropped a poo beside the trailer ramp so I know she’s in a state of FEAR even though she is trying and is taking the treats. The point is that a horse is a complex mammal and for a rich and full life, needs to experience all the emotions.
What About the Old Stuff?
Often when we start reward-based training we will still be using training that our horses learned through more conventional or natural systems, which we now understand to be based on punishment rather than reward. This can cause us some anxiety and distress and we get many students asking what they should do. For example, should they keep riding in the old way or stop everything and retrain it positively? The answer, of course, is “it depends” and we are very good at Connection Training in helping you to work out the best approach for you and your horse. We don’t have set programmes to follow, simply lots of resources and support to help you work in the best way for you. For example, even if a behaviour was originally taught using negative reinforcement, the question is “how does your horse feel about it now?”
Let’s take riding out on the trail as our example here. This activity puts your horse into his SEEKING system and is an important basic behaviour for horses to get out and explore their environments. If your horse is comfortable with being ridden then it will be more rewarding for your horse for you to keep riding the old way rather than have to stay at home and have this limitation until you have completely retrained it the new way. And the amazing thing is, as you bring in new rewarding behaviours outside of riding then one of two things will happen. Either your horse has got hidden issues about riding and, as you train other things positively, you will start to notice that your horse feels less good about the riding than he does about the other stuff. This will cause you to stop and ask what is happening and to make the riding more rewarding too. Or you’ll find that your horse is truly happy with the riding and it continues to feel good and happy and you can carry on as before but with more confidence.
It is truly win-win when you simply keep asking “how does my horse feel about this?” rather than worrying that you may have taught this with pressure and that is wrong and “oh god, I’m an awful human being.” Think back to the dog and her human video at the start of this blog and remember that that is what you are looking for with your horse…a happy connection where you are both feeling good and rewarded in your relationship.
Forget about the labels and focus on the feelings and all will be well.
In my next blog I’ll develop the model further to look at arousal states and whether it’s ever OK to train using punishment… in other words, to engage the emotional systems of FEAR, RAGE or PANIC.
For more info on these emotional brain systems, check out this blog, “Phew, it’s ok to be anthropomorphic!“
If you enjoyed this blog, please share it with your friends using the buttons below. Tell us your thoughts, questions and experiences in the comments section and/or sign up to our mailing list below. You will receive a FREE Video Seminar on The 4 Elements of Connection Training!
☟Scroll down the page to subscribe!☟
17 thoughts on “The Emotional Learning Quadrant”
Great blog Rachel! Very clear and simple. I have bags of empathy and plenty of common sense so all will be well. Thanks, this is a great confidence booster.
Thanks and I’m glad I’ve boosted your confidence. I think that’s the best training asset there is!
Thanks….great mantra that I will be using in future!
It seems to me that the emotions of fear, panic, and rage are most often experienced whenever the horse is punished…whether positive or negative punishment. Having trained horses using pressure and release as well as positive reinforcement, I’ve seen the highest development of the seeking and play emotions do long as they are promptly reinforced (either positive or negative reinforcement. So perhaps you could help me understand how negative punishment would bring out play versus frustration (rage emotion), and why you feel negative reinforcement would not bring out play and seeking emotions? I certainly appreciate the analysis of emotions alongside learning quadrants, but I’m afraid this can be easily used to cause people to shy away from negative reinforcement, and as you said so well, what really matters is how the horse feels about it. Thanks in advance for clarification.
Hi Grace, thanks for your thoughtful reply. The whole model is an over-simplification, of course, and in the next few blogs I will be adding more if’s and but’s, especially with regard to the particular relationship/bond between the horse and her human. Here I have been using the basic scientific terms from both behavioural analysis and the modern affective neuroscience. It looks from your question that I haven’t been clear enough in defining the terms and how I’m referring to them.
Let’s start with the easy bit…the positives! Positive, as I’m sure you know, in this context means something added. It does not mean “good”. When you are working with rewards, adding them in, you are definitely priming the SEEKING, PLAY and CARE systems. This doesn’t mean the other systems are not activated ever, but that these are the dominant emotional systems activated. Contrarily, when you utilise positive punishment, you will be activating FEAR, PANIC and/or RAGE as the dominant emotional systems. SEEKING will also be involved, but will be used in order to help work out how to get away or fight, not in the “enthusiastic, curious about the world” way that it it works with rewards.
Negative reinforcement is also fairly easy to understand from an emotional systems perspective. I have placed it with positive punishment because it is based on aversives…the negative part refers to the removal of an aversive. It is the relief from the discomfort/pressure/pain of the aversive that creates a feeling of relief. It is how to get to that relief faster (ideally to avoid the pressure) that creates the learning process.
This brings me to a problem with the original terminology derived from Skinner: the word “reinforcement”. Modern neuroscience is showing that the brain works with either rewards (also known as appetitives) or punishers (also known as aversives). As far as the brain is concerned there is no such thing as reonforcement in the way Skinner meant it. So really, negative reinforcement would better be described as pressure-relief. Interestingly, Shawna used the term “relief-reward” in her original book, which I think is a really useful term.
I have placed negative reinforcement beside positive punishment as it is the same emotional systems which are involved because, in order to get to the relief, the animal first has to experience the aversive, or the punisher. In other words, in order to obtain the learning from the removal of the aversive, positive punishment has to be used first to get the behaviour. People don’t like this fact but it is true if you think about it. For example, you shake a whip/rope/flag/your arms behind a horse to get him to move forward. This is inducing the flight response which means the FEAR system has been activated, which is punishing in the brain. It is aversive. No matter how gentle your stimulus and how quickly you release it, if you are relying on the flight response to get the behaviour, you are activating the FEAR system. This contrasts completely with reward-based training where you ask the animal to move forward following a target. That is almost classic SEEKING systems behaviour and will be experienced as rewarding in the brain.
However, how aversive it is to the particular horse depends on the horse-human interaction and, as you say, it can be followed by play and so it can be a means to light up the PLAY system at least. It’s harder to use it to light up SEEKING (exploration and curiosity) but maybe you can think of some examples. It worries me that we might think it’s a route into the CARE system as that is based on security not fear, but again, please come forward with your own ideas.
The final difficult one is negative punishment. This means that the opportunity to earn a reward is taken away. The question is, which emotional systems are involved? I have thought about this a lot and, in fact, am having a whole blog in this series devoted to it, so it’s a great question. I place it with Positive Reinforcement in this model in order to emphasise that we are still working with appetitives. It is still those emotional systems of SEEKING, CARE and PLAY we are addressing, but this time taking away the opportunity for them to be lit up in the brain . The question then is, does negative punishment really exist or, when we think about things from an emotional systems viewpoint, are we actually activating at least RAGE, as it usually induces some sort of frustration response, which is the RAGE system in action? Or, when we talk about negative punishment, are we really talking about extinction? Skinner’s rat pressing a lever that used to dispense a treat and now doesn’t?
These are great questions and I hope you’ll enjoy and join in the discussion as I progress through this series. I am using the blog to think out loud…the whole topic is really a Doctorate and I can’t wait until I do that because we’ve got horses to train here! Thank you very much for engaging with me on it and I’ll love carrying on the conversation
Rachel, thanks so much for your thoughtful reply. I applaud you for writing an article that talks about emotions in training – it really is a difficult topic.
You said in your reply “I have placed negative reinforcement beside positive punishment as it is the same emotional systems which are involved because, in order to get to the relief, the animal first has to experience the aversive, or the punisher.”
Technically, yes, the horse does have to experience the aversive (pressure), but that does not mean the feelings of fear/rage/panic automatically come with the aversive being applied. I also think “aversive” is a terrible term to use because we automatically think it must be BAD. If we looked at all applications of pressure as “bad”, one could argue that riding is an aversive for the horse and should not be done. However, I feel that we need to educate riders on HOW to apply pressure in a way that the horse understands and encourages the horse to think.
A few examples of the seeking emotion using negative reinforcement
1. Think of teaching a horse to target, just using negative reinforcement. You teach them that they are in control of the target (i.e., they look at it, then it goes away), they start to build confidence and hunt out the target. And once you stop removing the target, they usually become very curious and start to explore it on their own (SEEKING), just for the joy of discovering more about it.
2. I might use negative reinforcement to encourage a horse to walk on a rough surface. I usually apply just a little pressure, set them up, and hope they more an inch in the right direction. Once they figure out what I’m after, they often start to explore on their own by pawing, smelling, or stepping on the obstacle.
The goal should be to use just enough pressure to get their attention, but NOT so much as to elicit a flight/fear response.
As for the relationship between positive reinforcement and negative punishment. Whether or not negative punishment is actually extinction is a great question, but I’m sure we can all at least agree that we have seen a horse behave in a frustrated manner when a reward is removed. The goal is always to minimize this feeling of frustration, by rewarding frequently and being as clear as possible. BUT, again, I feel the horse can just as easily feel fear/rage in this area if the trainer is not very tactful. If a reward is withheld or the schedule of reinforcement is changed, this can be very frustrating for the horse. Even when using positive reinforcement, if we are asking for a lot of mental of physical effort, they still may feel frustrated or stressed in the process.
As you put so well: “Forget about the labels and focus on the feelings and all will be well”. I couldn’t agree more. My main concern was that someone would look at your diagram and miss the main message.
Thanks again and I look forward to reading your next post!
Grace, I’m sorry, but the definition of an aversive means it feels bad. An animal will then demonstrate avoidance of the situation that gave rise to those feelings. Those bad feelings are experienced by activation of (in Panksepp’s highly respected research; others have defined the systems slightly differently) RAGE, PANIC and FEAR. There is no way round it. If you use an aversive to get behaviour, you will have activated these emotional systems. It’s a simple fact.
I do not confuse pressure with tactile cues which have been trained using rewards and thus have the emotional systems of SEEKING and possibly PLAY associated with them. However, i agree that riding may well be experienced as aversive and believe that is so for most horses. However, I don’t think that it is necessarily aversive, and also I know riding can be highly appetitive. I know this from our horses who willingly, at liberty and repeatedly, present themselves at the mounting block for riding.
I agree that many horses experience frustration if the trainer uses negative punishment and don’t advocate it as part of the training repertoire. I actually prefer positive punishment/ negative reinforcement for behaviours which we don’t want (in other words, activation of FEAR and/or RAGE followed by relief when the unwanted behaviour stops). But I have lots of caveats attached to that statement which I’ll discuss in my next blog when I look at arousal states as well.
It’s all very fascinating, isn’t it?
I guess it’s hard for me to see that, as you said it is for most people, since my horses are not reacting or acting in a way that I see fits those “bad” emotions when I use negative reinforcement. I see calm, relaxed horses, who typically are enthusiastic and eager to figure out the next task. That is why I couldn’t wrap my head around them feeling RAGE/PANIC/FEAR. But I don’t train in a box either, so they may feel rewarded by other means, rather than just the removal of the aversive.
Anyhow, thanks for getting me thinking! I really appreciate you taking the time to engage with me. I will have to read more of Panksepp’s research and I will be on the look out for you next article. 🙂
Thank you Rachel, for a really great article! I am looking forward to hearing more.
I really appreciate this part;
“Forget about the labels and focus on the feelings and all will be well”
Thanks. It’s what Shawna and Hannah do and it seems to work for them!
Thanks that was really helpful!
Emotions flavor and color all our outlooks, so important to remember with dealing with sentient beings. I’ll have to reread this to absorb even more. TY
ExcellentI love these type of articles……….refreshing, reminds one how fascinating the whole topic of training It is forever an adventure.
Thank you for this, this has really got my brain working overtime! Short of explaining it away as a political act, I can’t understand why you have re-labelled negative punishment as reward/reinforcement or negative reinforcement as punishment… Is this a re-write of the scientific definitions or are you using a lay/new definition for these? For what my thoughts are worth, I think it would be useful to use different terms/words. As it seems to be a moral delineation, perhaps you could use the terms ‘bad’ and ‘good’ or something else without a fixed scientific meaning? I don’t think there is anything wrong or untoward about a political act BTW!
Thanks for all your thanks, everyone. I find it fascinating too. Kerry I love your question but am going out for dinner so will get back to it tomorrow. Love the idea of a political act…right up my street!
Hi Kerry, aarghh! You are quite right…I had changed that and sent Hannah the wrong diagram which is was got published. Now I need to get it changed again, although that one is out there.
In the original I labelled the green box “appetitives” and the red box “aversives”. Fairly straight-forward and you may have seen that presentation of the diagram before. The first person i saw present it that way was Dr Helen Spence at the Equine Clicker Conference in 2013. But i noticed that Word always underlined these words as if they were misspelled and I thought that many people might not have heard of them and that it would be confusing. So I re-labelled them rewards and punishers as +R and -P involve the addition or removal of rewards and +P and -R involve the addition or removal of punishers. Then, again, Word underlined the word “punishers” and I thought, well go back to reinforcement/reward and punishment, meaning the action in the brain on the emotional system, not the external use of the words. I then thought, no, I need to go back to appetitives and aversives and just explain them, which I did, but then managed to send Hannah an older version of the diagram.
So thanks for pointing it out and I’ll get it changed as soon as Hannah can get to it. I like the “political act” idea because I am challenging the use of “reinforcement” where I believe it should simply be “reward”. Panksepp pointed out that reinforcement is a confusing term when the brain works on rewards and punishers:
“Although we certainly understand that procedures of reinforcement, which follow the “law of effect” (i.e. positive rewards increase preceding responses and punishments reduce preceding responses), we are still far from understanding the process of reinforcement….Some believe no such process exists, and animals develop cognitive knowledge about their world” Affective Neuroscience, p373 Note 8
He cites research into these processes which was being done in the late 1990’s and it’s my intention to follow this up to see where they’ve got to in their understanding, but I haven’t had the time to do that yet. My own belief is that reinforcement is the process of myelination of neurons to make the messages clearer and quicker, like a well-worn footpath in the brain. I also suspect that latest studies will tell us that myelination is quicker and stronger when learning is reward rather than punishment based, but again that’s just my thinking out loud from general neuroscience I’ve read.
Thanks again for the question and the help and I’d love to hear any more ideas and information on the topic. Who’d have thought horse training would lead us into neuroscience???
Thanks Rachel… This really did get me thinking! I have to admit it’s taken me some time to gather my thoughts and analyse my initial visceral response and copious red emotions lol! After some consideration I think much of this emotion is about some fundamental disagreements I have with the acceptance of ‘science’ as the finder of facts about the world but that’s a whole other thing and I shall try not to bore you with that!
I understand where you are coming from and I can see the appeal of the word ‘reward’ over reinforcement. Sufficient to say I have more ‘social constructionist’ leanings than ‘behaviourist’ ones and so find many limitations in Skinners work. However, I do think Skinner made a valid point when he rejected the word ‘reward’ in his model. The word ‘reward’ is problematic to me as it puts the trainer/human back at the centre as the rewarder who expects a behaviour to increase as a result. In Skinners model it is only the learners behaviour which identifies whether something has acted as a positive reinforcer or not, by increasing the behaviour or not.
I think this differentiation is important as it allows us to understand that at the end of a training session we might jackpot by giving numerous ‘rewards’ – treats and scratches, saying lovely words in our best praising voice, taking off the saddle and bridle, walking away and allowing the horse to roll or do as she pleases. Whilst this can all be identified as ‘rewards’ or potentially ‘rewarding’ for the horse it has nothing to do with whether she has learned or not, in other words whether positive reinforcement has taken place or not. When people say “reward-based training doesn’t work” then semantically at least I have to agree with them, as the trainer identifies and chooses a ‘reward’ or ‘rewards’, learning might happen or it might not… If that makes sense?! I think I know what I mean!:-) I do realise Panksepp will have his own definition of ‘reward’ which might differ to Skinners, which just increases the challenge of combining two models limited by language!
I also wonder if you should change the word reward in the definitions above the line ie ‘the removal of reward’? I’m not sure this makes sense unless you redefine reward in the diagram somewhere (perhaps using Panksepps definition?). For example, learning through negative punishment might be the result of the removal of food, oxygen or freedom but I don’t think it makes sense to call any of these ‘rewards’ in most western cultural contexts, or is at least confusing. Below the line you have used the word ‘bad’ as the opposite of ‘reward’ which feels incongruent. ‘Bad’ is quite a loaded and unscientific term that suits being opposed to ‘good’ so well in my opinion. Again I think this has the effect of muddying the waters on what is meant by the term ‘reward’ in this context, is it a scientific term or a simple judgement?
In the interest of laying my own biases bare, I admit that I am not a fan of Panksepps methods, largely on the basis of my own personal moral philosophy, but whilst the mention of his name elicits the red emotions in me, that’s not to say I can deny his findings are interesting. Philosophically, what I really struggle with is the reductionist nature of scientific models and neuroscience in particular. I am not surprised that neuroscience has turned up in discussions about horses, more anxious and worried if I’m honest. Research evidence has demonstrated that the mention of neuroscientific terms interfere with our ability to follow the logic of an argument (Weisberg etal ‘The Seductive Allure of Neuroscience Explanations’, Journal of Cognitive Neuroscience. 2008 Mar; 20(3): 470–477).
Additionally, whilst learning theory and myelination of the axom might say something about the physiology of the learning process I feel it says very little about ‘relationship’. By that I mean the space in between two individuals, where relating happens, rather than what happens in a particular subjects brain. I am not saying that Panksepps emotions aren’t important (proving the existence of emotions in animals has no doubt had a huge and positive impact upon the scientific world), just that they are such a small piece of the jigsaw when it comes to the complexities of one individual relating to another, and the connection we all strive for in it’s broadest sense. At least that is to me.
Please know I understand your excellent intentions in writing this blog for the good of horses and the lives they individually experience, especially in terms of their emotional lives. I do hope that this information enhances some equine experiences and I feel it will. But I do worry that, rather than adding colour and complexity, the combining of two reductionist theories has the inherent risk of reducing our understanding of what it means to be a social animal even further. Whether that be horse or human.
Ok, I promise I will shut up now!