Clicker training
Encyclopedia
Clicker training is an operant conditioning
method for training an animal using a clicker
, or small mechanical noisemaker, as a marker for behavior. The method uses positive reinforcement - it is reward based. The clicker is used during the acquisition phase of training a new behavior, to allow the animal to rapidly identify that a behavior is sought and also the precise behavior of interest.
Clicker training was originated through Marian Bailey
(née Kruse) and Keller Breland, who as graduate students of psychologist and eminent behaviorist B.F. Skinner taught wild-caught pigeons to "bowl" (push a ball with their beaks) while participating in military research. According to their work, animal training was being needlessly hindered because traditional methods of praise and reward did not inform the animal of success with enough promptness and precision to create the required cognitive connections
for speedy learning
. Similar methods were later used in training at least 140 species including whales, bears, lions, chickens and domestic dogs and cats, and even humans (TAGteach).
A clicker is just one example of a conditioned reinforcer (secondary reinforcer) or "bridge". Technically a stimulus from any sensory mode may become a conditioned reinforcer (ex. light, smells).
first identified and described the principles of operant conditioning. Marian Kruse and Keller Breland, two of Skinner’s first students (and later married couple) first saw the potential of the new technique for animal training business.
After participating as research students with Skinner in pigeon behavior and training projects during World War II, the Brelands left graduate school and formed the first company to intentionally use operant conditioning, Animal Behavior Enterprises (ABE). They created the first free-flying bird shows and a host of commercial animal exhibits.
Bob Bailey was the US Navy's first Director of Training and later came to work at ABE in 1965. Keller Breland died in 1965 and Marian married Bob Bailey in 1976. Together they continued the pioneering work at ABE. Radio-carrying cats were steered through cities and into buildings under a contract with the CIA. Dolphins located targets many miles from their trainers, at sea. Ravens and other birds, carrying cameras and directed by lasers, could fly to a specific window of a skyscraper and photograph the people inside. Gulls, expert sea searchers by nature, could locate and report life rafts and swimmers far offshore.
. As a simple example, consider teaching a dog to turn in a circle (spin). At the instant that the dog completes the turn, the handler must let the dog know that it has done the correct thing. However, the traditional "good dog!" takes so long to say that the dog might already have moved on to some other behavior. At the least it is not immediately obvious that the "good boy" is earned at the precise moment of completing a circle. By the time the dog realizes it is being praised, it might be sitting and scratching or looking for something else to do. In the laboratory behavioral researchers including Norm Guttman, Marian Kruse and Keller Breland, realized that rats always stop what they are doing when they hear the hopper make a sound indicating it was beginning to deliver food, and they tend to do more of what they were doing when the sound occurred. Under the instruction of B.F. Skinner, they decided to try using a sound to mark behavior outside the operant chamber. Toy crickets, the earlier equivalent of today's clicker, were common in those days, and served the purpose very well. The clicker is likened to the surgeon's scalpel; it allows for precise timing and clear communication about what specific behavior is being reinforced, and enables the trainer to teach complex and difficult skills to the animal without the use of force or punishment.
As this type of training was practiced and improved upon, it became apparent that the variability of the human voice, and its presence during all activities make it a less than salient tool for marking behavior. Besides the imprecision in timing, using the trainer's voice for feedback means that the actual sounds for feedback will vary. A handler's voice, pronunciation, tone, loudness, and emphasis may change even during the same training session. Clicker trainers believe that it is better to use a "click" sound to avoid variations in sound. Many trainers opt to use clickers for training that requires precision and continue to use their voices in the form of praise for behaviors that do not need to be precise.
There is also some circumstantial evidence which suggests that the sound of the clicker is the kind of stimulus — like a bright flash of light or a loud, sudden sound — that reach the amygdala
(the center of emotion in the brain) first, before reaching the cortex (the thinking part of the brain). Clicker trainers often see rapid learning, long retention and a "joy" response to the sound of the click in the learning animal. This idea is not universally accepted, and no known research has confirmed it. Any reinforcer can produce joyful behaviors in learners if delivered correctly.
Tasks learned with the clicker are retained even years after the fact and with no additional practice after the initial learning has taken place. This is probably due to the fact that the animal participates fully in the learning process and applies itself to it, learning by trial and error rather than acting out of habit or a momentary response to a situation. Clicker–trained animals become great problem–solvers, develop confidence, and perform their work enthusiastically. This retention of learning is present in positive reinforcement training (including but not exclusive to clicker training), but does not happen with any regularity with correction-based training.
The marker can be any signal that the animal can perceive
, so long as the signal is brief (to prevent the problem of imprecise timing) and consistent (to prevent the problem of variations that may confuse the animal). For large sea animals the marker is usually a whistle rather than a clicker. However, not all conditioned reinforcers are sounds. Goldfish
and birds such as falcons and hawks can be trained using a quick flash of a flashlight as their "clicker". Deaf dogs can be trained with a vibrating collar.
As pointed out by Lindsay the advantages of the clicker may be particularly strong in some situations: "...the clicker's simplicity and clarity provide a significant advantage for some training activities..."
Other trainers, including Bob Bailey and the ABE Trainers, simply start training a behavior and following desired approximations with a click. ABE conducted experiments that demonstrated that for their purposes, where they may be training many animals at the same time, this method was more efficient. Today many clicker trainers use this method of introducing the clicker.
After that, the trainer uses the clicker to mark desired behaviors as they occur. At the exact instant the animal performs the desired behavior, the trainer clicks and promptly delivers a food reward or other reinforcer. One key to clicker training is the trainer's timing; clicking slightly too early or too late rewards and therefore may reinforce whatever behavior is occurring at that instant. The saying goes, "you get what you click for."
Clicker trainers often use the process of "shaping," which means gradually transforming a specific behavior into the desired behavior by rewarding successive approximations to it. A successive approximation is "a behavioral term that refers to gradually molding or training an organism to perform a specific [completed] response by [first] reinforcing responses that are similar to the desired response." Clicker trainers learn to split behavior instead of lumping it, i.e. to look for and reward small steps in the right direction rather than waiting for the whole, "perfect" behavior to appear on its own. It is important to create opportunities for the animal to earn rewards very frequently. A reinforcement rate of one click/treat (C/T) every two to three seconds is common among professional dog trainers. Criteria for receiving the click is tightened gradually, at the rate the animal is comfortable with and so that it will remain successful.
Many desired behaviors start with the nose-touch, where the dog learns to touch an identified target, such as a small piece of plastic, with its nose; that behavior can then be transported to perform useful tasks or interesting tricks such as flipping a light switch or ringing a bell to go outside.
Training the nose touch begins with getting the dog to touch a target with its nose; trainers sometimes use a guided method, such as placing a dab of peanut butter on a small plate or plastic target; others prefer shaping, where the target is placed in easy reach, such as in the trainer's hand between the trainer and the dog, and the dog is rewarded each time he moves in the target's direction or actually touches it.
When the dog is consistently touching the target, the trainer progresses to a target with and without food and in different positions. Eventually, the trainer can transfer the behavior to a bell, for example by holding the target behind the bell so that the dog has to touch the bell to get at the target, and then rewarding the touching of the bell. When the dog is reliably touching the bell, the trainer now adds the act of opening the door to the reward each time the dog strikes the bell.
Targeting for Horses:
For horses, loading or charging the clicker is usually not done. It's best for horses that a clear marker is used so that the horse does not expect "unearned" treats.
Operant conditioning
Operant conditioning is a form of psychological learning during which an individual modifies the occurrence and form of its own behavior due to the association of the behavior with a stimulus...
method for training an animal using a clicker
Clicker
A clicker is any device that makes a clicking sound, usually when deliberately activated by its user.They usually consist of a piece of thin metal or plastic held in a casing so that the metal is slightly torqued; depressing one end of the metal causes it to pop out of alignment and releasing it...
, or small mechanical noisemaker, as a marker for behavior. The method uses positive reinforcement - it is reward based. The clicker is used during the acquisition phase of training a new behavior, to allow the animal to rapidly identify that a behavior is sought and also the precise behavior of interest.
Clicker training was originated through Marian Bailey
Marian Breland Bailey
Marian Breland Bailey, born Marian Ruth Kruse and nicknamed "Mouse", was an American psychologist, an applied behavior analyst who played a major role in developing empirically validated and humane animal training methods and in promoting their widespread implementation...
(née Kruse) and Keller Breland, who as graduate students of psychologist and eminent behaviorist B.F. Skinner taught wild-caught pigeons to "bowl" (push a ball with their beaks) while participating in military research. According to their work, animal training was being needlessly hindered because traditional methods of praise and reward did not inform the animal of success with enough promptness and precision to create the required cognitive connections
Animal cognition
Animal cognition is the title given to the study of the mental capacities of non-human animals. It has developed out of comparative psychology, but has also been strongly influenced by the approach of ethology, behavioral ecology, and evolutionary psychology...
for speedy learning
Learning
Learning is acquiring new or modifying existing knowledge, behaviors, skills, values, or preferences and may involve synthesizing different types of information. The ability to learn is possessed by humans, animals and some machines. Progress over time tends to follow learning curves.Human learning...
. Similar methods were later used in training at least 140 species including whales, bears, lions, chickens and domestic dogs and cats, and even humans (TAGteach).
A clicker is just one example of a conditioned reinforcer (secondary reinforcer) or "bridge". Technically a stimulus from any sensory mode may become a conditioned reinforcer (ex. light, smells).
Co-founders
B. F. SkinnerB. F. Skinner
Burrhus Frederic Skinner was an American behaviorist, author, inventor, baseball enthusiast, social philosopher and poet...
first identified and described the principles of operant conditioning. Marian Kruse and Keller Breland, two of Skinner’s first students (and later married couple) first saw the potential of the new technique for animal training business.
After participating as research students with Skinner in pigeon behavior and training projects during World War II, the Brelands left graduate school and formed the first company to intentionally use operant conditioning, Animal Behavior Enterprises (ABE). They created the first free-flying bird shows and a host of commercial animal exhibits.
Bob Bailey was the US Navy's first Director of Training and later came to work at ABE in 1965. Keller Breland died in 1965 and Marian married Bob Bailey in 1976. Together they continued the pioneering work at ABE. Radio-carrying cats were steered through cities and into buildings under a contract with the CIA. Dolphins located targets many miles from their trainers, at sea. Ravens and other birds, carrying cameras and directed by lasers, could fly to a specific window of a skyscraper and photograph the people inside. Gulls, expert sea searchers by nature, could locate and report life rafts and swimmers far offshore.
Advantages
One of the challenges in training an animal is communicating exactly when the animal has done the behavior that the handler is attempting to reinforceReinforcement
Reinforcement is a term in operant conditioning and behavior analysis for the process of increasing the rate or probability of a behavior in the form of a "response" by the delivery or emergence of a stimulus Reinforcement is a term in operant conditioning and behavior analysis for the process of...
. As a simple example, consider teaching a dog to turn in a circle (spin). At the instant that the dog completes the turn, the handler must let the dog know that it has done the correct thing. However, the traditional "good dog!" takes so long to say that the dog might already have moved on to some other behavior. At the least it is not immediately obvious that the "good boy" is earned at the precise moment of completing a circle. By the time the dog realizes it is being praised, it might be sitting and scratching or looking for something else to do. In the laboratory behavioral researchers including Norm Guttman, Marian Kruse and Keller Breland, realized that rats always stop what they are doing when they hear the hopper make a sound indicating it was beginning to deliver food, and they tend to do more of what they were doing when the sound occurred. Under the instruction of B.F. Skinner, they decided to try using a sound to mark behavior outside the operant chamber. Toy crickets, the earlier equivalent of today's clicker, were common in those days, and served the purpose very well. The clicker is likened to the surgeon's scalpel; it allows for precise timing and clear communication about what specific behavior is being reinforced, and enables the trainer to teach complex and difficult skills to the animal without the use of force or punishment.
As this type of training was practiced and improved upon, it became apparent that the variability of the human voice, and its presence during all activities make it a less than salient tool for marking behavior. Besides the imprecision in timing, using the trainer's voice for feedback means that the actual sounds for feedback will vary. A handler's voice, pronunciation, tone, loudness, and emphasis may change even during the same training session. Clicker trainers believe that it is better to use a "click" sound to avoid variations in sound. Many trainers opt to use clickers for training that requires precision and continue to use their voices in the form of praise for behaviors that do not need to be precise.
There is also some circumstantial evidence which suggests that the sound of the clicker is the kind of stimulus — like a bright flash of light or a loud, sudden sound — that reach the amygdala
Amygdala
The ' are almond-shaped groups of nuclei located deep within the medial temporal lobes of the brain in complex vertebrates, including humans. Shown in research to perform a primary role in the processing and memory of emotional reactions, the amygdalae are considered part of the limbic system.-...
(the center of emotion in the brain) first, before reaching the cortex (the thinking part of the brain). Clicker trainers often see rapid learning, long retention and a "joy" response to the sound of the click in the learning animal. This idea is not universally accepted, and no known research has confirmed it. Any reinforcer can produce joyful behaviors in learners if delivered correctly.
Tasks learned with the clicker are retained even years after the fact and with no additional practice after the initial learning has taken place. This is probably due to the fact that the animal participates fully in the learning process and applies itself to it, learning by trial and error rather than acting out of habit or a momentary response to a situation. Clicker–trained animals become great problem–solvers, develop confidence, and perform their work enthusiastically. This retention of learning is present in positive reinforcement training (including but not exclusive to clicker training), but does not happen with any regularity with correction-based training.
The marker can be any signal that the animal can perceive
Perception
Perception is the process of attaining awareness or understanding of the environment by organizing and interpreting sensory information. All perception involves signals in the nervous system, which in turn result from physical stimulation of the sense organs...
, so long as the signal is brief (to prevent the problem of imprecise timing) and consistent (to prevent the problem of variations that may confuse the animal). For large sea animals the marker is usually a whistle rather than a clicker. However, not all conditioned reinforcers are sounds. Goldfish
Goldfish
The goldfish is a freshwater fish in the family Cyprinidae of order Cypriniformes. It was one of the earliest fish to be domesticated, and is one of the most commonly kept aquarium fish....
and birds such as falcons and hawks can be trained using a quick flash of a flashlight as their "clicker". Deaf dogs can be trained with a vibrating collar.
As pointed out by Lindsay the advantages of the clicker may be particularly strong in some situations: "...the clicker's simplicity and clarity provide a significant advantage for some training activities..."
Controversy
There are several common objections posed to clicker training. Proponents assert that while most of these can be a problem for the unskilled clicker trainer, these are all avoidable.- "The dog will never perform the behavior without the clicker." The clicker should be used to identify correct behavior during training, not to maintain behavior once the behavior has been learned. Once a behavior is performed each time the animal hears a specific cue (known as a command in traditional training), the clicker is discontinued.
- "Dogs will become distracted by the clicks of other trainers in a class or public setting." This is very short-lived problem. Participants in clicker classes find that dogs are easily able to discriminate that only the clicks from their handler pay off. Clicks that don't pay off are soon ignored by animals in learning situations.
- "Dogs become fat with clicker training because they get too many treats." Part 1 of the solution to this problem is either to use a portion of the dog's regular diet as the training treats or to use reinforcers other than food. Part 2 is to remember that a training treat for a dog the size of a Labrador RetrieverLabrador RetrieverThe Labrador Retriever is one of several kinds of retriever, a type of gun dog. A breed characteristic is webbed paws for swimming, useful for the breed's original purpose of retrieving fishing nets. The Labrador is the most popular breed of dog by registered ownership in Canada, the United...
should be about the size of a pea or an M&MM&MM&M may refer to:* M&M's, a chocolate confectionery coated with hard candy shell* Eminem, stage name of rap artist Marshall Mathers III* "M+M's", a song by the American band Blink-182 from Cheshire Cat...
. Smaller dogs get even smaller treats. Larger dogs get only slightly larger treats. Food is not the only reinforcer that can be used in training. A "reinforcer" is anything the animal is willing to work for in the current situation. Common non-food reinforcers include toys, attention, and the opportunity to do something the dog wants. For example, for a dog who wants to go for a walk, putting on the leash can reinforce sitting, going through the door can reinforce the dog who wants to go outside, and being greeted can reinforce a dog seeking attention. - "You can't clicker train in noisy environments." The influence of environmental reinforcers is a challenge sometimes. Training for distractions is done by first training without distractions and then gradually adding complexity to the training environment.
- "A dog may grow into adulthood and only listen and obey if the owner is carrying treats. If the owner does not have treats, often is the case that the dog is distracted and paying attention to whoever may have treats and food rewards available." This is actually a potential problem with the "Lure Reward" method of training where food is visible. In clicker training the food should not be visible to the animals until the behavior is completed. This could also happen when the trainer uses only one type of reinforcer. If the trainer uses only food, then the dog clearly learns that if food isn't present, then there can be no reinforcement. This is a trainer error. The solution is to use a variety of types of reinforcers and to hold training sessions where food isn't present. Also, you can include running to get the reinforcer into the reinforcement sequence.
- "There are some situations where a clicker may not be loud enough, such as in hunting or retrieving when the dog is 'working away' from the handler." The clicker is not magic; it is just one type of marker. If the dog can't hear the click, use a different marker such as a whistle or a tone on a collar. Deaf dogs are frequently trained with a flash of light or a hand signal.
- "Some dogs are sensitive to noise and frightened by a clicker, so clicker training won't work for them." If your dog is afraid of the clicker, then simply choose a different marker—perhaps even just a word, the clicking of a retractible pen, or a juice cap.
Methodology
The first step in clicker training is to teach the animal that the clicker sound means that they will get a primary reinforcer, usually food. To do this, some trainers "charge" or "load" the clicker. To do this the trainer clicks the clicker and immediately thereafter gives the animal a reward, usually a tasty treat, one small enough to be consumed almost instantly. Some animals tend to learn the association much more quickly than others. Progress may be tested by waiting until the dog's attention is elsewhere and then clicking. If the dog immediately looks toward the trainer as though expecting a reward, it is likely that the dog has made the association.Other trainers, including Bob Bailey and the ABE Trainers, simply start training a behavior and following desired approximations with a click. ABE conducted experiments that demonstrated that for their purposes, where they may be training many animals at the same time, this method was more efficient. Today many clicker trainers use this method of introducing the clicker.
After that, the trainer uses the clicker to mark desired behaviors as they occur. At the exact instant the animal performs the desired behavior, the trainer clicks and promptly delivers a food reward or other reinforcer. One key to clicker training is the trainer's timing; clicking slightly too early or too late rewards and therefore may reinforce whatever behavior is occurring at that instant. The saying goes, "you get what you click for."
Clicker trainers often use the process of "shaping," which means gradually transforming a specific behavior into the desired behavior by rewarding successive approximations to it. A successive approximation is "a behavioral term that refers to gradually molding or training an organism to perform a specific [completed] response by [first] reinforcing responses that are similar to the desired response." Clicker trainers learn to split behavior instead of lumping it, i.e. to look for and reward small steps in the right direction rather than waiting for the whole, "perfect" behavior to appear on its own. It is important to create opportunities for the animal to earn rewards very frequently. A reinforcement rate of one click/treat (C/T) every two to three seconds is common among professional dog trainers. Criteria for receiving the click is tightened gradually, at the rate the animal is comfortable with and so that it will remain successful.
Examples
Training the nose touch begins with getting the dog to touch a target with its nose; trainers sometimes use a guided method, such as placing a dab of peanut butter on a small plate or plastic target; others prefer shaping, where the target is placed in easy reach, such as in the trainer's hand between the trainer and the dog, and the dog is rewarded each time he moves in the target's direction or actually touches it.
When the dog is consistently touching the target, the trainer progresses to a target with and without food and in different positions. Eventually, the trainer can transfer the behavior to a bell, for example by holding the target behind the bell so that the dog has to touch the bell to get at the target, and then rewarding the touching of the bell. When the dog is reliably touching the bell, the trainer now adds the act of opening the door to the reward each time the dog strikes the bell.
Targeting for Horses:
For horses, loading or charging the clicker is usually not done. It's best for horses that a clear marker is used so that the horse does not expect "unearned" treats.
See also
- Animal trainingAnimal trainingAnimal training refers to teaching animals specific responses to specific conditions or stimuli. Training may be for the purpose of companionship, detection, protection, entertainment or all of the above....
- Dog trainingDog trainingDog training is the process of teaching skills or behaviors to a dog. This can include teaching a dog to respond to certain commands, or helping the dog learn coping skills for stressful environments. Dog training often includes operant conditioning, classical conditioning, or non-associative...
- Operant conditioningOperant conditioningOperant conditioning is a form of psychological learning during which an individual modifies the occurrence and form of its own behavior due to the association of the behavior with a stimulus...
- B. F. SkinnerB. F. SkinnerBurrhus Frederic Skinner was an American behaviorist, author, inventor, baseball enthusiast, social philosopher and poet...
- Marian Breland BaileyMarian Breland BaileyMarian Breland Bailey, born Marian Ruth Kruse and nicknamed "Mouse", was an American psychologist, an applied behavior analyst who played a major role in developing empirically validated and humane animal training methods and in promoting their widespread implementation...
- Karen Pryor
External links
- Reference article on Clicker Training your dog or puppy
- Canis Film Festival Clicker training video contest entries
- Nose Licks & Tail Wags - clicker training and dog care articles
- Deaf Dog Training
- Interview Marian and Bob Bailey
- ClickerSolutions Large collection of articles on clicker training
- Guide Dogs For The Blind service dog program now using clicker training
- The Parrot Forum debate about the merits of clicker training for parrots