Partial concurrent thinking aloud
Encyclopedia
Partial Concurrent Thinking Aloud (or partial concurrent think-aloud, or PCTA) is a method used to gather data in usability testing
with screen reader
users. It is a particular kind of think aloud protocol
(or TAP) created by Stefano Federici and Simone Borsci at the Interuniversity Center for Research on Cognitive Processing in Natural and Artificial Systems of University of Rome "La Sapienza". The Partial Concurrent Thinking Aloud is built up in order to create a specific usability assessment technique for blind users, eligible to maintain the advantages of concurrent and retrospective thinking aloud while overcoming their limits. Using PCTA blind users’ verbalizations of problems could be more pertinent and comparable to those given by sighted people who use a concurrent protocol. In the usability evaluation with blind people, the retrospective thinking aloud is often adopted as a functional solution to overcome the structural interference due to thinking aloud and hearing the screen reader imposed by the classic thinking aloud technique; such a solution has yet a relapse in the evaluation method, because the concurrent and the retrospective protocols measure usability from different points of view, one mediated by navigation experience (retrospective) one more direct and pertinent (concurrent) . The use of PCTA could be widened to both summative and formative usability evaluations with mixed panels of users, thus extending the number of problems' verbalizations according to disabled users’ divergent navigation processes and problem solving strategies.
and talking about the way of interacting with the computer implie a structural interference between action and verbalization. Undoubtedly, cognitive studies provided a lot of evidence supporting the idea that individuals can listen, verbalize, or manipulate, and rescue information in multiple task condition. As Colin Cherry
showed, subjects, when listening to two different messages from a single loudspeaker, can separate sounds from background noise, recognize the gender of the speaker, the direction, and the pitch (cocktail party effect
). At the same time, subjects that must verbalize the content of a message (attended message) listening to two different message simultaneously (attended and unattended message) have a reduced ability to report the content of the attended massage, while they are unable to report the content of the unattended message. Moreover, K. Anders Ericsson
and Walter Kintsch showed that, in a multiple task condition, subjects’ ability of rescuing information is not compromised by an interruption of the action flow (as it happens in the concurrent thinking aloud technique), thanks to the “Long Term Working Memory mechanism” of information retrieval (Working Memory
section Ericsson and Kintsch).
Even if users can listen, recognize, and verbalize multiple messages in a multiple task condition and they can stop and restart actions without losing any information, other cognitive studies underlined that the overlap of activities in a multiple task condition have an effect on the goal achievement: Kemper, Herman and Lian , analysing the users' abilities to verbalize actions in a multiple task condition, showed that the fluency of a user’s conversation is influenced by the overlap of actions. Adults are likely to continue to talk as they navigate in a complex physical environment. However, the fluency of their conversation is likely to change: Older adults are likely to speak more slowly than they would if resting; Young adults continue to speak just as rapidly while walking as while resting, but they adopt a further set of speech accommodations, reducing sentence length, grammatical complexity, and propositional density. Just by reducing length, complexity, and propositional density adults free up working memory resources.
We do not know how and how much the content of verbalizations could be influenced by the strategy of verbalization (i.e. the modification of fluency and the complexity in a multiple task condition). Anyway, we well know that users in the concurrent thinking aloud verbalize the problems in a more accurate and pertinent way (i.e. more focused on the problems directly perceived during the interaction) then in the retrospective one . The pertinence is granted to the user by the proximity of action-verbalization-next action; this multiple task proximity compels the subject to apply a strategy of verbalization that reduce the overload of the working memory. However, for blind users this time proximity between action and verbalization is lost: the use of the screen reader, in fact, increase the time for verbalization (i.e. in order to verbalize, blind users must first stop the [screen reader] and then restart it).
The first section is a modified concurrent protocol built up according to the three concurrent verbal protocols criteria described by K. Anders Ericsson
and Herbert Simon
:
The second PCTA section is a retrospective one in which users analyse those problems previously verbalized in a concurrent way. The memory signs, created by users ringing the desk-bell, overcome the limits of classic retrospective analysis; indeed, these signs allow the users to be pertinent and consistent with their concurrent verbalization, thus avoiding the influence of long term memory and perceptual reworking.
Usability testing
Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system...
with screen reader
Screen reader
A screen reader is a software application that attempts to identify and interpret what is being displayed on the screen . This interpretation is then re-presented to the user with text-to-speech, sound icons, or a Braille output device...
users. It is a particular kind of think aloud protocol
Think aloud protocol
Think-aloud protocol is a method used to gather data in usability testing in product design and development, in psychology and a range of social sciences...
(or TAP) created by Stefano Federici and Simone Borsci at the Interuniversity Center for Research on Cognitive Processing in Natural and Artificial Systems of University of Rome "La Sapienza". The Partial Concurrent Thinking Aloud is built up in order to create a specific usability assessment technique for blind users, eligible to maintain the advantages of concurrent and retrospective thinking aloud while overcoming their limits. Using PCTA blind users’ verbalizations of problems could be more pertinent and comparable to those given by sighted people who use a concurrent protocol. In the usability evaluation with blind people, the retrospective thinking aloud is often adopted as a functional solution to overcome the structural interference due to thinking aloud and hearing the screen reader imposed by the classic thinking aloud technique; such a solution has yet a relapse in the evaluation method, because the concurrent and the retrospective protocols measure usability from different points of view, one mediated by navigation experience (retrospective) one more direct and pertinent (concurrent) . The use of PCTA could be widened to both summative and formative usability evaluations with mixed panels of users, thus extending the number of problems' verbalizations according to disabled users’ divergent navigation processes and problem solving strategies.
Cognitive assumptions of Partial Concurrent Thinking Aloud
In general, in the usability evaluation both retrospective and concurrent TAP could be used according to the aims and goals of the study. Nevertheless, when a usability evaluation is carried out with blind people several studies propose to use the retrospective TAP: indeed, using a screen readerScreen reader
A screen reader is a software application that attempts to identify and interpret what is being displayed on the screen . This interpretation is then re-presented to the user with text-to-speech, sound icons, or a Braille output device...
and talking about the way of interacting with the computer implie a structural interference between action and verbalization. Undoubtedly, cognitive studies provided a lot of evidence supporting the idea that individuals can listen, verbalize, or manipulate, and rescue information in multiple task condition. As Colin Cherry
Colin Cherry
Edward Colin Cherry was a British cognitive scientist whose main contributions were in focused auditory attention, specifically regarding the cocktail party problem. This concerns the problem of following only one conversation while many other conversations are going on around us...
showed, subjects, when listening to two different messages from a single loudspeaker, can separate sounds from background noise, recognize the gender of the speaker, the direction, and the pitch (cocktail party effect
Cocktail party effect
The cocktail party effect describes the ability to focus one's listening attention on a single talker among a mixture of conversations and background noises, ignoring other conversations. The effect enables most people to talk in a noisy place...
). At the same time, subjects that must verbalize the content of a message (attended message) listening to two different message simultaneously (attended and unattended message) have a reduced ability to report the content of the attended massage, while they are unable to report the content of the unattended message. Moreover, K. Anders Ericsson
Anders Ericsson
Dr. K. Anders Ericsson is a Swedish psychologist and Conradi Eminent Scholar and Professor of Psychology at Florida State University who is widely recognized as one of the world's leading theoretical and experimental researchers on expertise....
and Walter Kintsch showed that, in a multiple task condition, subjects’ ability of rescuing information is not compromised by an interruption of the action flow (as it happens in the concurrent thinking aloud technique), thanks to the “Long Term Working Memory mechanism” of information retrieval (Working Memory
Working memory
Working memory has been defined as the system which actively holds information in the mind to do verbal and nonverbal tasks such as reasoning and comprehension, and to make it available for further information processing...
section Ericsson and Kintsch).
Even if users can listen, recognize, and verbalize multiple messages in a multiple task condition and they can stop and restart actions without losing any information, other cognitive studies underlined that the overlap of activities in a multiple task condition have an effect on the goal achievement: Kemper, Herman and Lian , analysing the users' abilities to verbalize actions in a multiple task condition, showed that the fluency of a user’s conversation is influenced by the overlap of actions. Adults are likely to continue to talk as they navigate in a complex physical environment. However, the fluency of their conversation is likely to change: Older adults are likely to speak more slowly than they would if resting; Young adults continue to speak just as rapidly while walking as while resting, but they adopt a further set of speech accommodations, reducing sentence length, grammatical complexity, and propositional density. Just by reducing length, complexity, and propositional density adults free up working memory resources.
We do not know how and how much the content of verbalizations could be influenced by the strategy of verbalization (i.e. the modification of fluency and the complexity in a multiple task condition). Anyway, we well know that users in the concurrent thinking aloud verbalize the problems in a more accurate and pertinent way (i.e. more focused on the problems directly perceived during the interaction) then in the retrospective one . The pertinence is granted to the user by the proximity of action-verbalization-next action; this multiple task proximity compels the subject to apply a strategy of verbalization that reduce the overload of the working memory. However, for blind users this time proximity between action and verbalization is lost: the use of the screen reader, in fact, increase the time for verbalization (i.e. in order to verbalize, blind users must first stop the [screen reader] and then restart it).
Protocol of Partial Concurrent Thinking Aloud
PCTA method is composed of two sections, one concurrent and one retrospective:The first section is a modified concurrent protocol built up according to the three concurrent verbal protocols criteria described by K. Anders Ericsson
Anders Ericsson
Dr. K. Anders Ericsson is a Swedish psychologist and Conradi Eminent Scholar and Professor of Psychology at Florida State University who is widely recognized as one of the world's leading theoretical and experimental researchers on expertise....
and Herbert Simon
Herbert Simon
Herbert Alexander Simon was an American political scientist, economist, sociologist, and psychologist, and professor—most notably at Carnegie Mellon University—whose research ranged across the fields of cognitive psychology, cognitive science, computer science, public administration, economics,...
:
| The first criterion |
|---|
| Subjects should be talking about the task at hand, not about an unrelated issue. In order to respect this rule, the time between problem retrieval, thinking and verbalization must be minimized to avoid the influence of a long perceptual reworking and the consequent verbalization of unrelated issues. Blind participants, using a screen reader, increase the time latency between identification and verbalization of a problem. To minimize this latency, users are trained to ring a desk-bell that stops both time and navigation. During this suspension, users can create a memory sign (i.e. ring the bell) and restart immediately the navigation. This setting modification allows to avoid the cognitive limitation problem and the influence of perceptual reworking, also creating a memory sign for the retrospective analysis. |
| The second criterion |
| To be pertinent, verbalizations should be logically consistent with the verbalizations that just preceded them. For any kind of user it is hard to be pertinent and consistent in a concurrent verbal protocol. Therefore, the practitioners could generally interrupt the navigation and ask for a clarification or stimulate the users to verbalize in a pertinent way. In order to do so and stop navigation to screen reader users, we propose to negotiate a specific physical sign with them: The practitioner, sitting behind the user, will put his hand on the user’s shoulder. This physical sign grants the verbalization pertinence and consistence. |
| The third criterion |
| A subset of the information needed during the task performance should be remembered. The concurrent model is based on the link between working memory and time latency. The proximity between the occurrence of a thought and its verbal report allows users to verbalize on the basis of their working memory. |
The second PCTA section is a retrospective one in which users analyse those problems previously verbalized in a concurrent way. The memory signs, created by users ringing the desk-bell, overcome the limits of classic retrospective analysis; indeed, these signs allow the users to be pertinent and consistent with their concurrent verbalization, thus avoiding the influence of long term memory and perceptual reworking.
See also
- Comparison of usability evaluation methodsComparison of usability evaluation methodsSource: Genise, Pauline. “Usability Evaluation: Methods and Techniques: Version 2.0” August 28, 2002. University of Texas.-See also:* Usability inspection* *Partial concurrent thinking aloud...
- Protocol analysisProtocol analysisProtocol analysis is a psychological research method that elicits verbal reports from research participants. Protocol analysis is used to study thinking in cognitive psychology , cognitive science , and behavior analysis...
- Think aloud protocolThink aloud protocolThink-aloud protocol is a method used to gather data in usability testing in product design and development, in psychology and a range of social sciences...