Component-Based Usability Testing
Encyclopedia
Component-based usability testing (CBUT) is a testing approach which aims at empirically testing the usability
of an interaction component. The latter is defined as an elementary unit of an interactive system, on which behaviour-based evaluation is possible. For this, a component needs to have an independent, and by the user perceivable and controllable state, such as a radio button, a slider or a whole word processor application. The CBUT approach can be regarded as part of component-based software engineering
branch of software engineering
.
views where a person’s mental process is split up in smaller mental processes. Both software architecture and cognitive architecture use the principle of hierarchical layering, in which low level processes are more elementary and for humans often more physical in nature, such as the coordination movement of muscle groups. Processes that operate on higher level layers are more abstract and focus on a person’s main goal, such as writing an application letter to get a job. The Layered Protocol Theory (LPT), which is a special version of Perceptual Control Theory
(PCT), brings these views together by suggesting that users interact with a system across several layers by sending messages. Users interact with components on high layers by sending messages, such as pressing keys, to components operating on lower layers, which on their turn relay a series of these messages into a single high level message, such as ‘DELETE *.*’, to a component on a higher layer. Components operating on higher layers, communicate back to the user by sending messages to components operating on lower level layers. Whereas this layered-interaction model explains how the interaction is established, control loops explain the purpose of the interaction. LPT sees the purpose of the users’ behaviour as the users’ attempt to control their perception, in this case the state of the component they perceive. This means that users will only act if they perceive the component to be in an undesirable state. For example, if a person have an empty glass but want a full glass of water, he or she will act (e.g. walk to the tap, turning the tap on to fill the glass). The action of filling the glass will continue until the person perceives the glass as full. As interaction with components takes places on several layers, interacting with a single device can include several control loops. The amount of effort put into operating a control loop is seen as an indicator for the usability of an interaction component.
Meta-analysis on the data from several lab experiments that used CBUT measures suggests that these measures can be statistically more powerful than overall (holistic) usability measures .
(SUS) examine the usability of a system on several dimensions such as defined in ISO 9241
Part 11 standard effectiveness, efficiency and satisfaction, a Component-Based Usability Questionnaire (CBUQ)
is a questionnaire which can be used to evaluate the usability of individual interaction components, such as the volume control or the play control of a MP3 player. To evaluate an interaction component, the six Perceived Ease-Of-Use (PEOU) statements from the Technology acceptance model
are taken with a reference to the interaction component, instead of to the entire system, for example:
Learning to operate the Volume Control would be easy for me.
Users are asked to rate these statements on a seven point Likert scale
. The average rating on these six statements is regarded as the user’s usability rating of the interaction component. Based on lab studies with difficult to use interaction components and easy to use interaction components, a break-even point of 5.29 on seven point Likert scale has been determined. Using a One-sample student's t-test
it is possible to examine whether users’ rating of an interaction component deviates from this break-even point. Interaction components that receive rating below this break-even point can be regarded as more comparable to the set of difficult to use interaction components, whereas ratings above this break-even point would be more comparable to the set if easy to use interaction components.
If engineers like to evaluate multiple interaction components simultaneously, the CBUQ questionnaire exists of separate sections, one for each interaction component, each with their own 6 PEOU statements.
Usability
Usability is the ease of use and learnability of a human-made object. The object of use can be a software application, website, book, tool, machine, process, or anything a human interacts with. A usability study may be conducted as a primary job function by a usability analyst or as a secondary job...
of an interaction component. The latter is defined as an elementary unit of an interactive system, on which behaviour-based evaluation is possible. For this, a component needs to have an independent, and by the user perceivable and controllable state, such as a radio button, a slider or a whole word processor application. The CBUT approach can be regarded as part of component-based software engineering
Component-based software engineering
Component-based software engineering is a branch of software engineering that emphasizes the separation of concerns in respect of the wide-ranging functionality available throughout a given software system...
branch of software engineering
Software engineering
Software Engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software...
.
Theory
CBUT is based on both software architectural views such as model–view–controller (MVC), presentation–abstraction–control (PAC), ICON and CNUCE agent models that split up the software in parts, and cognitive psychologyCognitive psychology
Cognitive psychology is a subdiscipline of psychology exploring internal mental processes.It is the study of how people perceive, remember, think, speak, and solve problems.Cognitive psychology differs from previous psychological approaches in two key ways....
views where a person’s mental process is split up in smaller mental processes. Both software architecture and cognitive architecture use the principle of hierarchical layering, in which low level processes are more elementary and for humans often more physical in nature, such as the coordination movement of muscle groups. Processes that operate on higher level layers are more abstract and focus on a person’s main goal, such as writing an application letter to get a job. The Layered Protocol Theory (LPT), which is a special version of Perceptual Control Theory
Perceptual control theory
Perceptual control theory is a model of the psychological and behavioral processes occurring within living beings, including humans. It demonstrates that animals are goal-driven, purposeful entities rather than automata repeating conditioned responses to external stimuli or computers planning...
(PCT), brings these views together by suggesting that users interact with a system across several layers by sending messages. Users interact with components on high layers by sending messages, such as pressing keys, to components operating on lower layers, which on their turn relay a series of these messages into a single high level message, such as ‘DELETE *.*’, to a component on a higher layer. Components operating on higher layers, communicate back to the user by sending messages to components operating on lower level layers. Whereas this layered-interaction model explains how the interaction is established, control loops explain the purpose of the interaction. LPT sees the purpose of the users’ behaviour as the users’ attempt to control their perception, in this case the state of the component they perceive. This means that users will only act if they perceive the component to be in an undesirable state. For example, if a person have an empty glass but want a full glass of water, he or she will act (e.g. walk to the tap, turning the tap on to fill the glass). The action of filling the glass will continue until the person perceives the glass as full. As interaction with components takes places on several layers, interacting with a single device can include several control loops. The amount of effort put into operating a control loop is seen as an indicator for the usability of an interaction component.
Testing
CBUT can be categorized according to two testing paradigms, the single-version testing paradigm (SVTP) and the multiple-versions testing paradigm (MVTP). In SVTP only one version of each interaction component in a system is tested. The focus is to identify interaction components that might reduce the overall usability of the system. SVTP is therefore suitable as part of a software-integration test. In MVTP on the other hand, multiple versions of a single component are tested while the remaining components in the system remain unchanged. The focus is on identifying the version with the highest usability of specific interaction component. MVTP therefore is suitable for component development and selection. Different CBUT methods have been proposed for SVTP and MVTP, which include measures based on recorded user interaction and questionnaires. Whereas in MVTP the recorded data can directly be interpreted by making a comparison between two versions of the interaction component, in SVTP log file analysis is more extensive as interaction with both higher and lower components must be considered .Meta-analysis on the data from several lab experiments that used CBUT measures suggests that these measures can be statistically more powerful than overall (holistic) usability measures .
Usability questionnaire
Whereas holistic oriented usability questionnaires such as system usability scaleSystem Usability Scale
In systems engineering, the system usability scale is a simple, ten-item attitude Likert scale giving a global view of subjective assessments of usability...
(SUS) examine the usability of a system on several dimensions such as defined in ISO 9241
ISO 9241
ISO 9241 is a multi-part standard from the International Organization for Standardization covering ergonomics of human-computer interaction. It is managed by the ISO...
Part 11 standard effectiveness, efficiency and satisfaction, a Component-Based Usability Questionnaire (CBUQ)
is a questionnaire which can be used to evaluate the usability of individual interaction components, such as the volume control or the play control of a MP3 player. To evaluate an interaction component, the six Perceived Ease-Of-Use (PEOU) statements from the Technology acceptance model
Technology acceptance model
The Technology Acceptance Model is an information systems theory that models how users come to accept and use a technology. The model suggests that when users are presented with a new technology, a number of factors influence their decision about how and when they will use it, notably:* Perceived...
are taken with a reference to the interaction component, instead of to the entire system, for example:
Learning to operate the Volume Control would be easy for me.
Users are asked to rate these statements on a seven point Likert scale
Likert scale
A Likert scale is a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale, or more accurately the Likert-type scale, even though...
. The average rating on these six statements is regarded as the user’s usability rating of the interaction component. Based on lab studies with difficult to use interaction components and easy to use interaction components, a break-even point of 5.29 on seven point Likert scale has been determined. Using a One-sample student's t-test
Student's t-test
A t-test is any statistical hypothesis test in which the test statistic follows a Student's t distribution if the null hypothesis is supported. It is most commonly applied when the test statistic would follow a normal distribution if the value of a scaling term in the test statistic were known...
it is possible to examine whether users’ rating of an interaction component deviates from this break-even point. Interaction components that receive rating below this break-even point can be regarded as more comparable to the set of difficult to use interaction components, whereas ratings above this break-even point would be more comparable to the set if easy to use interaction components.
If engineers like to evaluate multiple interaction components simultaneously, the CBUQ questionnaire exists of separate sections, one for each interaction component, each with their own 6 PEOU statements.
See also
- Cognitive ergonomicsCognitive ergonomicsAccording to the International Ergonomics Association, by definition, "Cognitive ergonomics is concerned with mental processes, such as perception, memory, reasoning, and motor response, as they affect interactions among humans and other elements of a system...
- Human–computer interactionHuman–computer interactionHuman–computer Interaction is the study, planning, and design of the interaction between people and computers. It is often regarded as the intersection of computer science, behavioral sciences, design and several other fields of study...
- Usability engineeringUsability engineeringUsability engineering is a field that is concerned generally with human-computer interaction and specifically with making human-computer interfaces that have high usability or user friendliness...
- Usability testingUsability testingUsability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system...
External links
- Example study that uses component-based usability questionnaire including instructions, questionnaires, data analysis and additional instructions .