E-assessment
Encyclopedia
In its broadest sense, e-assessment is the use of information technology
for any assessment-related activity. This definition embraces a wide range of student activity ranging from the use of a word processor
to on-screen testing. Due to its obvious similarity to e-learning
, the term e-assessment is becoming widely used as a generic term to describe the use of computer
s within the assessment process. Specific types of e-assessment include computerized adaptive testing and computerized classification test
ing.
E-assessment can be used to assess cognitive and practical abilities. Cognitive abilities are assessed using e-testing software; practical abilities are assessed using e-portfolios or simulation software.
. Once created, the engine uses the item bank to generate a test. Traditional paper-and-pencil testing is similar, but the test is pulled from the bank at only one time, when it is sent to publishing.
The creation of the item bank is more costly and time consuming than the installation and configuration of the assessment engine. This is due to the fact that assessment engines can be bought "off the shelf" whereas an item bank must be developed for each specific application.
An e-assessment system designed to focus on more sophisticated forms of knowledge requires some sort of interactive activity and a system for inviting students to reason or solve problems around that activity. One influential program of research is known as Evidence Centered Design, or ECD. ECD involves the use of Bayesian Inference Nets to create a sophisticated model of student cognition, and a set of activities or problems that students work on that allow the system to estimate the individuals understanding of the particular domain.
There are also disadvantages. E-assessment systems are expensive to establish and not suitable for every type of assessment (such as extended response questions). The main expense is not technical; it is the cost of producing high quality assessment items - although this cost is identical when using paper-based assessment.
The best examples follow a Formative Assessment
structure and are called "Online Formative Assessment". This involves making an initial formative assessment
by sifting out the incorrect answers. The author/teacher will then explain what the pupil should have done with each question. It will then give the pupil at least one practice at each slight variation of sifted out questions. This is the formative learning stage. The next stage is to make a Summative Assessment
by a new set of questions only covering the topics previously taught. Some will take this even further and repeat the cycle such as BOFA Online 11 plus papers which is aimed at the eleven plus exam set in the UK.
) provides a common format for describing and distributing question items across disparate systems.
.
Although these terms are commonly used interchangeably, they have distinct meanings.
Computer Assisted/Mediated Assessment refers to any application of computers within the assessment process; the role of the computer may be extrinsic or intrinsic. It is, therefore, a synonym for e-assessment which also describes a wide range of computer-related activities. Within this definition the computer often plays no part in the actual assessment of responses but merely facilitates the capture and transfer of responses between candidate and human assessor.
Computer-Based Assessment refers to assessment which is built around the use of a computer; the use of a computer is always intrinsic to this type of assessment. This can relate to assessment of IT practical skills or more commonly the on screen presentation of knowledge tests. The defining factor is that the computer is marking or assessing the responses provided from candidates.
Online assessment refers to assessment activity which requires the use of the internet
. In reality few high stakes assessment sessions are actually conducted online in real time but the transfer of data prior to and after the assessment session is conducted via the internet. There are many examples of practice and diagnostic tests being run real time over the internet.
Information technology
Information technology is the acquisition, processing, storage and dissemination of vocal, pictorial, textual and numerical information by a microelectronics-based combination of computing and telecommunications...
for any assessment-related activity. This definition embraces a wide range of student activity ranging from the use of a word processor
Word processor
A word processor is a computer application used for the production of any sort of printable material....
to on-screen testing. Due to its obvious similarity to e-learning
E-learning
E-learning comprises all forms of electronically supported learning and teaching. The information and communication systems, whether networked learning or not, serve as specific media to implement the learning process...
, the term e-assessment is becoming widely used as a generic term to describe the use of computer
Computer
A computer is a programmable machine designed to sequentially and automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem...
s within the assessment process. Specific types of e-assessment include computerized adaptive testing and computerized classification test
Computerized classification test
A computerized classification test refers to, as its name would suggest, a test that is administered by computer for the purpose of classifying examinees. The most common CCT is a mastery test where the test classifies examinees as "Pass" or "Fail," but the term also includes tests that classify...
ing.
E-assessment can be used to assess cognitive and practical abilities. Cognitive abilities are assessed using e-testing software; practical abilities are assessed using e-portfolios or simulation software.
Components
An e-testing system designed to focus on lower level associations comprises two components: (1) an assessment engine; and (2) an item bank. An assessment engine comprises the hardware and software required to create and deliver a test. Most e-testing engines run on standard hardware so the key characteristic is the software's functionality. There is a wide range of software packages. The software does not include the questions themselves; these are provided by an item bankItem bank
An item bank is a term for a repository of test items that belong to a testing program, as well as all information pertaining to those items. In most applications of testing and assessment, the items are of multiple choice format, but any format can be used...
. Once created, the engine uses the item bank to generate a test. Traditional paper-and-pencil testing is similar, but the test is pulled from the bank at only one time, when it is sent to publishing.
The creation of the item bank is more costly and time consuming than the installation and configuration of the assessment engine. This is due to the fact that assessment engines can be bought "off the shelf" whereas an item bank must be developed for each specific application.
An e-assessment system designed to focus on more sophisticated forms of knowledge requires some sort of interactive activity and a system for inviting students to reason or solve problems around that activity. One influential program of research is known as Evidence Centered Design, or ECD. ECD involves the use of Bayesian Inference Nets to create a sophisticated model of student cognition, and a set of activities or problems that students work on that allow the system to estimate the individuals understanding of the particular domain.
Advantages
E-assessment is becoming widely used. It has many advantages over traditional (paper-based) assessment. The advantages include:- lower long-term costs
- instant feedback to students
- greater flexibility with respect to location and timing
- improved reliability (machine marking is much more reliable than human marking)
- improved impartiality (machine marking does not 'know' the students so does not favour nor make allowances for minor errors)
- greater storage efficiency - tens of thousands of answer scripts can be stored on a server compared to the physical space required for paper scripts
- enhanced question styles which incorporate interactivity and multimedia.
There are also disadvantages. E-assessment systems are expensive to establish and not suitable for every type of assessment (such as extended response questions). The main expense is not technical; it is the cost of producing high quality assessment items - although this cost is identical when using paper-based assessment.
The best examples follow a Formative Assessment
Formative Assessment
Formative assessment is a range of formal and informal assessment procedures employed by teachers during the learning process in order to modify teaching and learning activities to improve student attainment. It typically involves qualitative feedback for both student and teacher that focuses on...
structure and are called "Online Formative Assessment". This involves making an initial formative assessment
Formative assessment
Formative assessment is a range of formal and informal assessment procedures employed by teachers during the learning process in order to modify teaching and learning activities to improve student attainment. It typically involves qualitative feedback for both student and teacher that focuses on...
by sifting out the incorrect answers. The author/teacher will then explain what the pupil should have done with each question. It will then give the pupil at least one practice at each slight variation of sifted out questions. This is the formative learning stage. The next stage is to make a Summative Assessment
Summative Assessment
Summative assessment refers to the assessment of the learning and summarizes the development of learners at a particular time. After a period of work, e.g. a unit for two weeks, the learner sits for a test and then the teacher marks the test and assigns a score. The test aims to summarize learning...
by a new set of questions only covering the topics previously taught. Some will take this even further and repeat the cycle such as BOFA Online 11 plus papers which is aimed at the eleven plus exam set in the UK.
E-assessment standards
In order to create a mechanism for the sharing of high quality assessment items, global standards have emerged. The IMS Question and Test Interoperability specification (QTIQTI
The IMS Question and Test Interoperability specification defines a standard format for the representation of assessment content and results, supporting the exchange of this material between authoring and delivery systems, repositories and other learning management systems. It allows assessment...
) provides a common format for describing and distributing question items across disparate systems.
Hand-held student response systems
An area of E-assessment that has seen extensive growth in recent years is the use of hand held student response devices (often referred to as clickers or voting devices). These allow a teacher to carry out whole group assessments, polls and surveys quickly and easily. They use either radio or infra red to communicate with a central hub that is usually attached to a computer. In many school classrooms these devices may also be used in combination with an interactive whiteboardInteractive whiteboard
An interactive whiteboard , is a large interactive display that connects to a computer and projector. A projector projects the computer's desktop onto the board's surface where users control the computer using a pen, finger, stylus, or other device...
.
Note on terminology
Various terms are used to describe the use of a computer for assessment purposes. These include:- Computer-Assisted Assessment or Computer-Aided Assessment (CAA)
- Computer-Mediated Assessment (CMA)
- Computer-Based Assessment (CBA)
- online assessment.
Although these terms are commonly used interchangeably, they have distinct meanings.
Computer Assisted/Mediated Assessment refers to any application of computers within the assessment process; the role of the computer may be extrinsic or intrinsic. It is, therefore, a synonym for e-assessment which also describes a wide range of computer-related activities. Within this definition the computer often plays no part in the actual assessment of responses but merely facilitates the capture and transfer of responses between candidate and human assessor.
Computer-Based Assessment refers to assessment which is built around the use of a computer; the use of a computer is always intrinsic to this type of assessment. This can relate to assessment of IT practical skills or more commonly the on screen presentation of knowledge tests. The defining factor is that the computer is marking or assessing the responses provided from candidates.
Online assessment refers to assessment activity which requires the use of the internet
Internet
The Internet is a global system of interconnected computer networks that use the standard Internet protocol suite to serve billions of users worldwide...
. In reality few high stakes assessment sessions are actually conducted online in real time but the transfer of data prior to and after the assessment session is conducted via the internet. There are many examples of practice and diagnostic tests being run real time over the internet.
External links
- The e-Assessment Association is a professional body for e-Assessment.
- International Annual Conference on e-Assessment.
- Transforming Assessment has a rage of examples of e-assessment in web 2.0 and virtual world learning environments (Based at the University of Adelaide, Australia)
- e-assessment section at JISC, UK.
- e-Learning Centre, UK has tools for e-assessment.
- eAssessment at the Open University, UK shows how to use an opensource computer-assisted assessment (CAA) system called OpenMark designed to mark less structured questions.
- The e-Assessment Handbook includes guidelines and examples (a print book).
- Independent, vendor neutral e-Assessment news from around the world independent news blog for all things e-Assessment.