QTI
Encyclopedia
The IMS Question and Test Interoperability specification (QTI) defines a standard format for the representation of assessment content and results, supporting the exchange of this material between authoring and delivery systems, repositories and other learning management systems. It allows assessment materials to be authored and delivered on multiple systems interchangeably. It is, therefore, designed to facilitate interoperability
between systems.
The specification consists of a data model
that defines the structure of questions, assessments and results from questions and assessments together with an XML data binding
that essentially defines a language for interchanging questions and other assessment material. The XML
binding is widely used for exchanging questions between different authoring tools and by publishers. The assessment and results parts of the specification are less widely used.
Learning Consortium, which is an industry and academic consortium that develops specifications for interoperable learning technology. QTI was inspired by the need for interoperability in question design, and to avoid people losing or having to re-type questions when technology changes. Developing and validating good questions is time consuming, and it's desirable to be able to create them in a platform and technology neutral format.
QTI version 1.0 was materially based on a proprietary Questions Markup Language (QML) language defined by QuestionMark
, but the language has evolved over the years and can now describe almost any reasonable question that one might want to describe. (QML is still in use by Questionmark and is generated for interoperability by tools like Adobe Captivate).
The most widely used version of QTI at the time of writing is version 1.2, which was finalized in 2002. This works well for exchanging simple question types, and is supported by many tools that allow the creation of questions.
Version 2.0 was released in 2005, with v2.1 due for release in 2008. 2.0 addressed the item (individual question) level of the specification only, with 2.1 covering assessments and results as well as correcting errors which had become apparent in 2.0. Version 2.x is a significant improvement on earlier versions, defining a new underlying interaction model
. It is also notable for its significantly greater degree of integration with other specifications (some of which did not exist during the production of v1): the specification addresses the relationship with IMS Content Packaging
v1.2, IEEE Learning Object Metadata
, IMS Learning Design
, IMS Simple Sequencing and other standards such as XHTML
. It also provides guidance on representing context-specific usage data and information to support the migration of content from earlier versions of the specification.
Because v2.0 was limited to items only, and v2.1 has yet to be formally released by IMS (although two public drafts plus an addendum are currently available), uptake of v2.x has been slow to date. The delay between the release of 2.0 and 2.1 (over three years to date) may have hindered uptake to some extent, with developers reluctant to commit to v2.0 knowing that v2.1 is in development. The use of a profile of v1.2.1 in the IMS Common Cartridge specification may exacerbate this. A number of implementations are emerging, however, and uptake may increase once the specification is finally available in a stable form.
In early 2009, the IMS Global Learning Consortium withdrew QTI 2.1, stating that "Adequate feedback on the specification has not been received, and therefore, the specification has been put back into the IMS project group process for further work." The most recent version of QTI that is fully endorsed by IMS GLC is v1.2.1. This decision met with disapproval on the IMS-QTI mailing list.
A further clarification on the QTI 2.1 withdrawal acknowledged the work done on implementing the QTI 2.1 draft specification, and cited criticism on the lack of interoperability of IMS specifications as a reason for endorsing only IMS QTI 1.2. A few weeks later IMS GLC reposted the QTI v2.1 draft specification on their website with a warning that the specification is incomplete:
Other software:
Interoperability
Interoperability is a property referring to the ability of diverse systems and organizations to work together . The term is often used in a technical systems engineering sense, or alternatively in a broad sense, taking into account social, political, and organizational factors that impact system to...
between systems.
The specification consists of a data model
Data model
A data model in software engineering is an abstract model, that documents and organizes the business data for communication between team members and is used as a plan for developing applications, specifically how data is stored and accessed....
that defines the structure of questions, assessments and results from questions and assessments together with an XML data binding
XML data binding
XML data binding refers to a means of representing information in an XML document as an object in computer memory. This allows applications to access the data in the XML from the object rather than using the DOM or SAX to retrieve the data from a direct representation of the XML itself.An XML data...
that essentially defines a language for interchanging questions and other assessment material. The XML
XML
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
binding is widely used for exchanging questions between different authoring tools and by publishers. The assessment and results parts of the specification are less widely used.
Background
QTI was produced by the IMS GlobalIMS Global
IMS Global Learning Consortium is a global, nonprofit, member organization that strives to enable the growth and impact of learning technology in the education and corporate learning sectors worldwide...
Learning Consortium, which is an industry and academic consortium that develops specifications for interoperable learning technology. QTI was inspired by the need for interoperability in question design, and to avoid people losing or having to re-type questions when technology changes. Developing and validating good questions is time consuming, and it's desirable to be able to create them in a platform and technology neutral format.
QTI version 1.0 was materially based on a proprietary Questions Markup Language (QML) language defined by QuestionMark
QuestionMark
The Questionmark Corporation is a supplier of online assessment software to educational institutions, public sector employers and commercial companies worldwide. The company was started in 1988 by current chairman John Kleeman. Primary markets are Europe and North America...
, but the language has evolved over the years and can now describe almost any reasonable question that one might want to describe. (QML is still in use by Questionmark and is generated for interoperability by tools like Adobe Captivate).
The most widely used version of QTI at the time of writing is version 1.2, which was finalized in 2002. This works well for exchanging simple question types, and is supported by many tools that allow the creation of questions.
Version 2.0 was released in 2005, with v2.1 due for release in 2008. 2.0 addressed the item (individual question) level of the specification only, with 2.1 covering assessments and results as well as correcting errors which had become apparent in 2.0. Version 2.x is a significant improvement on earlier versions, defining a new underlying interaction model
Interaction Model
In the context of e-Learning, interactivity is defined as "function of input required by the learner while responding to the computer, the analysis of those responses by the computer, and the nature of the action by the computer." - Variables of Interactivity :...
. It is also notable for its significantly greater degree of integration with other specifications (some of which did not exist during the production of v1): the specification addresses the relationship with IMS Content Packaging
Content package
A content package is a file containing content and metadata.A content package is used in e-learning to define some learning content or an assessment that can be delivered, for example by a Learning Management System...
v1.2, IEEE Learning Object Metadata
Learning object metadata
Learning Object Metadata is a data model, usually encoded in XML, used to describe a learning object and similar digital resources used to support learning...
, IMS Learning Design
IMS Learning Design
IMS Learning Design is a specification for a metalanguage which enables the modelling of learning processes. The specification is maintained by IMS Global Learning Consortium.-Background:...
, IMS Simple Sequencing and other standards such as XHTML
XHTML
XHTML is a family of XML markup languages that mirror or extend versions of the widely-used Hypertext Markup Language , the language in which web pages are written....
. It also provides guidance on representing context-specific usage data and information to support the migration of content from earlier versions of the specification.
Because v2.0 was limited to items only, and v2.1 has yet to be formally released by IMS (although two public drafts plus an addendum are currently available), uptake of v2.x has been slow to date. The delay between the release of 2.0 and 2.1 (over three years to date) may have hindered uptake to some extent, with developers reluctant to commit to v2.0 knowing that v2.1 is in development. The use of a profile of v1.2.1 in the IMS Common Cartridge specification may exacerbate this. A number of implementations are emerging, however, and uptake may increase once the specification is finally available in a stable form.
In early 2009, the IMS Global Learning Consortium withdrew QTI 2.1, stating that "Adequate feedback on the specification has not been received, and therefore, the specification has been put back into the IMS project group process for further work." The most recent version of QTI that is fully endorsed by IMS GLC is v1.2.1. This decision met with disapproval on the IMS-QTI mailing list.
A further clarification on the QTI 2.1 withdrawal acknowledged the work done on implementing the QTI 2.1 draft specification, and cited criticism on the lack of interoperability of IMS specifications as a reason for endorsing only IMS QTI 1.2. A few weeks later IMS GLC reposted the QTI v2.1 draft specification on their website with a warning that the specification is incomplete:
Caution: The QTIv2.1PD Version 2 specification is incomplete in its current state. The IMS QTI project group is in the process of evolving this specification based on input from market participants. Suppliers of products and services are encouraged to participate by contacting Mark McKell at[e-mail address removed] . This specification will be superseded by an updated release based on the input of the project group participants.
Please note that supplier's claims as to implementation of QTI v2.1 and conformance to it HAVE NOT BEEN VALIDATED by IMS GLC. While such suppliers are likely well-intentioned, IMS GLC member organizations have not yet put in place the testing process to validate these claims. IMS GLC currently grants a conformance mark to the Common Cartridge profile of QTI v1.2.1.
Timeline
| Date | Version | Comments |
| March 1999 | 0.5 | Internal to IMS |
| February 2000 | 1.0 public draft | |
| May 2000 | 1.0 final release | |
| August 2000 | 1.01 | |
| March 2001 | 1.1 | |
| January 2002 | 1.2 | |
| March 2003 | 1.2.1 addendum | |
| September 2003 | 2.0 charter | Initiation of working group |
| January 2005 | 2.0 final release | |
| January 2006 | 2.1 public draft | |
| July 2006 | 2.1 public draft version 2 | |
| April 2008 | 2.1 public draft addendum | |
| early 2009 | 2.1 removed from website | |
| April 2009 | 2.1 reinstated on website | |
Applications with IMS QTI support
| Name | QTI version | Type of tool | Comment | |
|---|---|---|---|---|
| Alphastudy Alphastudy Alphastudy is a private Australian company that enables users to create their own web based learning and knowledge portals. The company's office is located in Sydney, Australia and was founded in 2008 by David Dinh.- History :The idea started in 2008... |
2.0 | LMS Learning management system A learning management system is a software application for the administration, documentation, tracking, and reporting of training programs, classroom and online events, e-learning programs, and training content... |
also supports IMS-QTI 2.0 standard, SCORM 1.2 and SCORM 2004 | |
| ANGEL Learning Management Suite | 2.1 | LMS Learning management system A learning management system is a software application for the administration, documentation, tracking, and reporting of training programs, classroom and online events, e-learning programs, and training content... |
also supports IMS Common Cartridge | |
| APIS QTIv2 Assessment Engine | 2.0 draft | Java library & demo application. | Incomplete. Author recommends using QTITools instead. | |
| AQuRate | 2.1 | authoring tool | see QTITools | |
| ASDEL | 2.1 | assessment delivery system | see QTITools | |
| Assessment Tiger | 2.1 | Authoring and mobile assessment delivery | Advanced QTI 2.1 authoring - mobile QTI 2.1 rendering | |
| ATutor ATutor ATutor is an Open Source Web-based Learning Content Management System .ATutor is used in various contexts, including online course management, continuing professional development for teachers, career development, and academic research... |
1.2, 2.1 | LCMS | ||
| Canvas Learning | 1.2.1 | Authoring tools and SCORM compatible item renderer available as middle-ware solutions. | Creators - Can Studios contributed to the development of the QTI specification. A number of LMS systems used the Canvas Learning Player to achieve compatibility with the Becta learning platform conformance regime. The system is currently being distributed to schools in the UK as a result of this integration work. | |
| CCReader | 1.2.1 CC Profile | Common Cartridge Viewer | ||
| Chamilo Chamilo Chamilo is an open-source e-learning and content management system, aimed at improving access to education and knowledge globally... |
1.2 and 2.0 | LMS/LCMS | export QTI 1.2 & 2.0, import QTI 2.0 (1.2 disabled by default but available) (supports SCORM 1.2) | |
| Cognero | 1.2 and 2.1 | Assessment authoring and delivery system. | Cognero imports QTI 1.2 and exports QTI 1.2 and 2.1 to allow content to work with other systems. | |
| Content-e | 1.2 & 2.0 | Professional authoring tool Content-e. | Imports QTI 1.2 and 2.0. | |
| DB Primary | 2.0 | LMS | ||
| Diploma | 1.2, 2.1 | export QTI 1.2 & 2.1 | ||
| Dokeos Dokeos Dokeos is a company dedicated to open source Learning Management Systems. Its main product is a SCORM-compliant open source learning suite used by multinational companies, federal administrations and universities.... |
1.2 and 2.0 | LMS/LCMS | export QTI 1.2 & 2.0 (1.2 disabled by default but available) (supports SCORM 1.2) | |
| e-teach Q-Player | 1.2 | QTI 1.2 player. SCORM 1.2 and 2004 compliant | QTI 1.2 universal player - Adobe flash based | |
| it's learning It's learning itslearning is an internet-based VLE that is made for all levels of education all around the world. The website incorporates features such as a help center, web tutorials, mobile login and many other features designed to make working collaboratively and presenting work easy.According to its own... |
2.1 | VLE Virtual learning environment Defined largely by usage, the term virtual learning environment has most, if not all, of the following salient properties:* It is Web-based* It uses Web 2.0 tools for rich 2-way interaction* It includes a content management system... LMS Learning management system A learning management system is a software application for the administration, documentation, tracking, and reporting of training programs, classroom and online events, e-learning programs, and training content... |
import and export questions in QTI 2.1 format | |
| ILIAS ILIAS ILIAS is an open source web-based learning management system . It supports learning content management and tools for collaboration, communication, evaluation and assessment... |
not stated | LMS | supports SCORM 1.2 and SCORM 2004 | |
| Lectora Lectora Lectora is an Electronic learning development tool, also known as authoring software, developed by Trivantis Corporation. Lectora is used to create online training courses, assessments, and presentations... |
not stated | authoring tool | supports AICC, SCORM 1.2 and SCORM 2004 | |
| Mathqurate | 2.1 | authoring tool | see QTITools. Embedded Gecko engine and support for multiple interactions | |
| Moodle Moodle Moodle is a free source e-learning software platform, also known as a Course Management System, Learning Management System, or Virtual Learning Environment... |
not stated | LCMS | supports adaptive questions; QTI 2.0 export is still unfinished | |
| Online Learning And Training OLAT OLAT is an acronym for Online Learning And Training. It is a web application – a so called Learning Management System that supports any kind of online learning, teaching, and tutoring with few educational restrictions. OLAT is free software and is open source. Its development started in 1999 at the... |
QTI 1.2 | LCMS | QTI 2.1 compliance can be achieved with ONYX as plugin | |
| Onyx Editor (Elques) and Onyx Konverter | 1.2 and 2.1 | authoring tool and konverter | exports QTI 2.1 and QTI 1.2 (for LMS OLAT only); imports QTI 2.1, Tests from Blackboard and OLAT (kind of QTI 1.2 too); free version available | |
| Onyx Player and Onyx Reporter | 2.1 | modular assessment delivery system | QTI 2.1 import and export (report files); open-source; Onyx Reporter: Report Viewer for graphical visualization of QTI-Result-Files | |
| OWL Testing Software | not stated | test management system | can import IMS QTI | |
| QTITools | 2.1 | collection of tools and libraries | Test authoring tool Spectatus procudes QTI 2.1 | |
| Question Bank | 1.1 | Exports and imports QTI 1.1 | ||
| QuestionMark Perception QuestionMark The Questionmark Corporation is a supplier of online assessment software to educational institutions, public sector employers and commercial companies worldwide. The company was started in 1988 by current chairman John Kleeman. Primary markets are Europe and North America... |
not stated | authoring tool and delivery system | can export IMS QTI, an online tool provides QTI 1.2 import | |
| Question Writer 2.0 Publisher Edition Question Writer Question Writer is a quiz authoring tool for Microsoft Windows. It is used for authoring questions and compiling them into quizzes in the Flash file format. The software is published by Central Question.- User interface :... |
1.2 | authoring tool | Exports as QTI 1.2 and SCORM 1.2 | |
| Question Writer 3.5 Professional Question Writer Question Writer is a quiz authoring tool for Microsoft Windows. It is used for authoring questions and compiling them into quizzes in the Flash file format. The software is published by Central Question.- User interface :... |
1.2 | authoring tool | Exports as QTI 1.2 and SCORM 1.2 Also specific QTI Export for Pearson VUE | |
| Respondus | 1.2 | authoring tool | QTI export | |
| RM Test Authoring System | 2.1 | authoring tool | ||
| Sakai Sakai Project This page is about the software project, for other meanings, see Sakai.Sakai is a community of academic institutions, commercial organizations and individuals who work together to develop a common Collaboration and Learning Environment... |
1.2 | LMS | ||
| SToMP (Software Teaching of Modular Physics) | 2.1 | assessment system | mostly unavailable as of July 2008 | |
| Studywiz | 1.2 | Virtual Learning Environment Module | An optional module for creating and assigning QTI v1.2 questions to students. Available as of June 2008 | |
| Wimba Create | QTI Lite | authoring tool | only export |
Other software:
- QTI Migration Tool (University of Cambridge): converts QTI version 1.x data into QTI 2.0 content packages.
External links
- IMS Global Learning Consortium: IMS Question & Test Interoperability Specification
- TOIA (Technologies for Online Interoperable Assessment) - this project ended in 2007 and software is no longer available.
- QTI Tools
- JISC CETIS Assessment special interest group
- JISC CETIS wiki: Assessment tools, projects and resources
- IMS Question & Test Interoperability mailing list