Data-driven testing
Encyclopedia
Data-driven testing is a term used in the testing of computer
software to describe testing done using a table of conditions directly as test inputs and verifiable outputs as well as the process where test environment settings and control are not hard-coded. In the simplest form the tester supplies the inputs from a row in the table and expects the outputs which occur in the same row. The table typically contains values which correspond to boundary or partition input spaces. In the control methodology, test configuration is "read" from a database.
or programs
, several methodologies are available for implementing this testing. Each of these methods co-exist because they differ in the effort required to create and subsequently maintain. The advantage of Data-driven testing is the ease to add additional inputs to the table when new partitions are discovered or added to the product or System Under Test
. The cost aspect makes DDT cheap for automation but expensive for manual testing. One could confuse DDT with Table-driven testing, which this article needs to separate more clearly in future.
The data comprises variables used for both input values and output verification values. In advanced (mature) automation environments data can be harvested from a running system using a purpose-built custom tool or sniffer, the DDT framework thus performs playback of harvested data producing a powerful automated regression testing tool.
Navigation through the program
, reading of the data sources, and logging of test status and information are all coded in the test script.
The databases used for data-driven testing can include:-
Computer
A computer is a programmable machine designed to sequentially and automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem...
software to describe testing done using a table of conditions directly as test inputs and verifiable outputs as well as the process where test environment settings and control are not hard-coded. In the simplest form the tester supplies the inputs from a row in the table and expects the outputs which occur in the same row. The table typically contains values which correspond to boundary or partition input spaces. In the control methodology, test configuration is "read" from a database.
Introduction
In the testing of softwareComputer software
Computer software, or just software, is a collection of computer programs and related data that provide the instructions for telling a computer what to do and how to do it....
or programs
Computer program
A computer program is a sequence of instructions written to perform a specified task with a computer. A computer requires programs to function, typically executing the program's instructions in a central processor. The program has an executable form that the computer can use directly to execute...
, several methodologies are available for implementing this testing. Each of these methods co-exist because they differ in the effort required to create and subsequently maintain. The advantage of Data-driven testing is the ease to add additional inputs to the table when new partitions are discovered or added to the product or System Under Test
System under test
System under test refers to a system that is being tested for correct operation. The term is used mostly in software testing.A special case of a software system is an application which, when tested, is called an application under test....
. The cost aspect makes DDT cheap for automation but expensive for manual testing. One could confuse DDT with Table-driven testing, which this article needs to separate more clearly in future.
Methodology Overview
- Data-driven testing is the creation of test scripts to run together with their related data sets in a framework. The framework provides re-usable test logic to reduce maintenance and improve test coverage. Input and result (test criteria) data values can be stored in one or more central data sources or databaseDatabaseA database is an organized collection of data for one or more purposes, usually in digital form. The data are typically organized to model relevant aspects of reality , in a way that supports processes requiring this information...
s, the actual format and organisation can be implementation specific.
The data comprises variables used for both input values and output verification values. In advanced (mature) automation environments data can be harvested from a running system using a purpose-built custom tool or sniffer, the DDT framework thus performs playback of harvested data producing a powerful automated regression testing tool.
Navigation through the program
System under test
System under test refers to a system that is being tested for correct operation. The term is used mostly in software testing.A special case of a software system is an application which, when tested, is called an application under test....
, reading of the data sources, and logging of test status and information are all coded in the test script.
Data Driven
Anything that has a potential to change (also called "Variability" and includes such as environment, end points, test data and locations, etc), is separated out from the test logic (scripts) and moved into an 'external asset'. This can be a configuration or test dataset. The logic executed in the script is dictated by the data values.- Keyword-driven testingKeyword-driven testingKeyword-driven testing, also known as table-driven testing or action-word testing, is a software testing methodology for automated testing that separates the test creation process into two distinct stages: a Planning Stage, and an Implementation Stage.-Overview:Although keyword testing can be used...
is similar except that the test case is contained in the set of data values and not embedded or "hard-coded" in the test script itself. The script is simply a "driver" (or delivery mechanism) for the data that is held in the data source
The databases used for data-driven testing can include:-
- datapools
- ODBCOpen Database ConnectivityIn computing, ODBC is a standard C interface for accessing database management systems . The designers of ODBC aimed to make it independent of database systems and operating systems...
source's - csv filesComma-separated valuesA comma-separated values file stores tabular data in plain-text form. As a result, such a file is easily human-readable ....
- Excel files
- DAOData Access ObjectIn computer software, a data access object is an object that provides an abstract interface to some type of database or persistence mechanism, providing some specific operations without exposing details of the database. It provides a mapping from application calls to the persistence layer...
objects - ADOActiveX Data ObjectsMicrosoft's ActiveX Data Objects is a set of Component Object Model objects for accessing data sources. A part of MDAC, it provides a middleware layer between programming languages and OLE DB...
objects, etc.
See also
- Control tableControl tableControl tables are tables that control the program flow or play a major part in program control. There are no rigid rules concerning the structure or content of a control table - its only qualifying attribute is its ability to direct program flow in some way through its 'execution' by an associated...
- Keyword-driven testingKeyword-driven testingKeyword-driven testing, also known as table-driven testing or action-word testing, is a software testing methodology for automated testing that separates the test creation process into two distinct stages: a Planning Stage, and an Implementation Stage.-Overview:Although keyword testing can be used...
- Test Automation FrameworkTest automation frameworkA test automation framework is a set of assumptions, concepts and tools that provide support for automated software testing. The main advantage of such a framework is the low cost for maintenance. If there is change to any test case then only the test case file needs to be updated and the Driver...
- Test-Driven DevelopmentTest-driven developmentTest-driven development is a software development process that relies on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new...
- Hybrid Automation Framework
- Meta Data Driven Testing
- Modularity-driven testing
- Hybrid testing
- Model-based testing