A test method is a definitive procedure that produces a test result.

A test can be considered as technical operation that consists of determination of one or more characteristics of a given product, process or service according to a specified procedure. Often a test is part of an experiment

Experiment

An experiment is a methodical procedure carried out with the goal of verifying, falsifying, or establishing the validity of a hypothesis. Experiments vary greatly in their goal and scale, but always rely on repeatable procedure and logical analysis of the results...

.

The test result can be qualitative (yes/no), categorical

Categorical

See:* Categorical imperative* Morley's categoricity theorem* Categorical data analysis* Categorical distribution* Categorical logic* Categorical syllogism* Categorical proposition* Categorization* Categorical perception* Category theory...

, or quantitative

Quantitative

A quantitative property is one that exists in a range of magnitudes, and can therefore be measured with a number. Measurements of any particular quantitative property are expressed as a specific quantity, referred to as a unit, multiplied by a number. Examples of physical quantities are distance,...

 (a measured value). It can be a personal observation

Observation

Observation is either an activity of a living being, such as a human, consisting of receiving knowledge of the outside world through the senses, or the recording of data using scientific instruments. The term may also refer to any data collected during this activity...

 or the output of a precision measuring instrument

Measuring instrument

In the physical sciences, quality assurance, and engineering, measurement is the activity of obtaining and comparing physical quantities of real-world objects and events. Established standard objects and events are used as units, and the process of measurement gives a number relating the item...

.

Usually the test result is the dependent variable, the measured response based on the particular conditions of the test or the level of the independent variable

Independent variable

The terms "dependent variable" and "independent variable" are used in similar but subtly different ways in mathematics and statistics as part of the standard terminology in those subjects...

. Some tests, however, involve changing the independent variable to determine the level at which a certain response occurs: in this case, the test result is the independent variable.

Importance of test methods

In software development

Software development

Software development is the development of a software product...

, engineering

Engineering

Engineering is the discipline, art, skill and profession of acquiring and applying scientific, mathematical, economic, social, and practical knowledge, in order to design and build structures, machines, devices, systems, materials and processes that safely realize improvements to the lives of...

, science

Science

Science is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe...

, manufacturing

Manufacturing

Manufacturing is the use of machines, tools and labor to produce goods for use or sale. The term may refer to a range of human activity, from handicraft to high tech, but is most commonly applied to industrial production, in which raw materials are transformed into finished goods on a large scale...

, and business

Business

A business is an organization engaged in the trade of goods, services, or both to consumers. Businesses are predominant in capitalist economies, where most of them are privately owned and administered to earn profit to increase the wealth of their owners. Businesses may also be not-for-profit...

, it is vital for all interested people to understand and agree upon methods of obtaining data and making measurement

Measurement

Measurement is the process or the result of determining the ratio of a physical quantity, such as a length, time, temperature etc., to a unit of measurement, such as the metre, second or degree Celsius...

s. It is common for a physical property

Physical property

A physical property is any property that is measurable whose value describes a physical system's state. The changes in the physical properties of a system can be used to describe its transformations ....

 to be strongly affected by the precise method of testing or measuring that property. It is vital to fully document experiments and measurements and to provide needed definitions to specifications and contract

Contract

A contract is an agreement entered into by two parties or more with the intention of creating a legal obligation, which may have elements in writing. Contracts can be made orally. The remedy for breach of contract can be "damages" or compensation of money. In equity, the remedy can be specific...

s.

Using a standard test method, perhaps published by a respected standards organization

Standards organization

A standards organization, standards body, standards developing organization , or standards setting organization is any organization whose primary activities are developing, coordinating, promulgating, revising, amending, reissuing, interpreting, or otherwise producing technical standards that are...

, is a good place to start. Sometimes it is more useful to modify an existing test method or to develop a new one. Again, documentation and full disclosure are very necessary.

A well-written test method is important. However, even more important is choosing a method of measuring the correct property or characteristic. Not all tests and measurements are equally useful: usually a test result is used to predict or imply suitability for a certain purpose. For example, if a manufactured item has several components, test methods may have several levels of connections:

• test results of a raw material should connect with tests of a component made from that material
• test results of a component should connect with performance testing
  Performance testing
  In software engineering, performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload...
  
   of a complete item
• results of laboratory performance testing should connect with field performance

These connections or correlation

Correlation

In statistics, dependence refers to any statistical relationship between two random variables or two sets of data. Correlation refers to any of a broad class of statistical relationships involving dependence....

s may be based on published literature, engineering studies, or formal programs such as quality function deployment

Quality function deployment

Quality function deployment is a “method to transform user demands into design quality, to deploy the functions forming quality, and to deploy methods for achieving the design quality into subsystems and component parts, and ultimately to specific elements of the manufacturing process.”, as...

. Validation

Verification and Validation

In software project management, software testing, and software engineering, verification and validation is the process of checking that a software system meets specifications and that it fulfills its intended purpose...

 of the suitability of the test method is often required.

Content of a test method

Quality management system

Quality management system

A quality management system can be expressed as the organizational structure, procedures, processes and resources needed to implement quality management.-Elements of a Quality Management System:# Organizational structure# Responsibilities# Methods...

s usually require full documentation of the procedures used in a test. The document for a test method might include:

• Descriptive title
• Scope over which class(es) of materials or articles may be evaluated
• Date of last effective revision and revision designation
• Reference to most recent test method validation
• Person, office, or agency responsible for questions on the test method, updates, and deviations.
• The significance or importance of the test method and its intended use.
• Terminology and definition
  Definition
  A definition is a passage that explains the meaning of a term , or a type of thing. The term to be defined is the definiendum. A term may have many different senses or meanings...
  
  s to clarify the meanings of the test method
• A listing of the types of apparatus and measuring instrument (sometimes the specific device) required to conduct the test
• Safety
  Safety
  Safety is the state of being "safe" , the condition of being protected against physical, social, spiritual, financial, political, emotional, occupational, psychological, educational or other types or consequences of failure, damage, error, accidents, harm or any other event which could be...
  
   precautions
• Required calibration
  Calibration
  Calibration is a comparison between measurements – one of known magnitude or correctness made or set with one device and another measurement made in as similar a way as possible with a second device....
  
  s and metrology
  Metrology
  Metrology is the science of measurement. Metrology includes all theoretical and practical aspects of measurement. The word comes from Greek μέτρον , "measure" + "λόγος" , amongst others meaning "speech, oration, discourse, quote, study, calculation, reason"...
  
   systems
• Environmental
  Natural environment
  The natural environment encompasses all living and non-living things occurring naturally on Earth or some region thereof. It is an environment that encompasses the interaction of all living species....
  
   concerns and considerations
• Sampling
  Sampling (statistics)
  In statistics and survey methodology, sampling is concerned with the selection of a subset of individuals from within a population to estimate characteristics of the whole population....
  
   procedures: How samples are to be obtained, and Number of samples (sample size
  Sample size
  Sample size determination is the act of choosing the number of observations to include in a statistical sample. The sample size is an important feature of any empirical study in which the goal is to make inferences about a population from a sample...
  
  ).
• Conditioning or required environmental chamber
  Environmental chamber
  An environmental chamber is an enclosure used to test the effects of specified environmental conditions on biological items, industrial products, materials, and electronic devices and components. Environmental chambers help evaluate product quality and reliability, and identify manufacturing flaws...
  
  : temperature
  Temperature
  Temperature is a physical property of matter that quantitatively expresses the common notions of hot and cold. Objects of low temperature are cold, while various degrees of higher temperatures are referred to as warm or hot...
  
  , humidity
  Humidity
  Humidity is a term for the amount of water vapor in the air, and can refer to any one of several measurements of humidity. Formally, humid air is not "moist air" but a mixture of water vapor and other constituents of air, and humidity is defined in terms of the water content of this mixture,...
  
  , etc., including tolerances
  Tolerance (engineering)
  Engineering tolerance is the permissible limit or limits of variation in# a physical dimension,# a measured value or physical property of a material, manufactured object, system, or service,# other measured values ....
  
  
• Preparation of samples for the test and test fixture
  Test fixture
  A test fixture is something used to consistently test some item, device, or piece of software.-Electronics:Circuit boards, electronic components, and chips are held in place and subjected to controlled electronic test signals. One example is a bed of nails tester.-Software:Test fixture refers to...
  
  s
• Detailed procedure for conducting the test
• Calculations and analysis of data
• Interpretation of data and test method output
• Report: format, content, data, etc.

Test Method Validation

Test methods are often scrutinized for their validity, applicability, and accuracy. It is very important that the scope of the test method be clearly defined, and any aspect included in the scope is shown to be accurate and repeatable through validation.

Test method validations often encompass the following considerations:

• Accuracy and precision
  Accuracy and precision
  In the fields of science, engineering, industry and statistics, the accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual value. The precision of a measurement system, also called reproducibility or repeatability, is the degree to which...
  
  : Demonstration of accuracy may require the creation of a reference value if none is yet available.
• Repeatability
  Repeatability
  Repeatability or test-retest reliability is the variation in measurements if they would have been taken by a single person or instrument on the same item and under the same conditions. A less-than-perfect test-retest reliability causes test-retest variability. Such variability can be caused by, for...
  
   and Reproducibility
  Reproducibility
  Reproducibility is the ability of an experiment or study to be accurately reproduced, or replicated, by someone else working independently...
  
  , sometimes in the form of a Gauge R&R.
• Range, or a continuum scale over which the test method would be considered accurate. Example: 10 N to 100 N force test.
• Measurement resolution
  Image resolution
  Image resolution is an umbrella term that describes the detail an image holds. The term applies to raster digital images, film images, and other types of images. Higher resolution means more image detail....
  
  , be it spatial, temporal, or otherwise.
• Curve fitting
  Curve fitting
  Curve fitting is the process of constructing a curve, or mathematical function, that has the best fit to a series of data points, possibly subject to constraints. Curve fitting can involve either interpolation, where an exact fit to the data is required, or smoothing, in which a "smooth" function...
  
  , typically for linearity, which justifies interpolation
  Interpolation
  In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points....
  
   between calibrated reference points.
• Robustness, or the insensitivity to potentially subtle variables in the test environment or setup which may be difficult to control.
• Usefulness to predict end-use characteristics and performance
• Measurement uncertainty
  Measurement uncertainty
  In metrology, measurement uncertainty is a non-negative parameter characterizing the dispersion of the values attributed to a measured quantity. The uncertainty has a probabilistic basis and reflects incomplete knowledge of the quantity. All measurements are subject to uncertainty and a measured...
  
  
• Interlaboratory or round robin test
  Round robin test
  In experimental methodology, a round robin test is an interlaboratory test performed independently several times. This can involve multiple independent scientists performing the test with the use of the same method in different equipment, or a variety of methods and equipment...
  
  s
• other types of measurement systems analysis
  Measurement Systems Analysis
  A Measurement System Analysis is a specially designed experiment that seeks to identify the components of variation in the measurement....
  
  

See also

• Certified reference materials
  Certified reference materials
  Certified Reference Materials are ‘controls’ or standards used to check the quality and traceability of products. A reference standard for a unit of measurement is an artifact that embodies the quantity of interest in a way that ties its value to the reference base for calibration.At the highest...
  
  
• Data analysis
  Data analysis
  Analysis of data is a process of inspecting, cleaning, transforming, and modeling data with the goal of highlighting useful information, suggesting conclusions, and supporting decision making...
  
  
• Design of experiments
  Design of experiments
  In general usage, design of experiments or experimental design is the design of any information-gathering exercises where variation is present, whether under the full control of the experimenter or not. However, in statistics, these terms are usually used for controlled experiments...
  
  
• Document management system
  Document management system
  A document management system is a computer system used to track and store electronic documents and/or images of paper documents. It is usually also capable of keeping track of the different versions created by different users . The term has some overlap with the concepts of content management...
  
  
• EPA Methods
  EPA Methods
  The United States Environmental Protection Agency maintains test methods, which are approved procedures for measuring the presence and concentration of physical and chemical pollutants; evaluating properties, such as toxic properties, of chemical substances; or measuring the effects of substances...
  
  
• Integrated test facility
  Integrated test facility
  An integrated test facility creates a fictitious entity in a database to process test transactions simultaneously with live input. It can be used to incorporate test transactions into a normal production run of a system. Its advantage is that periodic testing does not require separate test processes...
  
  
• Measurement Systems Analysis
  Measurement Systems Analysis
  A Measurement System Analysis is a specially designed experiment that seeks to identify the components of variation in the measurement....
  
  
• Measurement uncertainty
  Measurement uncertainty
  In metrology, measurement uncertainty is a non-negative parameter characterizing the dispersion of the values attributed to a measured quantity. The uncertainty has a probabilistic basis and reflects incomplete knowledge of the quantity. All measurements are subject to uncertainty and a measured...
  
  
• Metrication
  Metrication
  Metrication refers to the introduction and use of the SI metric system, the international standard for physical measurements. This has involved a long process of independent and systematic conversions of countries from various local systems of weights and measures. Metrication began in France in...
  
  
• Observational error
  Observational error
  Observational error is the difference between a measured value of quantity and its true value. In statistics, an error is not a "mistake". Variability is an inherent part of things being measured and of the measurement process.-Science and experiments:...
  
  
• Replication (statistics)
  Replication (statistics)
  In engineering, science, and statistics, replication is the repetition of an experimental condition so that the variability associated with the phenomenon can be estimated. ASTM, in standard E1847, defines replication as "the repetition of the set of all the treatment combinations to be compared in...
  
  
• Sampling (statistics)
  Sampling (statistics)
  In statistics and survey methodology, sampling is concerned with the selection of a subset of individuals from within a population to estimate characteristics of the whole population....
  
  
• Specification (technical standard)
  Specification (technical standard)
  A specification is an explicit set of requirements to be satisfied by a material, product, or service. Should a material, product or service fail to meet one or more of the applicable specifications, it may be referred to as being out of specification;the abbreviation OOS may also be used...
  
  
• Test Management Approach
  Test Management Approach
  Test management approach is a software testing methodology. TMap is a method which combines insights on how to test and what to manage, as well as techniques for the individual test consultant.- History :...
  
  
• Verification and validation
  Verification and Validation
  In software project management, software testing, and software engineering, verification and validation is the process of checking that a software system meets specifications and that it fulfills its intended purpose...
  
  

General references, Books

• Pyzdek, T, "Quality Engineering Handbook", 2003, ISBN 0824746147
• Godfrey, A. B., "Juran's Quality Handbook", 1999, ISBN 007034003
• Kimothi, S. K., "The Uncertainty of Measurements: Physical and Chemical Metrology: Impact and Analysis", 2002, ISBN 0873895355

Standards

• ASTM D4356 Standard Practice for Establishing Consistent Test Method Tolerances
• ASTM E691 Standard Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
• ASTM E1169 Standard Guide for Conducting Ruggedness Tests
• ASTM E1488 Standard Guide for Statistical Procedures to Use in Developing and Applying Test Methods
• ASTM E2655 - Standard Guide for Reporting Uncertainty of Test Results and Use of the Term Measurement Uncertainty in ASTM Test Methods