Mathematical model
Encyclopedia
A mathematical model is a description of a system
using mathematical
concepts and language. The process of developing a mathematical model is termed mathematical modeling. Mathematical models are used not only in the natural science
s (such as physics
, biology
, earth science
, meteorology
) and engineering
disciplines (e.g. computer science
, artificial intelligence
), but also in the social sciences
(such as economics
, psychology
, sociology
and political science
); physicist
s, engineer
s, statistician
s, operations research
analysts and economist
s use mathematical models most extensively.
Mathematical models can take many forms, including but not limited to dynamical systems, statistical model
s, differential equations, or game theoretic models
. These and other types of models can overlap, with a given model involving a variety of abstract structures. In general, mathematical models may include logical model
s, as far as logic is taken as a part of mathematics. In many cases, the quality of a scientific field depends on how well the mathematical models developed on the theoretical side agree with results of repeatable experiments. Lack of agreement between theoretical mathematical models and experimental measurements often leads to important advances as better theories are developed.
Modeling requires selecting and identifying relevant aspects of a situation in the real world.
s.
A mathematical model usually describes a system by a set of variables and a set of equations that establish relationships between the variables.
The values of the variables can be practically anything; real
or integer
numbers, boolean values or strings, for example.
The variables represent some properties of the system, for example, measured system outputs often in the form of signals, timing data, counters, and event occurrence (yes/no).
The actual model is the set of functions that describe the relations between the different variables.
Decision variables are sometimes known as independent variables. Exogenous variables are sometimes known as parameters or constants.
The variables are not independent of each other as the state variables are dependent on the decision, input, random, and exogenous variables. Furthermore, the output variables are dependent on the state of the system (represented by the state variables).
Objectives and constraints of the system and its users can be represented as functions of the output variables or state variables. The objective functions will depend on the perspective of the model's user. Depending on the context, an objective function is also known as an index of performance, as it is some measure of interest to the user. Although there is no limit to the number of objective functions and constraints a model can have, using or optimizing the model becomes more involved (computationally) as the number increases.
or white box
models, according to how much a priori information is available of the system. A black-box model is a system of which there is no a priori information available. A white-box model (also called glass box or clear box) is a system where all necessary information is available. Practically all systems are somewhere between the black-box and white-box models, so this concept is useful only as an intuitive guide for deciding which approach to take.
Usually it is preferable to use as much a priori information as possible to make the model more accurate. Therefore the white-box models are usually considered easier, because if you have used the information correctly, then the model will behave correctly. Often the a priori information comes in forms of knowing the type of functions relating different variables. For example, if we make a model of how a medicine works in a human system, we know that usually the amount of medicine in the blood is an exponentially decaying function. But we are still left with several unknown parameters; how rapidly does the medicine amount decay, and what is the initial amount of medicine in blood? This example is therefore not a completely white-box model. These parameters have to be estimated through some means before one can use the model.
In black-box models one tries to estimate both the functional form of relations between variables and the numerical parameters in those functions. Using a priori information we could end up, for example, with a set of functions that probably could describe the system adequately. If there is no a priori information we would try to use functions as general as possible to cover all different models. An often used approach for black-box models are neural networks
which usually do not make assumptions about incoming data. The problem with using a large set of functions to describe a system is that estimating the parameters becomes increasingly difficult when the amount of parameters (and different types of functions) increases.
, experience
, or expert opinion, or based on convenience of mathematical form. Bayesian statistics
provides a theoretical framework for incorporating such subjectivity into a rigorous analysis: one specifies a prior probability distribution (which can be subjective) and then updates this distribution based on empirical data. An example of when such approach would be necessary is a situation in which an experimenter bends a coin slightly and tosses it once, recording whether it comes up heads, and is then given the task of predicting the probability that the next flip comes up heads. After bending the coin, the true probability that the coin will come up heads is unknown, so the experimenter would need to make an arbitrary decision (perhaps by looking at the shape of the coin) about what prior distribution to use. Incorporation of the subjective information is necessary in this case to get an accurate prediction of the probability, since otherwise one would guess 1 or 0 as the probability of the next flip being heads, which would be almost certainly wrong.
is a principle particularly relevant to modeling; the essential idea being that among models with roughly equal predictive power, the simplest one is the most desirable. While added complexity usually improves the realism of a model, it can make the model difficult to understand and analyze, and can also pose computational problems, including numerical instability. Thomas Kuhn
argues that as science progresses, explanations tend to become more complex before a Paradigm shift
offers radical simplification.
For example, when modeling the flight of an aircraft, we could embed each mechanical part of the aircraft into our model and would thus acquire an almost white-box model of the system. However, the computational cost of adding such a huge amount of detail would effectively inhibit the usage of such a model. Additionally, the uncertainty would increase due to an overly complex system, because each separate part induces some amount of variance into the model. It is therefore usually appropriate to make some approximations to reduce the model to a sensible size. Engineers often can accept some approximations in order to get a more robust and simple model. For example Newton's
classical mechanics
is an approximated model of the real world. Still, Newton's model is quite sufficient for most ordinary-life situations, that is, as long as particle speeds are well below the speed of light
, and we study macro-particles only.
s that can be used to fit the model to the system it is intended to describe. If the modeling is done by a neural network
, the optimization of parameters is called training. In more conventional modeling through explicitly given mathematical functions, parameters are determined by curve fitting
.
Defining a metric
to measure distances between observed and predicted data is a useful tool of assessing model fit. In statistics, decision theory, and some economic models, a loss function
plays a similar role.
While it is rather straightforward to test the appropriateness of parameters, it can be more difficult to test the validity of the general mathematical form of a model. In general, more mathematical tools have been developed to test the fit of statistical model
s than models involving differential equations. Tools from non-parametric statistics
can sometimes be used to evaluate how well the data fit a known distribution or to come up with a general model that makes only minimal assumptions about the model's mathematical form.
The question of whether the model describes well the properties of the system between data points is called interpolation
, and the same question for events or data points outside the observed data is called extrapolation
.
As an example of the typical limitations of the scope of a model, in evaluating Newtonian classical mechanics
, we can note that Newton made his measurements without advanced equipment, so he could not measure properties of particles travelling at speeds close to the speed of light. Likewise, he did not measure the movements of molecules and other small particles, but macro particles only. It is then not surprising that his model does not extrapolate well into these domains, even though his model is quite sufficient for ordinary life physics.
. This is usually (but not always) true of models involving differential equations. As the purpose of modeling is to increase our understanding of the world, the validity of a model rests not only on its fit to empirical observations, but also on its ability to extrapolate to situations or data beyond those originally described in the model. One can argue that a model is worthless unless it provides some insight which goes beyond what is already known from direct investigation of the phenomenon being studied.
An example of such criticism is the argument that the mathematical models of Optimal foraging theory
do not offer insight that goes beyond the common-sense conclusions of evolution
and other basic principles of ecology.
Specific applications
Philosophical background
System
System is a set of interacting or interdependent components forming an integrated whole....
using mathematical
Mathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
concepts and language. The process of developing a mathematical model is termed mathematical modeling. Mathematical models are used not only in the natural science
Natural science
The natural sciences are branches of science that seek to elucidate the rules that govern the natural world by using empirical and scientific methods...
s (such as physics
Physics
Physics is a natural science that involves the study of matter and its motion through spacetime, along with related concepts such as energy and force. More broadly, it is the general analysis of nature, conducted in order to understand how the universe behaves.Physics is one of the oldest academic...
, biology
Biology
Biology is a natural science concerned with the study of life and living organisms, including their structure, function, growth, origin, evolution, distribution, and taxonomy. Biology is a vast subject containing many subdivisions, topics, and disciplines...
, earth science
Earth science
Earth science is an all-embracing term for the sciences related to the planet Earth. It is arguably a special case in planetary science, the Earth being the only known life-bearing planet. There are both reductionist and holistic approaches to Earth sciences...
, meteorology
Meteorology
Meteorology is the interdisciplinary scientific study of the atmosphere. Studies in the field stretch back millennia, though significant progress in meteorology did not occur until the 18th century. The 19th century saw breakthroughs occur after observing networks developed across several countries...
) and engineering
Engineering
Engineering is the discipline, art, skill and profession of acquiring and applying scientific, mathematical, economic, social, and practical knowledge, in order to design and build structures, machines, devices, systems, materials and processes that safely realize improvements to the lives of...
disciplines (e.g. computer science
Computer science
Computer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...
, artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
), but also in the social sciences
Social sciences
Social science is the field of study concerned with society. "Social science" is commonly used as an umbrella term to refer to a plurality of fields outside of the natural sciences usually exclusive of the administrative or managerial sciences...
(such as economics
Economics
Economics is the social science that analyzes the production, distribution, and consumption of goods and services. The term economics comes from the Ancient Greek from + , hence "rules of the house"...
, psychology
Psychology
Psychology is the study of the mind and behavior. Its immediate goal is to understand individuals and groups by both establishing general principles and researching specific cases. For many, the ultimate goal of psychology is to benefit society...
, sociology
Sociology
Sociology is the study of society. It is a social science—a term with which it is sometimes synonymous—which uses various methods of empirical investigation and critical analysis to develop a body of knowledge about human social activity...
and political science
Political science
Political Science is a social science discipline concerned with the study of the state, government and politics. Aristotle defined it as the study of the state. It deals extensively with the theory and practice of politics, and the analysis of political systems and political behavior...
); physicist
Physicist
A physicist is a scientist who studies or practices physics. Physicists study a wide range of physical phenomena in many branches of physics spanning all length scales: from sub-atomic particles of which all ordinary matter is made to the behavior of the material Universe as a whole...
s, engineer
Engineer
An engineer is a professional practitioner of engineering, concerned with applying scientific knowledge, mathematics and ingenuity to develop solutions for technical problems. Engineers design materials, structures, machines and systems while considering the limitations imposed by practicality,...
s, statistician
Statistician
A statistician is someone who works with theoretical or applied statistics. The profession exists in both the private and public sectors. The core of that work is to measure, interpret, and describe the world and human activity patterns within it...
s, operations research
Operations research
Operations research is an interdisciplinary mathematical science that focuses on the effective use of technology by organizations...
analysts and economist
Economist
An economist is a professional in the social science discipline of economics. The individual may also study, develop, and apply theories and concepts from economics and write about economic policy...
s use mathematical models most extensively.
Mathematical models can take many forms, including but not limited to dynamical systems, statistical model
Statistical model
A statistical model is a formalization of relationships between variables in the form of mathematical equations. A statistical model describes how one or more random variables are related to one or more random variables. The model is statistical as the variables are not deterministically but...
s, differential equations, or game theoretic models
Game theory
Game theory is a mathematical method for analyzing calculated circumstances, such as in games, where a person’s success is based upon the choices of others...
. These and other types of models can overlap, with a given model involving a variety of abstract structures. In general, mathematical models may include logical model
Logical model
Logical model can refer to:* A model in logic, see model theory* In computer science a logical data model...
s, as far as logic is taken as a part of mathematics. In many cases, the quality of a scientific field depends on how well the mathematical models developed on the theoretical side agree with results of repeatable experiments. Lack of agreement between theoretical mathematical models and experimental measurements often leads to important advances as better theories are developed.
Examples of mathematical models
- PopulationPopulationA population is all the organisms that both belong to the same group or species and live in the same geographical area. The area that is used to define a sexual population is such that inter-breeding is possible between any pair within the area and more probable than cross-breeding with individuals...
Growth. A simple (though approximate) model of population growth is the Malthusian growth modelMalthusian growth modelThe Malthusian growth model, sometimes called the simple exponential growth model, is essentially exponential growth based on a constant rate of compound interest...
. A slightly more realistic and largely used population growth model is the logistic functionLogistic functionA logistic function or logistic curve is a common sigmoid curve, given its name in 1844 or 1845 by Pierre François Verhulst who studied it in relation to population growth. It can model the "S-shaped" curve of growth of some population P...
, and its extensions. - Model of a particle in a potential-field. In this model we consider a particle as being a point of mass which describes a trajectory in space which is modeled by a function giving its coordinates in space as a function of time. The potential field is given by a function V : R3 → R and the trajectory is a solution of the differential equationDifferential equationA differential equation is a mathematical equation for an unknown function of one or several variables that relates the values of the function itself and its derivatives of various orders...
-
- Note this model assumes the particle is a point mass, which is certainly known to be false in many cases in which we use this model; for example, as a model of planetary motion.
- Model of rational behavior for a consumer. In this model we assume a consumer faces a choice of n commodities labeled 1,2,...,n each with a market price p1, p2,..., pn. The consumer is assumed to have a cardinal utility function U (cardinal in the sense that it assigns numerical values to utilities), depending on the amounts of commodities x1, x2,..., xn consumed. The model further assumes that the consumer has a budget M which is used to purchase a vector x1, x2,..., xn in such a way as to maximize U(x1, x2,..., xn). The problem of rational behavior in this model then becomes an optimizationOptimization (mathematics)In mathematics, computational science, or management science, mathematical optimization refers to the selection of a best element from some set of available alternatives....
problem, that is:
-
-
- subject to:
-
-
-
- This model has been used in general equilibrium theory, particularly to show existence and Pareto efficiencyPareto efficiencyPareto efficiency, or Pareto optimality, is a concept in economics with applications in engineering and social sciences. The term is named after Vilfredo Pareto, an Italian economist who used the concept in his studies of economic efficiency and income distribution.Given an initial allocation of...
of economic equilibria. However, the fact that this particular formulation assigns numerical values to levels of satisfaction is the source of criticism (and even ridicule). However, it is not an essential ingredient of the theory and again this is an idealization.- Neighbour-sensing modelNeighbour-sensing modelThe neighbour-sensing model is the proposed hypothesis of the fungal morphogenesis. The hypothesis suggests that each hypha in the fungal mycelium generates a certain abstract field that decreases when increasing the distance. The proposed mathematical models deal with both scalar and vector fields...
explains the mushroomMushroomA mushroom is the fleshy, spore-bearing fruiting body of a fungus, typically produced above ground on soil or on its food source. The standard for the name "mushroom" is the cultivated white button mushroom, Agaricus bisporus; hence the word "mushroom" is most often applied to those fungi that...
formation from the initially chaotic fungalFungusA fungus is a member of a large group of eukaryotic organisms that includes microorganisms such as yeasts and molds , as well as the more familiar mushrooms. These organisms are classified as a kingdom, Fungi, which is separate from plants, animals, and bacteria...
network. - Computer ScienceComputer scienceComputer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...
: models in Computer Networks, data models, surface model,... - MechanicsMechanicsMechanics is the branch of physics concerned with the behavior of physical bodies when subjected to forces or displacements, and the subsequent effects of the bodies on their environment....
: movement of rocket model,...
- Neighbour-sensing model
Modeling requires selecting and identifying relevant aspects of a situation in the real world.
Background
Often when engineers analyze a system to be controlled or optimized, they use a mathematical model. In analysis, engineers can build a descriptive model of the system as a hypothesis of how the system could work, or try to estimate how an unforeseeable event could affect the system. Similarly, in control of a system, engineers can try out different control approaches in simulationSimulation
Simulation is the imitation of some real thing available, state of affairs, or process. The act of simulating something generally entails representing certain key characteristics or behaviours of a selected physical or abstract system....
s.
A mathematical model usually describes a system by a set of variables and a set of equations that establish relationships between the variables.
The values of the variables can be practically anything; real
Real number
In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...
or integer
Integer
The integers are formed by the natural numbers together with the negatives of the non-zero natural numbers .They are known as Positive and Negative Integers respectively...
numbers, boolean values or strings, for example.
The variables represent some properties of the system, for example, measured system outputs often in the form of signals, timing data, counters, and event occurrence (yes/no).
The actual model is the set of functions that describe the relations between the different variables.
Building blocks
There are six basic groups of variables namely: decision variables, input variables, state variables, exogenous variables, random variables, and output variables. Since there can be many variables of each type, the variables are generally represented by vectors.Decision variables are sometimes known as independent variables. Exogenous variables are sometimes known as parameters or constants.
The variables are not independent of each other as the state variables are dependent on the decision, input, random, and exogenous variables. Furthermore, the output variables are dependent on the state of the system (represented by the state variables).
Objectives and constraints of the system and its users can be represented as functions of the output variables or state variables. The objective functions will depend on the perspective of the model's user. Depending on the context, an objective function is also known as an index of performance, as it is some measure of interest to the user. Although there is no limit to the number of objective functions and constraints a model can have, using or optimizing the model becomes more involved (computationally) as the number increases.
Classifying mathematical models
Many mathematical models can be classified in some of the following ways:- Linear vs. nonlinear: Mathematical models are usually composed by variableVariable (mathematics)In mathematics, a variable is a value that may change within the scope of a given problem or set of operations. In contrast, a constant is a value that remains unchanged, though often unknown or undetermined. The concepts of constants and variables are fundamental to many areas of mathematics and...
s, which are abstractions of quantities of interest in the described systems, and operators that act on these variables, which can be algebraic operators, functions, differential operators, etc. If all the operators in a mathematical model exhibit linearLinearIn mathematics, a linear map or function f is a function which satisfies the following two properties:* Additivity : f = f + f...
ity, the resulting mathematical model is defined as linear. A model is considered to be nonlinear otherwise.
The question of linearity and nonlinearity is dependent on context, and linear models may have nonlinear expressions in them. For example, in a statistical linear modelLinear modelIn statistics, the term linear model is used in different ways according to the context. The most common occurrence is in connection with regression models and the term is often taken as synonymous with linear regression model. However the term is also used in time series analysis with a different...
, it is assumed that a relationship is linear in the parameters, but it may be nonlinear in the predictor variables. Similarly, a differential equation is said to be linear if it can be written with linear differential operatorDifferential operatorIn mathematics, a differential operator is an operator defined as a function of the differentiation operator. It is helpful, as a matter of notation first, to consider differentiation as an abstract operation, accepting a function and returning another .This article considers only linear operators,...
s, but it can still have nonlinear expressions in it. In a mathematical programmingOptimization (mathematics)In mathematics, computational science, or management science, mathematical optimization refers to the selection of a best element from some set of available alternatives....
model, if the objective functions and constraints are represented entirely by linear equationLinear equationA linear equation is an algebraic equation in which each term is either a constant or the product of a constant and a single variable....
s, then the model is regarded as a linear model. If one or more of the objective functions or constraints are represented with a nonlinearNonlinearityIn mathematics, a nonlinear system is one that does not satisfy the superposition principle, or one whose output is not directly proportional to its input; a linear system fulfills these conditions. In other words, a nonlinear system is any problem where the variable to be solved for cannot be...
equation, then the model is known as a nonlinear model.
Nonlinearity, even in fairly simple systems, is often associated with phenomena such as chaosChaos theoryChaos theory is a field of study in mathematics, with applications in several disciplines including physics, economics, biology, and philosophy. Chaos theory studies the behavior of dynamical systems that are highly sensitive to initial conditions, an effect which is popularly referred to as the...
and irreversibilityIrreversibilityIn science, a process that is not reversible is called irreversible. This concept arises most frequently in thermodynamics, as applied to processes....
. Although there are exceptions, nonlinear systems and models tend to be more difficult to study than linear ones. A common approach to nonlinear problems is linearizationLinearizationIn mathematics and its applications, linearization refers to finding the linear approximation to a function at a given point. In the study of dynamical systems, linearization is a method for assessing the local stability of an equilibrium point of a system of nonlinear differential equations or...
, but this can be problematic if one is trying to study aspects such as irreversibility, which are strongly tied to nonlinearity. - Deterministic vs. probabilistic (stochastic): A deterministic model is one in which every set of variable states is uniquely determined by parameters in the model and by sets of previous states of these variables. Therefore, deterministic models perform the same way for a given set of initial conditions. Conversely, in a stochasticStochastic processIn probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
model, randomness is present, and variable states are not described by unique values, but rather by probability distributions. - Static vs. dynamic: A static model does not account for the element of time, while a dynamic model does. Dynamic models typically are represented with difference equations or differential equations.
- Discrete vs. Continuous: A discrete model does not take into account the function of time and usually uses time-advance methods, while a Continuous model does. Continuous models typically are represented with f(t) and the changes are reflected over continuous time intervals.
- Deductive, inductive, or floating: A deductive model is a logical structure based on a theory. An inductive model arises from empirical findings and generalization from them. The floating model rests on neither theory nor observation, but is merely the invocation of expected structure. Application of mathematics in social sciences outside of economics has been criticized for unfounded models. Application of catastrophe theoryCatastrophe theoryIn mathematics, catastrophe theory is a branch of bifurcation theory in the study of dynamical systems; it is also a particular special case of more general singularity theory in geometry....
in science has been characterized as a floating model.
A priori information
Mathematical modeling problems are used often classified into black boxBlack box
A black box is a device, object, or system whose inner workings are unknown; only the input, transfer, and output are known characteristics.The term black box can also refer to:-In science and technology:*Black box theory, a philosophical theory...
or white box
White box (software engineering)
In software engineering white box, in contrast to a black box, is a subsystem whose internals can be viewed, but usually cannot be altered. This is useful during white box testing, where a system is examined to make sure that it fulfills its requirements....
models, according to how much a priori information is available of the system. A black-box model is a system of which there is no a priori information available. A white-box model (also called glass box or clear box) is a system where all necessary information is available. Practically all systems are somewhere between the black-box and white-box models, so this concept is useful only as an intuitive guide for deciding which approach to take.
Usually it is preferable to use as much a priori information as possible to make the model more accurate. Therefore the white-box models are usually considered easier, because if you have used the information correctly, then the model will behave correctly. Often the a priori information comes in forms of knowing the type of functions relating different variables. For example, if we make a model of how a medicine works in a human system, we know that usually the amount of medicine in the blood is an exponentially decaying function. But we are still left with several unknown parameters; how rapidly does the medicine amount decay, and what is the initial amount of medicine in blood? This example is therefore not a completely white-box model. These parameters have to be estimated through some means before one can use the model.
In black-box models one tries to estimate both the functional form of relations between variables and the numerical parameters in those functions. Using a priori information we could end up, for example, with a set of functions that probably could describe the system adequately. If there is no a priori information we would try to use functions as general as possible to cover all different models. An often used approach for black-box models are neural networks
Neural Networks
Neural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
which usually do not make assumptions about incoming data. The problem with using a large set of functions to describe a system is that estimating the parameters becomes increasingly difficult when the amount of parameters (and different types of functions) increases.
Subjective information
Sometimes it is useful to incorporate subjective information into a mathematical model. This can be done based on intuitionIntuition (knowledge)
Intuition is the ability to acquire knowledge without inference or the use of reason. "The word 'intuition' comes from the Latin word 'intueri', which is often roughly translated as meaning 'to look inside'’ or 'to contemplate'." Intuition provides us with beliefs that we cannot necessarily justify...
, experience
Experience
Experience as a general concept comprises knowledge of or skill in or observation of some thing or some event gained through involvement in or exposure to that thing or event....
, or expert opinion, or based on convenience of mathematical form. Bayesian statistics
Bayesian statistics
Bayesian statistics is that subset of the entire field of statistics in which the evidence about the true state of the world is expressed in terms of degrees of belief or, more specifically, Bayesian probabilities...
provides a theoretical framework for incorporating such subjectivity into a rigorous analysis: one specifies a prior probability distribution (which can be subjective) and then updates this distribution based on empirical data. An example of when such approach would be necessary is a situation in which an experimenter bends a coin slightly and tosses it once, recording whether it comes up heads, and is then given the task of predicting the probability that the next flip comes up heads. After bending the coin, the true probability that the coin will come up heads is unknown, so the experimenter would need to make an arbitrary decision (perhaps by looking at the shape of the coin) about what prior distribution to use. Incorporation of the subjective information is necessary in this case to get an accurate prediction of the probability, since otherwise one would guess 1 or 0 as the probability of the next flip being heads, which would be almost certainly wrong.
Complexity
In general, model complexity involves a trade-off between simplicity and accuracy of the model. Occam's razorOccam's razor
Occam's razor, also known as Ockham's razor, and sometimes expressed in Latin as lex parsimoniae , is a principle that generally recommends from among competing hypotheses selecting the one that makes the fewest new assumptions.-Overview:The principle is often summarized as "simpler explanations...
is a principle particularly relevant to modeling; the essential idea being that among models with roughly equal predictive power, the simplest one is the most desirable. While added complexity usually improves the realism of a model, it can make the model difficult to understand and analyze, and can also pose computational problems, including numerical instability. Thomas Kuhn
Thomas Kuhn
Thomas Samuel Kuhn was an American historian and philosopher of science whose controversial 1962 book The Structure of Scientific Revolutions was deeply influential in both academic and popular circles, introducing the term "paradigm shift," which has since become an English-language staple.Kuhn...
argues that as science progresses, explanations tend to become more complex before a Paradigm shift
Paradigm shift
A Paradigm shift is, according to Thomas Kuhn in his influential book The Structure of Scientific Revolutions , a change in the basic assumptions, or paradigms, within the ruling theory of science...
offers radical simplification.
For example, when modeling the flight of an aircraft, we could embed each mechanical part of the aircraft into our model and would thus acquire an almost white-box model of the system. However, the computational cost of adding such a huge amount of detail would effectively inhibit the usage of such a model. Additionally, the uncertainty would increase due to an overly complex system, because each separate part induces some amount of variance into the model. It is therefore usually appropriate to make some approximations to reduce the model to a sensible size. Engineers often can accept some approximations in order to get a more robust and simple model. For example Newton's
Isaac Newton
Sir Isaac Newton PRS was an English physicist, mathematician, astronomer, natural philosopher, alchemist, and theologian, who has been "considered by many to be the greatest and most influential scientist who ever lived."...
classical mechanics
Classical mechanics
In physics, classical mechanics is one of the two major sub-fields of mechanics, which is concerned with the set of physical laws describing the motion of bodies under the action of a system of forces...
is an approximated model of the real world. Still, Newton's model is quite sufficient for most ordinary-life situations, that is, as long as particle speeds are well below the speed of light
Speed of light
The speed of light in vacuum, usually denoted by c, is a physical constant important in many areas of physics. Its value is 299,792,458 metres per second, a figure that is exact since the length of the metre is defined from this constant and the international standard for time...
, and we study macro-particles only.
Training
Any model which is not pure white-box contains some parameterParameter
Parameter from Ancient Greek παρά also “para” meaning “beside, subsidiary” and μέτρον also “metron” meaning “measure”, can be interpreted in mathematics, logic, linguistics, environmental science and other disciplines....
s that can be used to fit the model to the system it is intended to describe. If the modeling is done by a neural network
Neural network
The term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...
, the optimization of parameters is called training. In more conventional modeling through explicitly given mathematical functions, parameters are determined by curve fitting
Curve fitting
Curve fitting is the process of constructing a curve, or mathematical function, that has the best fit to a series of data points, possibly subject to constraints. Curve fitting can involve either interpolation, where an exact fit to the data is required, or smoothing, in which a "smooth" function...
.
Model evaluation
A crucial part of the modeling process is the evaluation of whether or not a given mathematical model describes a system accurately. This question can be difficult to answer as it involves several different types of evaluation.Fit to empirical data
Usually the easiest part of model evaluation is checking whether a model fits experimental measurements or other empirical data. In models with parameters, a common approach to test this fit is to split the data into two disjoint subsets: training data and verification data. The training data are used to estimate the model parameters. An accurate model will closely match the verification data even though these data were not used to set the model's parameters. This practice is referred to as cross-validation in statistics.Defining a metric
Metric (mathematics)
In mathematics, a metric or distance function is a function which defines a distance between elements of a set. A set with a metric is called a metric space. A metric induces a topology on a set but not all topologies can be generated by a metric...
to measure distances between observed and predicted data is a useful tool of assessing model fit. In statistics, decision theory, and some economic models, a loss function
Loss function
In statistics and decision theory a loss function is a function that maps an event onto a real number intuitively representing some "cost" associated with the event. Typically it is used for parameter estimation, and the event in question is some function of the difference between estimated and...
plays a similar role.
While it is rather straightforward to test the appropriateness of parameters, it can be more difficult to test the validity of the general mathematical form of a model. In general, more mathematical tools have been developed to test the fit of statistical model
Statistical model
A statistical model is a formalization of relationships between variables in the form of mathematical equations. A statistical model describes how one or more random variables are related to one or more random variables. The model is statistical as the variables are not deterministically but...
s than models involving differential equations. Tools from non-parametric statistics
Non-parametric statistics
In statistics, the term non-parametric statistics has at least two different meanings:The first meaning of non-parametric covers techniques that do not rely on data belonging to any particular distribution. These include, among others:...
can sometimes be used to evaluate how well the data fit a known distribution or to come up with a general model that makes only minimal assumptions about the model's mathematical form.
Scope of the model
Assessing the scope of a model, that is, determining what situations the model is applicable to, can be less straightforward. If the model was constructed based on a set of data, one must determine for which systems or situations the known data is a "typical" set of data.The question of whether the model describes well the properties of the system between data points is called interpolation
Interpolation
In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points....
, and the same question for events or data points outside the observed data is called extrapolation
Extrapolation
In mathematics, extrapolation is the process of constructing new data points. It is similar to the process of interpolation, which constructs new points between known points, but the results of extrapolations are often less meaningful, and are subject to greater uncertainty. It may also mean...
.
As an example of the typical limitations of the scope of a model, in evaluating Newtonian classical mechanics
Classical mechanics
In physics, classical mechanics is one of the two major sub-fields of mechanics, which is concerned with the set of physical laws describing the motion of bodies under the action of a system of forces...
, we can note that Newton made his measurements without advanced equipment, so he could not measure properties of particles travelling at speeds close to the speed of light. Likewise, he did not measure the movements of molecules and other small particles, but macro particles only. It is then not surprising that his model does not extrapolate well into these domains, even though his model is quite sufficient for ordinary life physics.
Philosophical considerations
Many types of modeling implicitly involve claims about causalityCausality
Causality is the relationship between an event and a second event , where the second event is understood as a consequence of the first....
. This is usually (but not always) true of models involving differential equations. As the purpose of modeling is to increase our understanding of the world, the validity of a model rests not only on its fit to empirical observations, but also on its ability to extrapolate to situations or data beyond those originally described in the model. One can argue that a model is worthless unless it provides some insight which goes beyond what is already known from direct investigation of the phenomenon being studied.
An example of such criticism is the argument that the mathematical models of Optimal foraging theory
Optimal foraging theory
Optimal foraging theory is an idea in ecology based on the study of foraging behaviour and states that organisms forage in such a way as to maximize their net energy intake per unit time. In other words, they behave in such a way as to find, capture and consume food containing the most calories...
do not offer insight that goes beyond the common-sense conclusions of evolution
Evolution
Evolution is any change across successive generations in the heritable characteristics of biological populations. Evolutionary processes give rise to diversity at every level of biological organisation, including species, individual organisms and molecules such as DNA and proteins.Life on Earth...
and other basic principles of ecology.
See also
- Agent-based model
- Bio-inspired computing Biologically inspired computing
- CliodynamicsCliodynamicsthumb|Clio—detail from [[The Art of Painting|The Allegory of Painting]] by [[Johannes Vermeer]]Cliodynamics is a new multidisciplinary area of research focused at mathematical modeling of historical dynamics.-Origins:The term was originally coined by Peter...
- Computer simulationComputer simulationA computer simulation, a computer model, or a computational model is a computer program, or network of computers, that attempts to simulate an abstract model of a particular system...
- Conceptual modelConceptual modelIn the most general sense, a model is anything used in any way to represent anything else. Some models are physical objects, for instance, a toy model which may be assembled, and may even be made to work like the object it represents. They are used to help us know and understand the subject matter...
- Decision engineeringDecision engineeringDecision Engineering is a framework that unifies a number of best practices for organizational decision making. It is based on the recognition that, in many organizations, decision making could be improved if a more structured approach were used...
- List of computer simulation software
- Mathematical biologyMathematical biologyMathematical and theoretical biology is an interdisciplinary scientific research field with a range of applications in biology, medicine and biotechnology...
- Mathematical diagramMathematical diagramMathematical diagrams are diagrams in the field of mathematics, and diagrams using mathematics such as charts and graphs, that are mainly designed to convey mathematical relationships, for example, comparisons over time.- Argand diagram :...
- Mathematical models in physicsMathematical models in physicsMathematical models are of great importance in physics. Physical theories are almost invariably expressed using mathematical models, and the mathematics involved is generally more complicated than in the other sciences. Different mathematical models use different geometries that are not necessarily...
- Mathematical psychologyMathematical psychologyMathematical psychology is an approach to psychological research that is based on mathematical modeling of perceptual, cognitive and motor processes, and on the establishment of law-like rules that relate quantifiable stimulus characteristics with quantifiable behavior...
- Mathematical sociologyMathematical sociologyMathematical sociology is the usage of mathematics to construct social theories. Mathematical sociology aims to take sociological theory, which is strong in intuitive content but weak from a formal point of view, and to express it in formal terms...
Further reading
Books- Aris, Rutherford [ 1978 ] ( 1994 ). Mathematical Modelling Techniques, New York : Dover. ISBN 0-486-68131-9
- Bender, E.A. [ 1978 ] ( 2000 ). An Introduction to Mathematical Modeling, New York : Dover. ISBN 0-486-41180-X
- Lin, C.C. & Segel, L.A. ( 1988 ). Mathematics Applied to Deterministic Problems in the Natural Sciences, Philadelphia : SIAM. ISBN 0-89871-229-7
- Gershenfeld, N., The Nature of Mathematical Modeling, Cambridge University Press, (1998). ISBN 0521570956
- Yang, X.-S., Mathematical Modelling for Earth Sciences, Dudedin Academic, (2008). ISBN 1903765927
Specific applications
- PeierlsRudolf PeierlsSir Rudolf Ernst Peierls, CBE was a German-born British physicist. Rudolf Peierls had a major role in Britain's nuclear program, but he also had a role in many modern sciences...
, Rudolf. Model-making in physics, Contemporary Physics, Volume 21 (1), January 1980, 3-17 - Korotayev A., Malkov A., Khaltourina D. ( 2006 ). Introduction to Social Macrodynamics: Compact Macromodels of the World System Growth. Moscow : Editorial URSS. ISBN 5-484-00414-4
External links
General reference material- McLaughlin, Michael P. ( 1999 )
- Plus teacher and student package: Mathematical Modelling. Brings together all articles on mathematical modeling from Plus, the online mathematics magazine produced by the Millennium Mathematics Project at the University of Cambridge.
Philosophical background
- Frigg, R. and S. Hartmann, Models in Science, in: The Stanford Encyclopedia of Philosophy, (Spring 2006 Edition)
- Griffiths, E. C. (2010) What is a model?