Windows software trace preprocessor
Encyclopedia
The Windows
software trace preprocessor (abbreviated WPP; the preprocessor and related support tools are known as WPP Software Tracing) is a preprocessor
that simplifies the use of WMI
event tracing to implement efficient software tracing
in drivers
and applications that target Windows 2000
and later operating systems. WPP was created by Microsoft
and is included in the Windows DDK
. Although WPP is wide in its applicability, it is not included in the Windows SDK, and therefore is primarily used for drivers and driver support software produced by software vendors that purchase the Windows DDK.
. In contrast to event logging, the primary purpose of which is to produce records of events that can be audited by system administrator
s (see for example Event Viewer
) or analyzed by management tools, software tracing is primarily a debugging aid for software developer
s. As such, many of the non-functional requirements
of event logging, such as localizability or a standards-based
output format, are explicitly non-goals for most applications of software tracing. On the other hand, software tracing has special requirements for performance
that are not generally as important in event logging. For example, one common use of software tracing, in/out tracing, produces output at the entry point and return of functions or methods
so that a developer can visually follow the execution path, often including parameters
and return values, in a debugger or text-based log file (this can be seen as a run-time analog of a sequence diagram
). This type of tracing, although useful for developers, can greatly hurt performance of a software product if it cannot be disabled (either at compile-time
via conditional compilation, or at run-time via flags
).
Additional considerations special to software tracing include the following:
Due to the first two considerations, traditional methods of software tracing use conditional compilation to enable or disable tracing (and inclusion of tracing data) at compile-time. For example, using the C preprocessor
, one might define a macro
#ifdef _DEBUG
#define DebugOut(msg, ...) \
DebugPrintf(__FUNCTION__ "(" __FILE__ ":" TO_STRING(__LINE__) ")\t" \
msg, __VAR_ARGS__)
#else
#define DebugOut(msg, ...)
#endif
where
-like function that might for example output text to the debugger.
Then, the following code:
DebugOut("Error %d occurred\n", error_code);
would produce output similar to the following on debug builds only:
SomeFunction(file.c:78) Error 217 occurred
Another technique for certain types of tracing (particularly in/out tracing) is to use instrumentation
. While this technique can address many of the major concerns, it is not always available (typically only in managed code
).
WMI event tracing is an example of a technology that addresses in particular performance of tracing in performance-critical code such as drivers. It can also address the concern of controlling the distribution of sensitive trace information by letting a developer define the human-readable tracing data (
// File: file.cxx
// This file is an example of using WPP
#include "file.tmh"
WPP's understanding of C
/C++
syntax is very limited. In particular, it does not expand macros (except in special circumstances where it is necessary), nor does it handle pragma
s or perform any semantic analysis.
A developer specifies one or more tracing macros that WPP should handle, via a configuration file, special annotations in comments, command line parameters, or some combination of these methods. Each time WPP encounters one of the macros that it is supposed to handle, it generates a trace message macro. In other words, if for example
How trace message macros are generated by WPP depends on a template file (the format of the file is undocumented). The default template files included with WPP specify that the string of a trace message should be included in an annotation
(using the __annotation feature of the Microsoft Compiler). These strings are not included in the compiled code, but are included in the debugger symbol file in a format that tools included with WPP can understand. The trace message macros also include the logic for enabling or disabling tracing via flags and the calls to WMI event tracing APIs.
#define CheckForErrors(error_code) \
if (IsError(error_code)) \
{ \
DoTrace("Error %d occurred\n", err); \
HandleError(error_code); \
}
then WPP will not generate the trace message macros for
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
software trace preprocessor (abbreviated WPP; the preprocessor and related support tools are known as WPP Software Tracing) is a preprocessor
Preprocessor
In computer science, a preprocessor is a program that processes its input data to produce output that is used as input to another program. The output is said to be a preprocessed form of the input data, which is often used by some subsequent programs like compilers...
that simplifies the use of WMI
Windows Management Instrumentation
Windows Management Instrumentation is a set of extensions to the Windows Driver Model that provides an operating system interface through which instrumented components provide information and notification...
event tracing to implement efficient software tracing
Tracing (software)
In software engineering, tracing is a specialized use of logging to record information about a program's execution. This information is typically used by programmers for debugging purposes, and additionally, depending on the type and detail of information contained in a trace log, by experienced...
in drivers
Device driver
In computing, a device driver or software driver is a computer program allowing higher-level computer programs to interact with a hardware device....
and applications that target Windows 2000
Windows 2000
Windows 2000 is a line of operating systems produced by Microsoft for use on personal computers, business desktops, laptops, and servers. Windows 2000 was released to manufacturing on 15 December 1999 and launched to retail on 17 February 2000. It is the successor to Windows NT 4.0, and is the...
and later operating systems. WPP was created by Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
and is included in the Windows DDK
Windows DDK
The Windows Driver Kit is a software toolset from Microsoft that enables the development of device drivers for the Microsoft Windows platform. It includes documentation, samples, build environments, and tools for driver developers.-History of WDK:...
. Although WPP is wide in its applicability, it is not included in the Windows SDK, and therefore is primarily used for drivers and driver support software produced by software vendors that purchase the Windows DDK.
Background
Software tracing is a specialized use of logging to record information about a program's execution. This information is commonly used for debuggingDebugging
Debugging is a methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece of electronic hardware, thus making it behave as expected. Debugging tends to be harder when various subsystems are tightly coupled, as changes in one may cause bugs to emerge...
. In contrast to event logging, the primary purpose of which is to produce records of events that can be audited by system administrator
System administrator
A system administrator, IT systems administrator, systems administrator, or sysadmin is a person employed to maintain and operate a computer system and/or network...
s (see for example Event Viewer
Event Viewer
Event Viewer, a component of Microsoft's Windows NT line of operating systems, lets administrators and users view the event logs on a local or remote machine. In Windows Vista, Microsoft overhauled the event system.- Overview :...
) or analyzed by management tools, software tracing is primarily a debugging aid for software developer
Software developer
A software developer is a person concerned with facets of the software development process. Their work includes researching, designing, developing, and testing software. A software developer may take part in design, computer programming, or software project management...
s. As such, many of the non-functional requirements
Non-functional requirements
In systems engineering and requirements engineering, a non-functional requirement is a requirement that specifies criteria that can be used to judge the operation of a system, rather than specific behaviors. This should be contrasted with functional requirements that define specific behavior or...
of event logging, such as localizability or a standards-based
Standardization
Standardization is the process of developing and implementing technical standards.The goals of standardization can be to help with independence of single suppliers , compatibility, interoperability, safety, repeatability, or quality....
output format, are explicitly non-goals for most applications of software tracing. On the other hand, software tracing has special requirements for performance
Performance tuning
Performance tuning is the improvement of system performance. This is typically a computer application, but the same methods can be applied to economic markets, bureaucracies or other complex systems. The motivation for such activity is called a performance problem, which can be real or anticipated....
that are not generally as important in event logging. For example, one common use of software tracing, in/out tracing, produces output at the entry point and return of functions or methods
Method (computer science)
In object-oriented programming, a method is a subroutine associated with a class. Methods define the behavior to be exhibited by instances of the associated class at program run time...
so that a developer can visually follow the execution path, often including parameters
Parameter (computer science)
In computer programming, a parameter is a special kind of variable, used in a subroutine to refer to one of the pieces of data provided as input to the subroutine. These pieces of data are called arguments...
and return values, in a debugger or text-based log file (this can be seen as a run-time analog of a sequence diagram
Sequence diagram
A sequence diagram in Unified Modeling Language is a kind of interaction diagram that shows how processes operate with one another and in what order. It is a construct of a Message Sequence Chart....
). This type of tracing, although useful for developers, can greatly hurt performance of a software product if it cannot be disabled (either at compile-time
Compiler
A compiler is a computer program that transforms source code written in a programming language into another computer language...
via conditional compilation, or at run-time via flags
Flag (computing)
In computer programming, flag can refer to one or more bits that are used to store a binary value or code that has an assigned meaning, but can refer to uses of other data types...
).
Additional considerations special to software tracing include the following:
- In proprietary softwareProprietary softwareProprietary software is computer software licensed under exclusive legal right of the copyright holder. The licensee is given the right to use the software under certain conditions, while restricted from other uses, such as modification, further distribution, or reverse engineering.Complementary...
, tracing data may include sensitive information about the product's source codeSource codeIn computer science, source code is text written using the format and syntax of the programming language that it is being written in. Such a language is specially designed to facilitate the work of computer programmers, who specify the actions to be performed by a computer mostly by writing source...
. - If tracing is enabled or disabled at run-time, many methods of tracing require a significant amount of additional data be included in the binary, which can indirectly hurt performance even when tracing is disabled.
- If tracing is enabled or disabled at compile-time, getting trace data for an issue on a customer machine depends on the customer being willing and able to install a special, tracing enabled, version of your software.
- Certain types of software, such as drivers, need to meet strict performance requirements even with tracing enabled.
Due to the first two considerations, traditional methods of software tracing use conditional compilation to enable or disable tracing (and inclusion of tracing data) at compile-time. For example, using the C preprocessor
C preprocessor
The C preprocessor is the preprocessor for the C and C++ computer programming languages. The preprocessor handles directives for source file inclusion , macro definitions , and conditional inclusion ....
, one might define a macro
DebugOut as follows:#ifdef _DEBUG
#define DebugOut(msg, ...) \
DebugPrintf(__FUNCTION__ "(" __FILE__ ":" TO_STRING(__LINE__) ")\t" \
msg, __VAR_ARGS__)
#else
#define DebugOut(msg, ...)
#endif
where
TO_STRING is a macro that converts the line number (__LINE__) to a string and DebugPrintf is a printfPrintf
Printf format string refers to a control parameter used by a class of functions typically associated with some types of programming languages. The format string specifies a method for rendering an arbitrary number of varied data type parameter into a string...
-like function that might for example output text to the debugger.
Then, the following code:
DebugOut("Error %d occurred\n", error_code);
would produce output similar to the following on debug builds only:
SomeFunction(file.c:78) Error 217 occurred
Another technique for certain types of tracing (particularly in/out tracing) is to use instrumentation
Instrumentation (computer programming)
In context of computer programming, instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and to write trace information. Programmers implement instrumentation in the form of code instructions that monitor specific components in a system...
. While this technique can address many of the major concerns, it is not always available (typically only in managed code
Managed code
Managed code is a term coined by Microsoft to identify computer program code that requires and will only execute under the "management" of a Common Language Runtime virtual machine ....
).
WMI event tracing is an example of a technology that addresses in particular performance of tracing in performance-critical code such as drivers. It can also address the concern of controlling the distribution of sensitive trace information by letting a developer define the human-readable tracing data (
"Error %d occurred\n" in the example above) separately from the code so that it is not built into the product (in the code, a specific message is referred to by its message number). However, there are some important limitations:
- WMI event tracing cannot, by itself, automatically generate the
"SomeFunction(file.c:78)"part of the trace message. This is a limitation of all such technologies, not specific to WMI event tracing. - Requiring the human-readable part of the tracing data to be separated from the code can decrease the readability of the code.
- Using this technique can introduce significant development overhead for "one-shot" tracing messages.
Operation of WPP
WPP is run prior to compilation (in other words, before even the C preprocessor), and generates a trace message header for each file that it processes (by default this header isfilename.tmh, where filename is the name of the processed source file). This header must then be explicitly included into the source file, for example:// File: file.cxx
// This file is an example of using WPP
#include "file.tmh"
WPP's understanding of C
C (programming language)
C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
/C++
C++
C++ is a statically typed, free-form, multi-paradigm, compiled, general-purpose programming language. It is regarded as an intermediate-level language, as it comprises a combination of both high-level and low-level language features. It was developed by Bjarne Stroustrup starting in 1979 at Bell...
syntax is very limited. In particular, it does not expand macros (except in special circumstances where it is necessary), nor does it handle pragma
Pragma
Pragma may refer to:* πράγμα, the Greek word that William James identified to be the root of the word pragmatism* A directive , an instruction communicating additional information to a computer software compiler...
s or perform any semantic analysis.
A developer specifies one or more tracing macros that WPP should handle, via a configuration file, special annotations in comments, command line parameters, or some combination of these methods. Each time WPP encounters one of the macros that it is supposed to handle, it generates a trace message macro. In other words, if for example
DoTrace is a tracing macro, WPP will generate a separate macro for each occurrence of DoTrace. The generated trace message macros are disambiguated by file name and line number, and, using various preprocessor tricks, WPP in turn defines the original tracing macro so that it will expand the appropriate trace message macro at each occurrence.How trace message macros are generated by WPP depends on a template file (the format of the file is undocumented). The default template files included with WPP specify that the string of a trace message should be included in an annotation
Annotation
An annotation is a note that is made while reading any form of text. This may be as simple as underlining or highlighting passages.Annotated bibliographies give descriptions about how each source is useful to an author in constructing a paper or argument...
(using the __annotation feature of the Microsoft Compiler). These strings are not included in the compiled code, but are included in the debugger symbol file in a format that tools included with WPP can understand. The trace message macros also include the logic for enabling or disabling tracing via flags and the calls to WMI event tracing APIs.
Limitations
- Because WPP doesn't expand macros, it won't recognize an instantiation of a tracing macro that is included in the definition of another macro. For example, if
DoTraceis a tracing macro, and a macroCheckForErrorsis defined as:
#define CheckForErrors(error_code) \
if (IsError(error_code)) \
{ \
DoTrace("Error %d occurred\n", err); \
HandleError(error_code); \
}
then WPP will not generate the trace message macros for
DoTrace where CheckForErrors occurs. WPP provides an ad hoc workaround for this issue, but there still exists a small class of macros that cannot be expressed even using the workaround.
- The default template file generates code that will only work properly with the Microsoft compiler. Although this is not an inherent limitation of the preprocessor, the fact that the template file (which controls what code is generated in the trace message header) uses an undocumented format means that in practice WPP will only work properly with the Microsoft compiler.
- Earlier versions of WPP caused compiling errors when more than one trace macro header was included into a source file (for example, if a source file with tracing included a header that had tracing in inline functions). This is fixed in the most recent version. Note that this is also a limitation of the template file, not the WPP tool itself.
- Because trace message macros are disambiguated by file and line number, there can be only one tracing macro per line in the source code.