Message Understanding Conference
Encyclopedia
The Message Understanding Conferences (MUC) were initiated and financed by DARPA (Defense Advanced Research Projects Agency) to encourage
the development of new and better methods of information extraction
.The character of this competition—many concurrent research teams competing against one another—required the development of standards
for evaluation, e.g. the adoption of metrics like precision and recall
.
information. From the second conference the output format, by which the participants'
systems would be evaluated, was prescribed. For each topic fields were given, which had to be
filled with information
from the text. Typical fields were, for example, the cause, the agent, the time and place of an event,
the consequences etc. The number of fields increased from conference to conference.
At the sixth conference (MUC-6) the task of recognition of named entities
and coreference
was added.
For named entity all phrases in the text were supposed to be marked as person, location, organization,
time or quantity.
The topics and text sources, which were processed, show a continuous move from military to civil themes, which mirrored
the change in business interest in information extraction
taking place at the time.
the development of new and better methods of information extraction
Information extraction
Information extraction is a type of information retrieval whose goal is to automatically extract structured information from unstructured and/or semi-structured machine-readable documents. In most of the cases this activity concerns processing human language texts by means of natural language...
.The character of this competition—many concurrent research teams competing against one another—required the development of standards
for evaluation, e.g. the adoption of metrics like precision and recall
Precision and recall
In pattern recognition and information retrieval, precision is the fraction of retrieved instances that are relevant, while recall is the fraction of relevant instances that are retrieved. Both precision and recall are therefore based on an understanding and measure of relevance...
.
Topics and Exercises
Only for the first conference (MUC-1) could the participant choose the output format for the extractedinformation. From the second conference the output format, by which the participants'
systems would be evaluated, was prescribed. For each topic fields were given, which had to be
filled with information
from the text. Typical fields were, for example, the cause, the agent, the time and place of an event,
the consequences etc. The number of fields increased from conference to conference.
At the sixth conference (MUC-6) the task of recognition of named entities
Named entity recognition
Named-entity recognition is a subtask of information extraction that seeks to locate and classify atomic elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc.Most research on NER...
and coreference
Coreference
In linguistics, co-reference occurs when multiple expressions in a sentence or document refer to the same thing; or in linguistic jargon, they have the same "referent."...
was added.
For named entity all phrases in the text were supposed to be marked as person, location, organization,
time or quantity.
The topics and text sources, which were processed, show a continuous move from military to civil themes, which mirrored
the change in business interest in information extraction
Information extraction
Information extraction is a type of information retrieval whose goal is to automatically extract structured information from unstructured and/or semi-structured machine-readable documents. In most of the cases this activity concerns processing human language texts by means of natural language...
taking place at the time.
| Conference | Year | Text Source | Topic (Domain) |
| MUC-1 | 1987 | Mil. reports | Fleet Operations |
| MUC-2 | 1989 | Mil. reports | Fleet Operations |
| MUC-3 | 1991 | News reports | Terrorist activities in Latin America |
| MUC-4 | 1992 | News reports | Terrorist activities in Latin America |
| MUC-5 | 1993 | News reports | Corporate Joint Ventures, Microelectronic production |
| MUC-6 | 1995 | News reports | Negotiation of Labor Disputes and Corporate Management Succession |
| MUC-7 | 1997 | News reports | Airplane crashes, and Rocket/Missile Launches |
Literature
- Ralph Grishman, Beth Sundheim: Message Understanding Conference - 6: A Brief History. In: Proceedings of the 16th International Conference on Computational Linguistics (COLING), I, Kopenhagen, 1996, 466–471.