Music information retrieval
Encyclopedia
Music information retrieval (MIR) is the interdisciplinary science of retrieving information
from music
. MIR is a small but growing field of research with many real-world applications. Those involved in MIR may have a background in musicology
, psychology
, academic music study, signal processing
, machine learning
or some combination of these.
, for example, uses experts to tag the music with particular qualities such as "female singer" or "strong bassline". Many other systems find users whose listening history is similar and suggests unheard music to the users from their respective collections. MIR techniques for similarity in music are now beginning to form part of such systems.
is required.
, mp3
, and ogg
are used when the audio itself is part of the analysis. Lossy formats such as mp3 and ogg work well with the human ear but may be missing crucial data for study. Additionally some encodings create artefacts which could be misleading to any automatic analyser. Despite this the ubiquity of the mp3 has meant much research in the field involves these as the source material. Increasingly, metadata mined from the web is incorporated in MIR for a more rounded understanding of the music within its cultural context, and this recently includes analysis of social tags for music.
of a piece of music. Other features may be employed to represent the chords, harmonies, melody, main pitch, beats per minute or rhythm in the piece.
Information
Information in its most restricted technical sense is a message or collection of messages that consists of an ordered sequence of symbols, or it is the meaning that can be interpreted from such a message or collection of messages. Information can be recorded or transmitted. It can be recorded as...
from music
Music
Music is an art form whose medium is sound and silence. Its common elements are pitch , rhythm , dynamics, and the sonic qualities of timbre and texture...
. MIR is a small but growing field of research with many real-world applications. Those involved in MIR may have a background in musicology
Musicology
Musicology is the scholarly study of music. The word is used in narrow, broad and intermediate senses. In the narrow sense, musicology is confined to the music history of Western culture...
, psychology
Psychology
Psychology is the study of the mind and behavior. Its immediate goal is to understand individuals and groups by both establishing general principles and researching specific cases. For many, the ultimate goal of psychology is to benefit society...
, academic music study, signal processing
Signal processing
Signal processing is an area of systems engineering, electrical engineering and applied mathematics that deals with operations on or analysis of signals, in either discrete or continuous time...
, machine learning
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
or some combination of these.
Applications of MIR
MIR is being used by businesses and academics to categorize, manipulate and even create music.Recommender systems
Several recommender systems for music already exist, but surprisingly few are based upon MIR techniques, instead making use of similarity between users or laborious data compilation. PandoraPandora
In Greek mythology, Pandora was the first woman. As Hesiod related it, each god helped create her by giving her unique gifts...
, for example, uses experts to tag the music with particular qualities such as "female singer" or "strong bassline". Many other systems find users whose listening history is similar and suggests unheard music to the users from their respective collections. MIR techniques for similarity in music are now beginning to form part of such systems.
Track separation and instrument recognition
Track separation is about extracting the original tracks as recorded, which could have more than one instrument played per track. Instrument recognition is about identifying the instruments involved and/or separating the music into one track per instrument. Various programs have been developed than can separate music into its component tracks without access to the master copy. In this way e.g. karaoke tracks can be created from normal music tracks, though the process is not yet perfect owing to vocals occupying some of the same frequency space as the other instruments.Automatic score creation
In combination with the above technique the written music for a piece can be generated from the audio content alone. This task becomes more difficult with greater numbers of instruments and greater similarity between instruments.Automatic categorization
Musical genre categorization is a common task for MIR and is the usual task for the yearly Music Information Retrieval Evaluation eXchange(MIREX). Machine learning techniques such as Support Vector Machines tend to perform well, despite the somewhat subjective nature of the classification. Other potential classifications include identifying the artist, the place of origin or the mood of the piece. Where the output is expected to be a number rather than a class, regression analysisRegression analysis
In statistics, regression analysis includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables...
is required.
Music generation
The automatic generation of music is a goal held by many MIR researchers. Attempts have been made with limited success in terms of human appreciation of the results.Data Source
Scores give a clear and logical description of music from which to work, but access to a weird but unknown and common score(also known as sheet music which no-one get) is often impractical. MIDI music has also been used for similar reasons, but some data is lost in the conversion to MIDI from any other format, unless the music was written with the MIDI standards in mind, which is rare. Digital audio formats such as WAVWAV
Waveform Audio File Format , is a Microsoft and IBM audio file format standard for storing an audio bitstream on PCs...
, mp3
MP3
MPEG-1 or MPEG-2 Audio Layer III, more commonly referred to as MP3, is a patented digital audio encoding format using a form of lossy data compression...
, and ogg
Ogg
Ogg is a free, open container format maintained by the Xiph.Org Foundation. The creators of the Ogg format state that it is unrestricted by software patents and is designed to provide for efficient streaming and manipulation of high quality digital multimedia.The Ogg container format can multiplex...
are used when the audio itself is part of the analysis. Lossy formats such as mp3 and ogg work well with the human ear but may be missing crucial data for study. Additionally some encodings create artefacts which could be misleading to any automatic analyser. Despite this the ubiquity of the mp3 has meant much research in the field involves these as the source material. Increasingly, metadata mined from the web is incorporated in MIR for a more rounded understanding of the music within its cultural context, and this recently includes analysis of social tags for music.
Feature Representation
Analysis can often require some summarising, and for music (as with many other forms of data) this is achieved by feature extraction, especially when the audio content itself is analysed and machine learning is to be applied. The purpose is to reduce the sheer quantity of data down to a manageable set of values so that learning can be performed within a reasonable time-frame. One common feature extracted is the Mel-Frequency Cepstral Coefficient (MFCC) which is a measure of the timbreTimbre
In music, timbre is the quality of a musical note or sound or tone that distinguishes different types of sound production, such as voices and musical instruments, such as string instruments, wind instruments, and percussion instruments. The physical characteristics of sound that determine the...
of a piece of music. Other features may be employed to represent the chords, harmonies, melody, main pitch, beats per minute or rhythm in the piece.
Statistics and Machine Learning
- Computational methods for classification, clustering, and modelling — musical feature extraction for mono- and polyphonic music, similarity and pattern matchingPattern matchingIn computer science, pattern matching is the act of checking some sequence of tokens for the presence of the constituents of some pattern. In contrast to pattern recognition, the match usually has to be exact. The patterns generally have the form of either sequences or tree structures...
, retrieval - Formal methods and databases — applications of automated music identification and recognition, such as score followingScore followingScore following is the process of automatically listening to a live music performance and tracking the position in the score. It is an active area of research and stands at the intersection of artificial intelligence, pattern recognition, signal processing, and musicology...
, automatic accompaniment, routing and filtering for music and music queries, query languages, standards and other metadata or protocols for music information handling and retrievalInformation retrievalInformation retrieval is the area of study concerned with searching for documents, for information within documents, and for metadata about documents, as well as that of searching structured storage, relational databases, and the World Wide Web...
, multi-agent systemMulti-agent systemA multi-agent system is a system composed of multiple interacting intelligent agents. Multi-agent systems can be used to solve problems that are difficult or impossible for an individual agent or a monolithic system to solve...
s, distributed search) - Software for music information retrieval — Semantic WebSemantic WebThe Semantic Web is a collaborative movement led by the World Wide Web Consortium that promotes common formats for data on the World Wide Web. By encouraging the inclusion of semantic content in web pages, the Semantic Web aims at converting the current web of unstructured documents into a "web of...
and musical digital objects, intelligent agents, collaborative software, web-based search and semantic retrieval, query by hummingQuery by hummingQuery by humming is a music retrieval system that branches off the original classification systems of title, artist, composer, and genre. It normally applies to songs or other music with a distinct single theme or melody. The system involves taking a user-hummed melody and comparing it to an...
, acoustic fingerprinting - Music analysis and knowledge representation — automatic summarization, citing, excerpting, downgrading, transformation, formal models of music, digital scores and representations, music indexing and metadataMetadataThe term metadata is an ambiguous term which is used for two fundamentally different concepts . Although the expression "data about data" is often used, it does not apply to both in the same way. Structural metadata, the design and specification of data structures, cannot be about data, because at...
.
Other Issues
- Human-computer interaction and interfaces — multi-modal interfaces, user interfaceUser interfaceThe user interface, in the industrial design field of human–machine interaction, is the space where interaction between humans and machines occurs. The goal of interaction between a human and a machine at the user interface is effective operation and control of the machine, and feedback from the...
s and usabilityUsabilityUsability is the ease of use and learnability of a human-made object. The object of use can be a software application, website, book, tool, machine, process, or anything a human interacts with. A usability study may be conducted as a primary job function by a usability analyst or as a secondary job...
, mobile applications, user behavior - Music perception, cognition, affect, and emotions — music similarity metrics, syntactical parameters, semantic parameters, musical forms, structures, styles ands, music annotation methodologies
- Music archives, libraries, and digital collections — music digital librariesDigital libraryA digital library is a library in which collections are stored in digital formats and accessible by computers. The digital content may be stored locally, or accessed remotely via computer networks...
, public access to musical archives, benchmarks and research databases - Intellectual propertyIntellectual propertyIntellectual property is a term referring to a number of distinct types of creations of the mind for which a set of exclusive rights are recognized—and the corresponding fields of law...
rights and music — national and international copyrightCopyrightCopyright is a legal concept, enacted by most governments, giving the creator of an original work exclusive rights to it, usually for a limited time...
issues, digital rights managementDigital rights managementDigital rights management is a class of access control technologies that are used by hardware manufacturers, publishers, copyright holders and individuals with the intent to limit the use of digital content and devices after sale. DRM is any technology that inhibits uses of digital content that...
, identification and traceability - Sociology and Economy of music — music industry and use of MIR in the production, distribution, consumption chain, user profiling, validation, user needs and expectations, evaluation of music IR systems, building test collections, experimental design and metrics
See also
- Audio miningAudio miningAudio mining is a technique by which the content of an audio signal can be automatically analysed and searched. It is most commonly used in the field of automatic speech recognition, where the analysis tries to identify any speech within the audio...
- Artificial intelligenceArtificial intelligenceArtificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
- Digital rights managementDigital rights managementDigital rights management is a class of access control technologies that are used by hardware manufacturers, publishers, copyright holders and individuals with the intent to limit the use of digital content and devices after sale. DRM is any technology that inhibits uses of digital content that...
- Digital signal processingDigital signal processingDigital signal processing is concerned with the representation of discrete time signals by a sequence of numbers or symbols and the processing of these signals. Digital signal processing and analog signal processing are subfields of signal processing...
- EthnomusicologyEthnomusicologyEthnomusicology is defined as "the study of social and cultural aspects of music and dance in local and global contexts."Coined by the musician Jaap Kunst from the Greek words ἔθνος ethnos and μουσική mousike , it is often considered the anthropology or ethnography of music...
- Multimedia Information RetrievalMultimedia Information RetrievalMultimedia Information Retrieval is a research discipline of computer science that aims at extracting semantic information from multimedia data sources. Data sources include directly perceivable media such as audio, image and video, indirectly perceivable sources such as text, biosignals as well...
- Music notation
- MusicologyMusicologyMusicology is the scholarly study of music. The word is used in narrow, broad and intermediate senses. In the narrow sense, musicology is confined to the music history of Western culture...
- MusipediaMusipediaMusipedia is a search engine for identifying pieces of music. This can be done by whistling a theme, playing it on a virtual piano keyboard, tapping the rhythm on the computer keyboard, or entering the Parsons code...
- Parsons codeParsons codeThe Parsons code, formally named the Parsons Code for Melodic Contours, is a simple notation used to identify a piece of music through melodic motion—the motion of the pitch up and down. Denys Parsons developed this system for his 1975 book, The Directory of Tunes and Musical Themes...
- Sound and music computingSound and music computingSound and Music Computing is a research field that studies the whole sound and music communication chain from a multidisciplinary point of view...
External links
- International Society for Music Information Retrieval
- Music Information Retrieval research
- J. Stephen Downie: Music information retrieval
- Nicola Orio: Music Retrieval: A Tutorial and Review
- Intelligent Audio Systems: Foundations and Applications of Music Information Retrieval, introductory course at Stanford University's Center for Computer Research in Music and Acoustics
- Micheline Lesaffre: Music Information Retrieval: Conceptual Framework, Annotation and User behavior.
- The Echo Nest: a company specialising in MIR research and applications.
- Imagine Research : develops platform and software for MIR applications
Example MIR applications
- Musipedia — A melody search engine that offers several modes of searching, including whistling, tapping, piano keyboard, and Parsons code.
- The Listen Game — UCSD Computer Audition Lab MIR music ranking game
- Peachnote — A melody search engine and n-gram viewer that searches through digitized music scores