Problem Statement Increasing growth and popularity of multimedia resources available on the Web brought the need to provide new, more advanced tools needed for their search. However, searching through multimedia data is highly non-trivial task that requires content-based indexing of the data. Our research will focus on automatic extraction of information about the sound timbre, and indexing sound data with information about musical instrument playing in a given segment. Sound timbre is a very important factor that can affect the perceptual grouping of music. The real use of timbre-based grouping of music is very nicely discussed in (Bregman, 1990). The investigators will carry out this interdisciplinary research on the basis of the disciplines they are experts in, including knowledge discovery in databases (KDD), music, digital signal processing, and mathematical logic.

Our aim is to perform automatic classification of musical instrument sound from real recordings for broad range of sounds, independently of the fundamental frequency of the sound. We will focus our research on musical instruments of definite pitch, used in contemporary orchestras. Full range of musical scale for each instrument will be investigated. We will start our investigations with descriptors depicted in MPEG-7. Although MPEG-7 provides some tools for indexing with musical instrument names, this information is inserted rather manually (for instance, tracks are labeled with voices/instruments in recording studios). There are no algorithms included in MPEG-7 to automate this task. In order to index enormous amount of audio files of various origin which are available for users on the Web, special processing and new algorithms are needed to extract this kind of knowledge directly from audio signals.

We plan to base our research on low-level descriptors that can be easily extracted automatically for any audio signal. Apart from observing descriptor set for a given frame, we will also trace descriptor changes in time. Finally, if MPEG-7 becomes commonly used as standard, the results of our research will provide its interoperability for various applications in the music domain. Automatic sound indexing should allow labeling sound segments with instrument names. We will start with singular, homophonic sounds of musical instruments, and then extend our investigations to simultaneous, polyphonic sounds. Knowledge discovery techniques will be applied at this stage of research. First of all, we have to discover rules that recognize various musical instruments. Next, we apply these rules, one by one, to unknown sounds. By identifying so called supporting rules, we should be able to point out which instrument is playing (or is dominating) in a given segment, and in what time instants this instrument starts and ends playing. Additionally, we plan to extract sound parameter called pitch information which is one of the important factors in sound classification. By combining melody and timbre information, we should be able to search successfully for favorite tunes played by favorite instruments.