Feature Extraction for Musical Genre Classification MUS15

Transcription

1 Feature Extraction for Musical Genre Classification MUS15 Feature Extraction for Musical Genre Classification MUS15 Kilian Merkelbach June 22, 2015 Kilian Merkelbach June 21, 22, /26 1/26

2 Intro - Contents 1 Intro 2 History 3 Methods Tzanetakis & Cook Correa, Costa & Saito 4 Conclusion Kilian Merkelbach June 22, /26

3 Intro - What are genres? Kilian Merkelbach June 22, /26

4 Intro - Genres provide order Kilian Merkelbach June 22, /26

5 Intro - Pipeline - Feature extraction Kilian Merkelbach June 22, /26

6 Intro - Pipeline - Training and Classification Kilian Merkelbach June 22, /26

7 History - Contents 1 Intro 2 History 3 Methods Tzanetakis & Cook Correa, Costa & Saito 4 Conclusion Kilian Merkelbach June 22, /26

8 History - MP3 revolution 1998: First MP3 players available 1999: Napster, mp3.com go online Commercial and private music collections explode Kilian Merkelbach June 22, /26

9 History - First steps 2001: Tzanetakis and Cook build foundation by publishing first paper from then on: methods evolve from previous ones, always trying to improve performance Kilian Merkelbach June 22, /26

10 Methods - Contents 1 Intro 2 History 3 Methods Tzanetakis & Cook Correa, Costa & Saito 4 Conclusion Kilian Merkelbach June 22, /26

11 Methods - State of the Art Tzanetakis, Cook: "Musical Genre Classification of Audio Signals" Correa, Costa, Saito: "Tracking the Beat: Classification of Music Genres and Synthesis of Rhythms" Kilian Merkelbach June 22, /26

12 Methods - Tzanetakis & Cook Features Overview Timbral Texture Features (19 dim.) Rhythmic Content Features (6 dim.) Pitch Content Features (5 dim.) Kilian Merkelbach June 22, /26

13 Methods - Tzanetakis & Cook Timbral Texture Features I based on short time Fourier transform (STFT) Analysis Window (23ms) vs. Texture Window (1s) [1] Kilian Merkelbach June 22, /26

14 Methods - Tzanetakis & Cook Timbral Texture Features II calculate four features from STFT output and use their mean and variance for classification e.g. Spectral Flux: F t = N (N t [n] N t 1 [n]) 2 n=1 additional features based on Mel-frequency cepstral coefficients (MFCCs) Kilian Merkelbach June 22, /26

15 Methods - Tzanetakis & Cook Rhythmic Content Features Discrete Wavelet Transform (DWT) to analyze signal autocorrelation function recognizes strong beats (example) map peaks into Beat Histogram Kilian Merkelbach June 22, /26

16 Methods - Tzanetakis & Cook Beat Histogram [3] Kilian Merkelbach June 22, /26

17 Methods - Tzanetakis & Cook Pitch Content Features Pitch Histogram: like Beat Histogram ( s) but with shorter time frame (2-50ms) Kilian Merkelbach June 22, /26

18 Methods - Tzanetakis & Cook Results 59% accuracy with 10 different genres 77% among classical genres 61% among jazz genres [3] Kilian Merkelbach June 22, /26

19 Methods - Correa, Costa & Saito Tracking the Beat Songs given as MIDI files Use directed graphs to describe song Kilian Merkelbach June 22, /26

20 Methods - Correa, Costa & Saito Digraphs Weighted directed graphs with note lengths as vertices Weights defined by frequency of note sequence [2] Kilian Merkelbach June 22, /26

21 Methods - Correa, Costa & Saito Features Build digraph for each song: = 324 dim. Additional features from digraph: 15 dim. Use PCA to obtain 52 dim. feature vector Kilian Merkelbach June 22, /26

22 Methods - Correa, Costa & Saito Results 85.72% accuracy with 4 different genres (blues, bossa-nova, reggae and rock) Kilian Merkelbach June 22, /26

23 Conclusion - Contents 1 Intro 2 History 3 Methods Tzanetakis & Cook Correa, Costa & Saito 4 Conclusion Kilian Merkelbach June 22, /26

24 Conclusion - Prediction by humans College students found correct genre (out of 10) with 70% accuracy after 3 seconds [3] Machines are on par Human experts still better if there are many genres Kilian Merkelbach June 22, /26

25 Conclusion - Future Research step away from speech recognition features (e.g. MFCCs) fuzzy classification (e.g. 90% Rock, 10% Blues) larger genre sets, higher accuracy Kilian Merkelbach June 22, /26

26 Conclusion - References Wikimedia Commons. Short time fourier transform, Debora C Correa, Luciano da F Costa, and Jose H Saito. Tracking the beat: Classification of music genres and synthesis of rhythms. G. Tzanetakis and P. Cook. Musical genre classification of audio signals. Speech and Audio Processing, IEEE Transactions on, 10(5): , Jul Kilian Merkelbach June 22, /26