Creation and Exploration of Musical Information Spaces

Size: px

Start display at page:

Download "Creation and Exploration of Musical Information Spaces"

Edmund Patrick
5 years ago
Views:

1 Creation and Exploration of Musical Information Spaces Andreas Rauber Vienna University of Technology ICDL 2004, New Delhi

2 From Directories to Music Spaces

3 Outline Motivation Feature Extraction Organization: SOM&Co Demonstration and Evaluation

4 Motivation Music omnipresent Large collections - on small devices Increasingly distributed electronically but Difficult to search for textual query by humming

5 Motivation Automatic genre/style - based organization Navigational interface Playlist generation Discovering unknown titles/artists Challenge: How to compute similarity of music?

6 Outline Motivation Feature Extraction Organization: SOM&Co Demonstration and Evaluation

7 Feature Extraction - Overview raw audio data Preprocessing Feature Extraction Specific Loudness Sensation downsampling: 44kHz to 11kHz stereo to mono cut into 6sec segments remove lead-in and fade out keep every 3rd segment PCM data segments Rhythm Patterns

8 Feature Extraction - Loudness Elise Freak on a Leash PCM Audio Signal Power Spectrum Frequency Bands Masking Effects Phon Sone

9 Feature Extraction - Rhythm Elise Freak on a Leash Loudness Modulation Amplitude Fluctuation Strength Filter (Gradient, Gauss Median 1200-dim feature vec.

10 Outline Motivation Feature Extraction Organization: SOM&Co Demonstration and Evaluation

11 SOM Basics Self-Organizing Map, Kohonen Map determines mapping from high-dimensional input space to 2-dim output space ("map") such that neighborhood relationships in data are preserved "spatially smooth k-means"

12 Self-Organizing Map Self-Organizing Map (SOM)

13 Self-Organizing Map Self-Organizing Map (SOM)

14 Self-Organizing Map Self-Organizing Map (SOM)

15 Self-Organizing Map Self-Organizing Map (SOM) Smoothed Data Histograms (SDH)

16 GHSOM: Growing Hierarchical SOM based on SOM model (GG & HierSOM) dynamic hierarchical growth (divisive alg., top-down refinement) dynamic horizontal growth (granularity gain per layer) unbalanced structure according to data requirements

17 GHSOM Architecture

18 Outline Motivation Feature Extraction Organization: SOM&Co Demonstration and Evaluation

19 Experiments - Coll77 77 pieces of music (~5 hrs) variety of different genres

20 Experiments - Coll pieces of music (~24 hrs) variety of different genres 2x4 top-layer map all units expanded as layer 2 maps 25 out of 64 units expanded on layer 3

21 Experiments - Coll359

22 Experiments - Coll359 Toccata and Fuge in D Minor Vanessa Mae: Scherzo in D Minor Partita #3 in E for Solo Violin Tequila Mockingbird Tocata and Fuge in D Minor The 4 Seasons Red Violin Classical Gas

23 Experiments - Dance Sport 1129 pieces of dance music (~56 hrs) 10 dances (IDSF): LATIN Samba Cha-Cha-Cha Rumba Paso Doble Jive BALLROOM Slow Waltz Tango Viennese Waltz Slow Foxtrott Quickstep GHSOM top 4x2, all expanded on layer 2

24 Experiments - Dance Sport

25 Experiments - Dance Sport Slow Waltz Slow Foxtrott Viennese Waltz Tango Quickstep Rumba Paso Doble Jive Samba Cha-Cha-Cha JI37 JI63 SA44 SA01 QS67 RU61

26 Conclusions Organization ofmusic by sound similarity Content-based access and retrieval SOMeJB prototype available at

29 Experiments - Dance Sport

30 Experiments - Dance Sport knn, k=5: CC JI LW PD QS RU SA SF TG WW

31 Experiments - Others On automatic genre-based evaluation: Classic-Cluster: 14 classical orchestra pieces plus: Heavy Metal: Metallica: The Extasy of Gold Jazz-Cluster: 8 Jazz titles plus: Text/Speech: Dorfer: Freispiel

32 Feature Extraction - Hanning

33 Feature Extraction - Bark Scale

34 Feature Extraction - Phon equal loudness contours for 3, 20, 40, 60, 80, 100 phon

35 Feature Extraction - Sone

36 Feature Extraction - Fluctuation

37 SOM Training x m(t+1) m(t)

38 GHSOM Demonstration layer 1 layer 2 layer 3

Welcome to the 2015 Jacksonville Star Ball

Welcome to the 2015 Jacksonville Star Ball Dear Dancers, We have a very exciting event lined up for you at this year s Jacksonville Star Ball. We have outlined our tentative schedule for this weekend to