Song Classification for Dancing

1
Song Classification for Dancing

• Manolis Cristodoukalis, Costas Iliopoulos, M.
  Sohel Rahman, W.F. Smyth

2
Classical Pattern Matching

• Input A text T T1..n, A Pattern P P1..m,
  both over the alphabet ?.
• Output
• Whether P occurs in T
• If yes, then the set occ(P) i Ti..im-1
  P

3
Example Pattern matching
Pattern P AAGCTA
Text T CAAGCTAAGCTAC
A
A
G
C
T
A
Pattern
A
A
G
C
T
A
1
10
9
11
2
3
4
5
6
7
8
12
13
Text
C
C
G
T
A
A
G
C
T
A
A
A
C
Occ(P) 2, 7
4
Our Case

• Input
• Text A musical Sequence
• Pattern Rhythm
• Output
• Slightly different from Classical PM
• A extended/modified notion of PM

5
Musical Sequence

• A string t t1t2tn
• ti ? ?N
• Example
• 0,50,100,200,250,300,350,400,500,550
• Significance Sequence of events!

6
Musical Sequence

• A string t t1t2tn
• ti ? ?N
• Example
• 0,50,100,200,250,300,350,400,500,550
• Some (musical) event occurs at 0 ms

7
Musical Sequence

• A string t t1t2tn
• ti ? ?N
• Example
• 0,50,100,200,250,300,350,400,500,550
• Some (musical) event occurs at 0 ms
• Some (musical) event occurs at 50 ms

8
Musical Sequence

• A string t t1t2tn
• ti ? ?N
• Example
• 0,50,100,200,250,300,350,400,500,550
• Some (musical) event occurs at 0 ms
• Some (musical) event occurs at 50 ms
• Some (musical) event occurs at 100 ms
• And so on

9
Musical Sequence

• 0,50,100,200,250,300,350,400,500,550
• Alternative Representation (We use)
• 50,50,100,50,50,50,50,100,50
• Significance Sequence of duration of theevents!

10
Musical Sequence
First Representation
0
50
100
200
250
300
350
450
500
550
50
Duration of First Event 50 0 50
11
Musical Sequence
First Representation
0
50
100
200
250
300
350
450
500
550
50
50
Duration of Second Event 100 50 50
12
Musical Sequence
First Representation
0
50
100
200
250
300
350
450
500
550
50
50
100
Duration of Second Event 200 100 100
13
Musical Sequence
First Representation
0
50
100
200
250
300
350
400
500
550
50
50
100
50
50
50
50
100
50
Alternative Representation
14
Rhythm

• A string r r1r2rm
• rj ? Q, S
• Q means a quick(er) event
• S means a slow(er) event
• S is double the length (duration) of Q
• Exact length of Q or S is not a priori known

15
New Notion of Match

• Let Q?? q ? N
• Then S ? 2 ? q
• Q matches ti..i iff
• q ti ti1 ti
• 1 ? i ? i ? n
• If i i then the match is SOLID

16
New Notion of Match
Let q 150
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
Q
50100 150 q
17
New Notion of Match
Let q 150
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
Q
Q
100 50 150 q
18
New Notion of Match
Let q 150
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
Q
Q
Q
Q
Q
Q
19
New Notion of Match
Let q 100
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
Q
Q
Q
Q
Q
Q
Solid Match
20
New Notion of Match

• S ? 2 ? q
• S matches ti..i iff either of following is
  true
• i i and ti 2q (SOLID)
• i ?? i and there exists i ? i1 ? i such that
• q ti ti1 ti1 ti11 ti1
  ti
• 1 ? i ? i ? n
• So, S is either solid or a tile of 2 consecutive
  Qs

21
New Notion of Match
Let q 100, then S ? 2q 200
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
S
t1 t2 100 t3
22
New Notion of Match
Let q 100, then S ? 2q 200
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
S
S
t4 t5 100 t6 t7
23
New Notion of Match
Let q 100, then S ? 2q 200
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
S
S
S
S
24
New Notion of Match
Let q 100, then S ? 2q 200
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
S
S
S
S
S
Although, t2 t3 t4 200 2q
25
New Notion of Match
Let q 100, then S ? 2q 200
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
S
S
S
S
S
S
26
New Notion of Match
Let r QSS and Let q 50
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
Q
S
S
r
r matches t2..5
27
New Notion of Match
Let r QSS and Let q 50
1
2
3
4
5
6
7
8
9
Q
S
50
50
100
50
50
50
50
100
50
S
Q
S
S
r
r
r matches t5..8
28
New Notion of Match
Let r QSS and Let q 50
1
2
3
4
5
6
7
8
9
Q
S
50
50
100
50
50
50
50
100
50
S
Q
S
S
r
r
r covers t2..8
29
Our Problem

• Input
• A musical Sequence t
• A rhythm r
• Output
• The longest substring ti..i that is covered by
  r

30
Restriction

• For each match of r, at least one S must be solid

Let r QSS and Let q 50
1
2
3
4
5
6
7
8
9
r
50
50
100
50
50
50
50
50
100
Q
S
S
SOLID S
31
Restriction

• For each match of r, at least one S must be solid

Let r QSS and Let q 50
1
2
3
4
5
6
7
8
9
r
50
50
100
50
50
50
50
50
100
Q
S
S
r
At least one S must be Solid
32
Restriction

• For each match of r, at least one S must be solid

Let r QSS and Let q 50
1
2
3
4
5
6
7
8
9
r
50
50
100
50
50
50
50
50
100
Q
S
SOLID S
S
At least one S must be Solid
r
33
Motivation

• Our aim is to classify music according to dancing
  rhythm
• Music seq can be considered as a series of events
  corresponding to music signals
• Drum beats
• Guiter picks
• Horn hits

34
Motivation

• The intervals between these events characterize
  how the song is danced
• Basically two dancing rhythms Quick Slow
• Example
• cha-cha ? SSQQSSSQQS
• foxtrot ? SSQQSSQQ
• jive ? SSQQSQQS

35
Motivation

• So solution to our problem can classify songs
  according to dancing rhythms!

36
Algorithm

• Stage 1 Find all occurrences of S for a chosen
  value ?? ? ? such that ?/2 ? ?.
• Stage 2 Transform areas around each S into
  sequences of Q.
• Stage 3 Find the matches of r and consequently
  the cover.

37
Algorithm Stage 1

• We construct two arrays
• first1..? first? i iff the first
  occurrence of symbol ? appears at position i.
• next1..n nexti j iff the next occurrence
  of symbol at ti appears at tj.

38
Algorithm Stage 1
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
? 50, 100, assume indexed alphabet
39
Algorithm Stage 1
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
? 50, 100, assume indexed alphabet
First occurrence of 50
first1 1
40
Algorithm Stage 1
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
? 50, 100, assume indexed alphabet
first1 1
first2 3
First occurrence of 100
41
Algorithm Stage 1
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
next1 2
Next occurrence of t1 50
42
Algorithm Stage 1
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
next1 2
next2 4
Next occurrence of t2 50
43
Algorithm Stage 1
1
2
3
4
5
6
7
8
9
50
50
100
50
50
50
50
100
50
next1 2
next2 4
next3 8
Next occurrence of t3 100
And so on
44
Algorithm Stage 2
1
2
3
4
5
6
7
8
9
10
11

100
50
25
25
100
50
15
30
5
30
50

S
Consider an S at t5.
45
Algorithm Stage 2
1
2
3
4
5
6
7
8
9
10
11

100
50
25
25
100
50
15
30
5
30
50

S
Q
46
Algorithm Stage 2
1
2
3
4
5
6
7
8
9
10
11

100
50
25
25
100
50
15
30
5
30
50

S
Q
Q
47
Algorithm Stage 2
1
2
3
4
5
6
7
8
9
10
11

100
50
25
25
100
50
15
30
5
30
50

S
3050 gt 50
Q
Q
So Stop!
48
Algorithm Stage 2
1
2
3
4
5
6
7
8
9
10
11

100
50
25
25
100
50
15
30
5
30
50

S
Q
Q
Q
49
Algorithm Stage 2
1
2
3
4
5
6
7
8
9
10
11

100
50
25
25
100
50
15
30
5
30
50

Q
S
Q
Q
Q
So we get a new sequence t? consisting of Q and S
50
Algorithm Stage 3

• Here we get t? consisting of Q and S.
• We first want to find the matches of r in t?.
• Define St? ? S in t?, Qt? ? Q in t?
• Define Sr ? S in r, Qr ? Q in r

51
Algorithm Stage 3

• Construct t?? as follows
• St? 01, Qt? 1
• Construct Invalid set I where I contains the
  position of 1 due to St?.

Q
Q
S
Q
Q
1
1
1
1
0
1
52
Algorithm Stage 3

• Construct t?? as follows
• St? 01, Qt? 1
• Construct Invalid set I where I contains the
  position of 1 due to St?.

No match can occur at position 4 because it is
not a real position in t?
Q
Q
S
Q
Q
1
2
3
5
4
1
2
3
5
6
4
I 4
1
1
0
1
1
1
53
Algorithm Stage 3

• Construct r? as follows
• Sr 10, Qr 0

S
Q
0
1
0
54
Algorithm Stage 3

• Construct r? as follows
• Sr 10, Qr 0

S
Q
1
2
3
1
0
0
55
Algorithm Stage 3

• Find matches of r in t.
• Perform bitwise or at each position.
• If the result is all 1 and the position is not in
  invalid set then a match
• Otherwise no match

56
Algorithm Stage 3

• Find matches of r in t.

I 4
1
2
3
5
6
4
1
1
0
1
1
1
1
0
0
Bitwise Or
1
1
0
No Match!!!
57
Algorithm Stage 3

• Find matches of r in t.

I 4
1
2
3
5
6
4
1
1
0
1
1
1
1
0
0
Bitwise Or
1
0
0
No Match!!!
58
Algorithm Stage 3

• Find matches of r in t.

I 4
1
2
3
5
6
4
1
1
0
1
1
1
1
0
0
Bitwise Or
1
1
1
All 1s!!! And 3 ? I. So a MATCH!!!
59
Algorithm Stage 3

• Find matches of r in t.

I 4
1
2
3
5
6
4
1
1
0
1
1
1
1
0
0
Bitwise Or
1
1
1
All 1s!!! But, 4 ? I. So NO MATCH!!!
60
Algortihm

• In this way we can find the matches then we
  can easily compute the cover

61
Conclusion

• In practical cases size of the rhuthm is 10?13.
  So we have assumed m to be constant.
• It would be interesting, however, to remove the
  dependency on m
• It would be interesting to remove the restriction
  of one S being Solid
• Applying another restriction on the number of
  additions in the numeric text may turn out to be
  useful