Tomohiko Nakamura

Acoustic Automatic Accompaniment System for Musical Performance with Errors and Arbitrary Repeats and Skips

Tomohiko Nakamura, Eita Nakamura, and Shigeki Sagayama (The University of Tokyo)

We propose a system that can synchronize musical accompaniment automatically to a human performance with insertion/deletion/substitution errors and arbitrary repeats/skips in real time [1].

Accompanying to Human Performance during Practice

Accompanying to Human Performance with a Deletion Error

Score for the deletion-error example

Accompanying to Human Performance with Repeats and Errors

A repeat and tempo changes
Some repeats
Many errors and repeats in a short time (00:39 to end)
<figure class="demo-accompaniment-score">
  <figcaption>Whole Score: Do-Re-Mi</figcaption>
  <p>The parts highlighted in pink are played by the human, and the others are played by the proposed system.</p>
  <a href="images/doremi_mod2.png" target="_blank" rel="noopener">
    <img src="images/doremi_mod2.png" alt="Do-Re-Mi score highlighting the human player's parts" loading="lazy">
  </a>
  <p><a href="images/doremi_mod2.png" target="_blank" rel="noopener">View full-size score</a></p>
</figure>

References

  1. Tomohiko Nakamura, Eita Nakamura, and Shigeki Sagayama, “Real-time audio-to-score alignment of music performances containing errors and arbitrary repeats and skips,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 2, pp. 329–339, Feb. 2016.
    Bib 
    @article{TNakamura201602IEEEACMTASLP,
    author = "Nakamura, Tomohiko and Nakamura, Eita and Sagayama, Shigeki",
    journal = "IEEE/ACM Transactions on Audio, Speech, and Language Processing",
    month = "February",
    number = "2",
    pages = "329--339",
    title = "Real-time audio-to-score alignment of music performances containing errors and arbitrary repeats and skips",
    volume = "24",
    year = "2016",
    doi = "10.1109/TASLP.2015.2507862"
}
    arXiv Demo