While automatic text recognition is already fairly advanced, research into scanning and interpreting musical notes has not been able to achieve satisfactory results so far.
“Text typically contains only a small number of symbols, whereas music consists of composites. Musical notes can be combined not only horizontally but also vertically,” says Stadelmann. “By combining sub-symbols, we can basically generate an indefinite number of symbols, for example chords made up from individual note heads, stems and flags.” Thilo Stadelmann and his team have analysed around 300,000 pages of sheet music in order to develop this new method. In doing so, the ZHAW researchers have created the largest fully annotated data set worldwide in the field of computer-aided object recognition – and they have made it publicly and freely accessible for further research. The new method developed at the ZHAW can thus be applied to other problems in computer-aided object recognition.

The researchers tested their system with digital and printed sheet music for several months, continually improving it. Their development work, which is financially supported by Innosuisse, will be used for a concrete product in the future. A Swiss start-up company called ScorePad offers a digital music stand for tablets. This device can scan printed sheet music. The new app allows individual editing of the digitised notes. This includes zooming in and out, transposing to different keys and selecting musical instruments. The app also facilitates playing the music. It is no longer necessary to turn the pages by hand, because the software recognises the notes played with the help of the tablet’s integrated microphone and scrolls down automatically.