How to Convert Sheet Music to Code and XML with Audiveris

If you've ever tried to scan old sheet music and play it back on a synthesizer or open it in a music editor like MuseScore, you probably know how agonizing the process can be. Standard OCR like Tesseract is helpless here: text characters are tied to a staff of five lines, while note durations, chords, rests, and dynamic markings form a two-dimensional graph structure.

This is where OMR — Optical Music Recognition — comes onto the scene. There aren't many projects in this area, and mature open-source systems can be counted on one hand. Audiveris is perhaps the most well-known open-source music recognition engine, actively developed for many years.

The Main Challenge in Music Recognition

The developers of Audiveris honestly acknowledge in the documentation an important fact: achieving 100% accuracy in music recognition purely through algorithmic means is practically impossible. Real scans from archives like IMSLP often suffer from paper creases, skewing, faded ink, and non-standard fonts from 19th-century publishers. If an error creeps into the key signature at the beginning of a line, the entire subsequent decoding turns into cacophony.

Therefore, the concept behind Audiveris is built on two closely linked components:

• Computational engine (OMR Engine)
• Built-in graphical editor (OMR Editor)

First, the engine performs recognition, and then a human steps in. In the program's interface, you can correct skewing, fix recognized symbols, or adjust algorithm parameters on the fly, without reprocessing the entire document.

How the Recognition Engine Works

The architecture of Audiveris can be described as hybrid. The creators didn't blindly rely solely on classical computer analysis or solely on deep learning. For each type of music notation element, they selected their own method:

1. Staff lines and grid are detected using specialized straight-line detection algorithms.
2. Note beams (strokes connecting eighth and sixteenth notes) are recognized through mathematical image morphology.
3. Text elements like titles, tempo markings, and lyrics are passed to an external OCR engine.
4. Note heads are identified through template matching.
5. All other fixed-size symbols are processed by a neural network.

The result is saved in an internal XML format with the .omr extension or exported to standard MusicXML 4.0. This format is understood by virtually all modern music notation software: Finale, Sibelius, Dorico, and MuseScore.

Watch Out for Domain Names

An interesting detail from the repository README: the authors specifically warn users about a fake website audiveris.com.

The real project lives on GitHub and its associated GitHub Pages site. A site with the .com domain zone mimics the official resource but redirects people to third-party services with cryptocurrencies and betting. Be careful if you decide to download a ready-made distribution.

How to Build and Run the Project

The code is written in Java and built with Gradle. You'll need JDK 17 or newer and Git to work with it.

The project uses a classic branching scheme:

• The master branch contains only stable releases.
• All active development happens in the development branch.

Clone the repository and build the project from the development branch:

git clone https://github.com/Audiveris/audiveris.git
cd audiveris
git checkout development
./gradlew build

Once the build is complete, you can run the application directly via Gradle:

./gradlew run

If you don't plan to modify the code itself and just want to convert sheet music, the developers provide ready-made installers for Windows (.msi), Linux (.deb and Flatpak on Flathub), and macOS (.dmg). They already include the required Java Runtime Environment, so you won't need to set up the environment manually.

Why This Matters for Developers

Audiveris is interesting not only to musicians. If you're developing services for working with media content, digital archives, or educational programs, this project offers a couple of great opportunities:

• Use the Java API. You can integrate Audiveris as a library into your project and automate the processing of sheet music PDFs on a server.
• Access the original document structure. The .omr format contains detailed information about the geometry of recognized elements, which is convenient for analysis or training your own models.

The learning curve here is quite reasonable, and the code licensed under AGPLv3 provides a good example of how to build complex hybrid image processing systems.

Conclusion

If you're facing the task of digitizing sheet music archives or embedding score recognition into your application, looking at proprietary tools costing hundreds of dollars is no longer necessary. Audiveris demonstrates a pragmatic approach to OMR: combining automatic algorithms with convenient manual correction delivers excellent practical results.