Audio to MIDI converter

What audio to MIDI does

Audio is a recording of what a sound did. MIDI is the instruction to play it: which note, when it starts, how long it lasts and how hard it was played. An audio to MIDI converter listens to a recording and writes those instructions back. You then play the same notes with any instrument in your DAW, fix a wrong one, change the key or print them as a score.

The conversion in this tool is polyphonic, which means it can find several notes at the same moment. That matters for a piano, a guitar with chords or a layered synth. It also works on a single line such as a voice, a bass or a saxophone.

You can use it as an MP3 to MIDI converter, a WAV to MIDI converter, a voice to MIDI tool or a humming to MIDI tool. The file type does not change the method: the audio is decoded in your browser and the same model listens to it.

How this converter works

The model is Basic Pitch, an open source model from Spotify's research team, published at ICASSP 2022 under the Apache 2.0 license. It is small, about 0.2 MB. Your file is first mixed to mono and resampled to 22,050 Hz. It is then cut into windows of two seconds that overlap a little, and the model reads each window.

For every slice of time (about 86 per second) it gives three answers for the 88 notes of a piano: which notes are sounding, where a note starts, and the exact pitch curve. A second step, which we ported from Spotify's published code, turns those answers into notes. It looks for starts above a threshold, follows each one until the sound fades, drops notes that are too short and then looks again for long notes that had no clear start. The pitch curve gives the optional pitch bends.

All of it runs in a worker in your browser, using ONNX Runtime Web. The page keeps the matrices from the model, so when you move a slider the notes are decoded again at once and the model does not run a second time. Your recording is never sent anywhere.

Get cleaner MIDI from your recording

The best results come from one instrument at a time, played clearly and without much reverb: a piano take, a vocal line, a guitar riff, a bass. Noise, echo and effects that smear a note make the model less sure. If your recording has a lot of room sound, a dry take almost always gives better notes.

A full song with drums, bass, chords and vocals is the hardest case. The model was not trained to tell instruments apart, so you get one mixed set of notes, and drums add stray ones. Split the song into stems first and convert the part you want. Our stem splitter does that in the browser too.

If notes are missing, raise the sensitivity or lower the shortest note. If you see too many small notes, do the opposite, or narrow the pitch range to the instrument you recorded. For a single voice or a lead, switch on one note at a time: the strongest note wins when two overlap, which clears most of the stray notes.

The settings, in plain words

Sensitivity sets how sure the model must be before it writes a note. Higher gives more notes, including some that are not there. Lower keeps only the clear ones. The middle is Spotify's own default.

Shortest note throws away notes shorter than this many milliseconds. Raising it cleans up a shaky voice. Lowering it keeps fast runs. Pitch range switches off everything above and below the notes you choose, which is the quickest way to stop a bass line from producing high stray notes.

One note at a time keeps only the loudest note at each moment, for voice, bass and lead lines. Pitch bends add the small slides of pitch, up to two semitones, as MIDI pitch bend messages, so a sung glide or a guitar bend survives. Bends are only written for notes that do not overlap another note, because a MIDI channel has one bend at a time.

The four starting points, piano, voice or humming, guitar and bass, are settings we chose by ear. They are not magic. Pick the closest one, then adjust with your own ears.

What people do with the MIDI

Producers use it to sample ideas without sampling sound. Hum a bass line into your phone, convert it and give it to a synth. Record a piano idea, get the notes and re-voice them with another instrument. Pull the melody out of a reference track and change it enough to make it yours.

DJs and remixers use it to find the notes of a hook or a bass line so they can write a new part in the same key, then check the tempo and key with our other tools. Singers and songwriters turn a voice memo into notes to send to a producer or to see which notes they actually sang.

Teachers and students use the piano roll to see a melody they hear, practice by ear and check themselves. Arrangers can convert a part, clean it up in the DAW and print it from there.

What is inside the MIDI file

You get a standard MIDI file, format 1, with a tempo track and one part that holds all the notes. Velocity comes from how strong each note sounded. If you switch on pitch bends, the bend curve is written at plus or minus two semitones, which is the default range of most instruments.

The notes are placed in real time, so they line up with your original recording. The file also carries a tempo so your DAW grid has something to show. We estimate it from the audio and show it in a box you can type over. If the estimate is half or double the real tempo, the notes still sound at the right moment, only the bars are drawn differently. Drag the file into Ableton Live, Logic, FL Studio, Cubase, Reaper, GarageBand or any other DAW.

What it cannot do

We would rather tell you than let you find out. It does not transcribe drums. It does not name chords or write a score. It does not separate instruments in a mix. On a dense full song it will miss notes and add wrong ones. Very low notes and very high notes are less reliable, and fast, quiet passages blur together.

It reads up to 10 minutes of audio at a time, one file at once. This is a beta: tell us at info@obra.art what it gets wrong and we will look at it.