Song chords from audio

How chord recognition works

A chord is a few notes sounding together, and in a full mix they are tangled with the voice, the drums and every other part. A chord detector cannot read notes off a score, so it looks at the sound itself. We turn the song into a constant-Q spectrogram: 144 frequency bands, two per semitone, across six octaves from the lowest C of a piano upwards, about ten times per second. On that picture a chord shows up as a pattern of bright lines that repeats in every octave.

The pattern goes to BTC, a bi-directional transformer for chord recognition published at ISMIR 2019 by Jonggwon Park and colleagues, trained by its authors on a few hundred pop and rock songs with hand-made chord annotations. It looks at about ten seconds of music at a time, forwards and backwards, and gives each moment a probability for 170 classes: 12 roots times 14 chord types, plus no chord. We use the same official model and the same preprocessing as our TuneApp worker, ported to your browser and checked against it frame by frame.

A probability per moment flickers, so a decoder picks the most likely chord sequence, where every change has a cost. Then the beat: a second network, Beat This!, finds every beat and downbeat, and chord changes are only allowed on a downbeat, a beat or the eighth note between two beats. That is what puts the chords in time with the music instead of a few tenths of a second off.

Reading the chord sheet

The sheet is written in bars, four to a line, the way session players write charts. Each cell between two bars lines is one bar. A cell with one chord means the chord fills the bar. Two chords mean the bar is split in half; in four four time that is two beats each.

When a change falls somewhere else, the cell shows a slot per beat, with a dot where the chord carries on: A . . E is A for three beats and E on the fourth. N.C. means no chord, a silence or a part with no harmony we can hear, such as a drum break.

Chord names follow the usual habits. A letter alone is major, m is minor, 7 is a dominant seventh, maj7 a major seventh, m7b5 half diminished, sus2 and sus4 replace the third. The first bar can be a pickup when the song starts before its first downbeat.

Play along with capo and transpose

Transpose moves every chord up or down by semitones, and the key with it. Use it to sing the song in a range that suits your voice, or to match a recording that was tuned a little off from where you play.

The capo is for guitar shapes. Clamp a capo on the second fret and the chords on screen become the shapes you play, while they still sound in the song's key. If the song is in B flat, a capo on the third fret turns most of its chords into the open G shapes. We suggest the capo that gives the most open shapes, and the diagrams show every chord on guitar and piano.

Sharps or flats is about spelling, not sound: A sharp and B flat are the same key on a piano. Auto writes them the way the key is usually written. Triads only turns seventh and sixth chords into plain major and minor, which is the easiest way to start playing a song.

Accuracy and limits

On a clear pop or rock recording the result is usually close to what a musician would write, and the tests we ran agree with our reference worker. It is not perfect, and you should check it by ear. Its usual mistakes are honest ones: it confuses a chord with its relative, such as C and A minor, when the bass is quiet, and a seventh can come and go when a melody note passes over it.

Some music is harder. Dense jazz harmony, chords with added ninths and elevenths, distorted guitars, songs with no clear bass, and passages where only a voice or a lead line plays all confuse a chord detector. It writes chords in root position, with no slash chords or inversions, and it does not tell you how the chord is voiced or strummed.

Tuning matters too. The model assumes standard tuning at A 440 Hz. A song that sits between two keys can come out a semitone off; transpose fixes the names. The beat grid follows the tempo of the song, so a rubato ballad or a track with no pulse gets a looser grid.

For producers: MIDI chords into your DAW

Download MIDI and you get two tracks: the chords as held notes in a close voicing around middle C, and the root of each chord in the bass. The file carries the song's tempo and starts at zero, so drop it next to the audio at the start of your session and it lines up.

Use it to re-voice a progression with your own instruments, build a remix or a cover on the same harmony, check a sample's chords before you flip it, or sketch a bass line. The MIDI follows your settings: simplify, transpose and spelling apply to it as they do on screen.

To write new progressions, open the chord progression generator. To name a chord you play on a keyboard, the chord finder does it. To turn a melody or a bass line into notes, use audio to MIDI.

Learn more, or get a person on it

Read the mastering guides, comparisons and data, or send your tracks to an engineer with OBRA ULTRA.