3.2 Finding piano note frequencies

Windowing

We now want to find the frequency that each note is played at. To do this, we want to isolate each note using the times provided by the note onset-detection code. Our program uses a simple rectangular window to get the part of the song we want to analyze. Rectangular windows introduce some high-frequency noise to the spectrum because both ends of the sound vector now have steep edges, but it is a quick method and it did not noticeably impact the performance of the code.

Frequency spectrum

Now that we have a single note isolated, we take its Fourier transform and normalize it so we can analyze it in the frequency domain. The note has strong peaks at frequencies which correspond to frequencies of the note which was played. Harmonics of the note also appear quite strong, depending on the specific piano used to play the note and the note itself.

First, we must determine that a note is being played at all. If a note is being played, there will be peaks in the frequency spectrum that are large compared to the rest of the spectrum. We determined that if peaks exist with a magnitude of 100 or greater, that a note was certainly being played. This algorithm runs into some trouble with very soft parts of songs, but is very reliable for most songs.

In order to determine which notes are being played, we use a simple threshold detector. If the peak is above a certain threshold (our final code uses 0.1), then we mark it as a candidate for being a note that was played. The peak must also be far enough away from other peaks so that we do not register a double-peak. The program then iterates over all the candidates to determine which are actual notes being played and which are harmonics.

Harmonics of a vibrating string occur at integer multiples of the fundamental frequency. Since a piano consists of many vibrating strings, we know that a peak should show up at frequencies corresponding to integer multiples of the note that is being played. Knowing this, we can always determine that the peak with the lowest frequency must be a note being played because it cannot be a multiple of any lower frequencies. Using this lowest frequency, we then remove any harmonics that are integer multiples of it. We then search for harmonics of the next-lowest frequency peak that is still a candidate, and repeat until we have determined that every remaining peak is not a harmonic of any other peak.

Unfortunately, octaves present a unique challenge. The note one octave above any note occurs at twice the frequency of the lower note, which is the same frequency as its second harmonic. There is no sure-fire way to determine when a peak is a harmonic and when it is a second note being played one octave above the first note, so we implemented another threshold system. If the peak in question is more than .3 times the value of the lower-frequency peak, then our program interprets it as a second note being played instead of a harmonic. This value seems to work in most cases. Harmonics above the second tend to be very weak in the frequency domain, but they could potentially cause a problem on certain pianos.

One of the hardest parts of finding the peaks is setting a good threshold value. It should be low enough that all the necessary peaks are set as candidates, but high enough that the noise is never picked up. We square the spectrum to get the power spectrum, which makes the peaks stand out an allows us to easily see where we should set the threshold. Changing threshold values is much easier when looking at this view.

All of the candidates that remain are determined to be notes that were played and the frequencies are returned by the program.