I should have posted this a while ago, but while I was at MAGIC, I did some work on finding an appropriate mapping between hand locations in space and vowel formant frequencies. If you’re interested, you can download the poster.

The project that I’m currently working on has been written up in Maclean’s magazine. The DIVA performer in the image above is Marguerite Witvoet.

Academic Collaborators:

• Sid Fels, MAGIC, HCT Laboratory, UBC
• Bob Pritchard, MAGIC, School of Music, UBC

While working for TC-Helicon, I developed an audio effect to convert singing voices into the sound of a choir. For example, starting from four harmony voices (mp3 wav), the effect multiplies the apparent number of voices, resulting in a choir effect (mp3 wav).

This effect is included in one of TC-Helicon’s latest products, the VoiceLive 2. You can find another sample of the choir effect in action in the sidebar of the VoiceLive 2 web page.

Project

My PhD research started in collaboration with IVL Technologies and TC-Helicon, two voice processing companies in Victoria. TC-Helicon produces vocal effects products for the music industry with a focus on pitch correction and automatic harmony creation. IVL produces hand held karaoke products that produce a variety of vocal effects while being plugged into a TV. In recent years, vocal effects have become more common including chorus based effects and even distortion.

The goal of the research was to enhance a digital effect that adds noise to a voice to simulate breathiness. If a voice already sounds breathy, it is easy to add noise to increase the perception of breathiness. However, it is difficult to add noise to voices that exhibit high-effort (i.e. when the voice sounds strained). The added noise does not blend into the voice and instead sounds like a separate stream of noise. In my research, I developed a technique to adaptively manipulate the perception of vocal effort. This enabled the added noise to blend more effectively, thereby improving the breath effect.

An important outcome of this research is to highlight the fact that Linear Prediction (LP) (a common voice modeling technique) does not appropriately model the voice. I presented Adaptive Pre-emphasis Linear Prediction (APLP) as a technique to appropriately compensate for variations in vocal effort.

Academic and Industry Collaborators

• Peter Driessen, MISTIC, Electrical Engineering, UVic
• George Tzanetakis, MISTIC, Computer Science, UBC
• Glen Rutledge, 3dB Research (previously at TC-Helicon and IVL Audio)
• Kevin Alexander, TC-Helicon
• Brian Gibson, IVL Audio
• John Esling, Linguistics
• Mathieu Lagrange, MISTIC, Computer Science

Publications

• papers
• dissertation

Original voice:
I started with this sound sample: wav. With this kind of voice, it is difficult to add breathiness in a way that sounds natural.

Breath effect with constant pre-emphasis linear prediction (LP):
Constant pre-emphasis LP was carried out, noise was added to the residual, and the voice was resynthesized: wav. This is the technique used in current voice processors to apply a breath effect.

Breath effect with APLP:
APLP can be used to transform the spectral envelope of the voice to match that of a typical breathy voice. This reduces the perceived vocal effort and improves the blending of noise into the voice: wav.

Breath effect with APLP and breath modulation:
The amount of breathiness in a voice varies with the amount of effort. As such, it makes sense to add less breathiness during times of excessive effort. The APLP algorithm was further improved by modulating the quantity of added noise according to the quantity of vocal effort: wav.

Now listen to the original voice again.

With the breath effect, some voices work better than others. Here is how the breath effect sounds on another voice:

Original voice: wav.
Breath effect with constant pre-emphasis LP: wav.
Breath effect with APLP: wav.
Breath effect with APLP and breath modulation: wav.

________________
Technical Note:
Previous iterations of the algorithm included glottal closure detection to improve the blending of noise into the voice. Glottal closure detection was eliminated from this iteration of the algorithm. This ensures that the sound samples are representative of what can be achieved on real-time voice signals in a musical context.