Animating kinematic vocal folds

The lt.KinematicVocalFolds class is furnished with the method draw3d() to make 3D visualization of the vocal-fold model easier. This example demonstrates how to accomplish the 3D animation with matplotlib.animation.FuncAnimation in Jupyter Notebook.

First we load the necessary libraries

%matplotlib agg

from matplotlib import pyplot as plt
from matplotlib.animation import FuncAnimation
from IPython.display import HTML

import letalker as lt

Symmetric Periodic Vibration

The key to the animation is controlling the number of animation frames per glottal cycle. For a basic demonstration of a periodic vibration, we only need one cycle, and let the animation engine to repeat it at much slower speed than the real time, (similar to playing back highspeed videoendoscopy video at much slower rate).

For the first example, let’s use the default symmetric vocal folds at \(f_o=\) 100 Hz.

fo = 100
vf = lt.KinematicVocalFolds(fo)

Now, we need to set up the simulation rate and duration.

We only need one glottic cycle, which lasts in 10 ms. With the default sampling rate of 44.1 kHz, we would end up with 441 frames per cycle, which is too much detail necessary.

Instead, let’s aim for 30 frames per cycle by changing the sampling rate, then run the model for 1 cycle or 10 ms. Also, set the animation to play back at the rate of 1 cycle per second.

fpc = 30  # target frames per cycle
fs = fpc * fo  # new sampling rate to meet fpc target for the specified fo

# apply the new sampling rate to pyLeTalker
lt.core.set_sampling_rate(fs)

print(f"new sampling rate = {lt.fs} Hz")

new sampling rate = 3000 Hz

Now, the 3D model at each sample instance is drawn on Matplotlib’s 3D axes, and the animation is iteratively recorded by FuncAnimation as follows

fig = plt.figure()
ax = fig.add_subplot(projection="3d")

ax.view_init(22.5, -70) # good perspective view
# ax.view_init(90, -90)  # top view
ax.set_proj_type("ortho")
ax.set_aspect("equal")

def animate(i):
    vf.draw3d(i, axes=ax)
    ax.set_title(f'$t={i/fs:0.3f}$')

ani = FuncAnimation(fig, animate, frames=fpc, interval=1 / fpc)

Finally, we can show the animation in Jupyter Notebook with the following command:

HTML(ani.to_jshtml(fpc))

Once Loop Reflect

Entrained left-right biphonation

How about a case of 2:3 biphonation? Keeping one vocal fold to vibrate at 100 Hz, the other vocal fold would then vibrate at 150 Hz. These yield the true/overall fundamental frequency of 50 Hz, 100-Hz vocal fold vibrates twice while 150-Hz one vibrates thrice. Let the 150-Hz vocal fold to be animated with the same 30 frames/cycle recording rate and playback at 1 second/cycle speed. This needs the animation to be 3-cycles long or 90 frames, and we need the sampling rate of 4.5 kHz (150 Hz \(\times\) 3).

fo1 = 100
fo2 = 150
vf = lt.KinematicVocalFolds([fo1, fo2])

fs = fpc * fo2  # new sampling rate to meet fpc target for the specified fo

# apply the new sampling rate to pyLeTalker
lt.core.set_sampling_rate(fs)

print(f"new sampling rate = {lt.fs} Hz")

fig = plt.figure()
ax = fig.add_subplot(projection="3d")

ax.view_init(22.5, -70)  # good perspective view
# ax.view_init(90, -90)  # top view
ax.set_proj_type("ortho")
ax.set_aspect("equal")

ani = FuncAnimation(fig, animate, frames=3*fpc, interval=1 / fpc)

HTML(ani.to_jshtml(fpc))

new sampling rate = 4500 Hz

Once Loop Reflect

Stroboscopic animation

If \(f_o\) varies, then the vibration is no longer truly periodic. Then, the above method no longer applies. pyLeTalker is equipped with lt.utils.strobe_timing() to mimic the videostroboscopy, which tracks the fundamental frequency of the voice and precisely time the shutter intervals to make the captured vibration to appear periodic in video.

lt.utils.strobe_timing() takes the output sampling rate (video framerate), fundamental frequency, and duration of the animation, and the strobe rate fstrobe. It then produces the simulation sample indices to show fstrobe cycles per second.

Below, we synthesize symmetric vocal folds with time-varying \(f_o\) using lt.Interpolator to design the \(f_o\) contour, varying between 85 to 120 Hz over 3 seconds. Despite simulating at the default 44.1 kHz, the produced animation shows smooth vibration.

lt.core.set_sampling_rate(None) # revert to the default

# 3 second synthesis
T = 3

# time-varying fo
fo = lt.Interpolator([0, 0.9, 1.5, 2.4, 3.0], [100, 90, 85, 110, 120])

# define the model
vf = lt.KinematicVocalFolds(fo)

fr = 30  # video framerate

# select the sampling instances for video (display 1 cycles/second)
nstrobe = lt.utils.strobe_timing(fr, fo, T, fstrobe=1)

fig = plt.figure()
ax = fig.add_subplot(projection="3d")

ax.view_init(22.5, -70) # good perspective view
# ax.view_init(90, -90)  # top view
ax.set_proj_type("ortho")
ax.set_aspect("equal")

def animate(i):
    vf.draw3d(nstrobe[i], axes=ax)
    ax.set_title(f'$t={nstrobe[i]/lt.fs:0.3f}$')

ani = FuncAnimation(fig, animate, frames=len(nstrobe), interval=1 / fr)

HTML(ani.to_jshtml(fr))

Once Loop Reflect