Lip sync: delay, smoothing and expressiveness improvements
I tried the lip sync feature yesterday and found that the smoothing feature is great. However, I found that I had to deal with the consonants and vowels differently. I might be wrong, but smoothing the consonants seemed to smooth them too much. The consonants still need to be sufficiently expressed because if the mouth or tongue does not close, it is not possible to pronounce the consonant. However, this is not the case with vowels. So maybe consider a slider for each viseme that determines how strongly the smoothing affects it.
Also give us the opportunity to overdrive the expressiveness of a single viseme in the timeline to override the lip options. I had to make several cuts in the options so that I could increase the exressiveness of single visemes next to other visemes that I toned down.
In my preliminary testing, I think 6 frames works best, but maybe it depends on the visemes. The problem with 6 frames is that the mouth seems to close too early once he stops speaking so there should be some slow fade out.
Also consider variable smoothing. Smooth and decrease the expressiveness of the the vowels more strongly when there are several visemes in quick succession.
Third, it would be cool if we could provide alternative visemes and custom visemes. A round-robin algorithm with two or three slight variations of each viseme would be cool, but maybe more important are different visemes for different expressiveness. Instead of dialing one visueme up and down, you should consider blending between visemes with different expressiveness. Especially the ones with a lot of jaw movement could benefit from alternative visemes with less jaw movement.
I hope that the bending jaw bone is fixed in the future and moves like an actual bone.