Enable Lip Sync Options & Reduce Lip Keys for ALL iClone iAvatars
Known multi-year issues: Even with crystal clear, well-normalized 44K 16bit mono vocal wav files 1. IClone lip sync engine produces 50 to 70% REDUNDANT viseme keys 2. IClone lip sync engine fails to line up at minimum 50% of usable keys
IClone lip sync inaccuracies are probably acceptable (or indiscernible) to hobby movie makers To discerning users, experienced animation artists or those who've used other prosumer lip sync tools (Mimic or Unity plugins etc) iClone's auto-generated lip keys are at best 25% accurate which means, tedious manual adjustment and fine-tuning or, may as well start from scratch: ERASE the whole auto-generated Viseme>Lips track and then MANUALLY match little viseme keys, patiently, one by one, to audio file
RL is aware of the issue and have attempted to fix it. IClone6 introduced fixes: - Reduce Lip Keys (activated only for CC) - Lip Sync Options (activated only for CC)
RL also finally added Audio scrubbing in a recent update. All helped. However Deep into iClone 6 dev cycle...yet Reduce Lip Keys and Lip Sync Options have yet to be enabled GLOBALLY for ALL IAVATARS!
G5, G6, Genesis1, Genesis 2, Genesis 3, Maya import, Max import, Blender import, Poser import, Mixamo import can all benefit from iClone's progress in lip sync. After all, IClone is marketed as a MULTI-CHARACTER-PLATFORM-SUPPORTING character animation tool.
Thank you for listening and thank you for making iClone a more EFFICIENT animation tool for discerning animators.
If the lipsync visemes are exposed in the upcoming Python scripting we can greatly improve it by using outside programs and/or just other scripts. I have written routines myself which will improve such tracks 100% (or more). Please consider making it possible to access these (both reading and writing, but particularly writing to that track).
gogstudios
Can you tell us why it isn't enabled for non CC characters? Is there some technical limitation that's stopping you guys from doing it?
pedrosura
Agree with @Animagic. It is possible to create the iTalkFile in CT8 and import into IC6. Then also importing the real audio (dropping into the Sound Track) and muting the TTS voice you imported from CT8 which is on the Visame track. Then breaking up the Visame to match the sound. A LOT OF WORK where if we just had Iclone to type the text and match the next to the Sound file for timing, I am certain the result would be astounding. To do this procedure for a long movie is extremely laborious. I hope this could be implemented in a future update.
animagic
Recently, 3DTest published a procedure to improve lip-synching, which suggests that the ingredients are there, they just need to be put together. 3DTest found that the lip-synching in CrazyTalk 8 when using TTS is superior to lip-synching using an audio file (in either CT8 or iClone) or even using TTS in iClone. As it is easier to line up accurate visemes than to try to edit incorrect ones, 3DTest suggest saving the viseme assignments from CT8 as iTalk file and then using them in iClone by importing that iTalk file. The actual speech file will then be added as an audio file.
This is a lot of work, but it confirms that an approach were the input text is available as a guide to produce the visemes (which is most likely the way that this done for CT8 TTS) is quite feasible and bound to give better results.
