I have 2 GTX 970 GPUs and one RTX 2080 Ti with 32 GB host ram and Intel i7 @4 GHZ, 30 GB set for Virtual memory and plenty SSD space. I don't have the exact error message but it would say something like failed to allocate memory on each GTX 970.
I hope Reallusion would adopt the Vulkan API because it apparently doesn't care if you're using different GPUs and memory.. I don't know if that would directly help allocate and distribute resources better to Iray but it might free up resources for iClone and reduce bottlenecks overall.