>>85177256The program will have to properly judge the perspective of the scene, so it knows what size the model should be when it's in X location. It should also be able to track realistic walking, to avoid the "sliding" or "moonwalking" effect where the movement of the legs doesn't match the motion across the ground. It should also maintain the vtuber's location irrespective of the camera. So it has a complete understanding of the environment and where a particular object is placed in that environment. Therefore the vtuber can walk through the environment as if it's a videogame level, and the camera moves indepedently.
This technology is already 100% possible and is already being used for other programs. But the tricky part is setting up a program that moves the vtuber around realistically. I think the cherry-on-top would be the ability to move around people in the crowd realistically.
The only thing the vtuber controls is the face inputs, an maybe arm-tracking too.
I think it wouldn't even be that difficult for the most part, even a small team could program it.
