We needed 3d characters on screen to move interactively with actor.
Theater, Interactive, Mocap
For capturing actor movement we used Rokoko mocp suite witch allowed us to capture smallest movements. Modeling, rigging and texturing all assets was done in blender.
For interactivity and performance we used game engine UE4 where cloud particles was emitted from the character.
Cameras from UE4 was routed via Spout to Resolume where the live content was mixed with prerecorded elements.