That's how we did this; https://m.youtube.com/watch?v=7jfZ6zLkWF0 (was running live on-set)

That's so impressive!

What company do you work for? This is super cool.

Can you talk about the rest of your stack? Or other projects of a similar nature?

http://newchromantics.com/ is my own little company, (website is criminally out of date) I did the tech, http://analogstudio.co.uk/ did the art/comp/post work.

The setup was essentially an app (c++ engine on osx running models), ingesting camera feeds, outputting skeletons, running sound->mouth poses, to unity which puppeteered 2D sprites.

On-set we output to a monitor which mixed the real shots with our graphics.

Nothing crazy! But it was a solid setup (except for overheating DSLRs)

That’s so epic! Can you please give a little bit more detail on your process?

That doesn't look like it needs person segmentation.

No, but the post above was about skeletal tracking, not the OP content.

Really cool ! Tensor js ?

Do you mean 3D animation?

It looks like the interface returns 2D coordinates for the joints, not 3D coordinates; so it can't directly create 3D animation data.

It was also very jumpy in my browser, so it would require smoothing to be usable; or maybe if it wasn't running real-time it would be more accurate?