I filmed myself turning in place, then filmed a few seconds of waving. From those two clips, a model learned what I look like as about 100,000 tiny coloured blobs (3-D Gaussians) wrapped around a digital skeleton, and that skeleton can be posed any way you like. Try it below.
Drag the avatar or use the slider to turn it up to 60° either way.
Loading the avatar…
I never waved both hands at once, squatted or did jumping jacks on camera. Each move is typed in as joint angles for the skeleton, and the learned appearance follows it. That is the point of an avatar: it can do things the person was never recorded doing.
Each frame here was rendered by the model on my GPU, at nine turn angles and fourteen moments per move. The page just plays them back, so it is fast on any phone. The colours and shading are the model's own; nothing is drawn by hand.
A slow turn in place (about 30 seconds) and a few seconds of free movement, on a phone, against a plain wall, with no markers or special suit. Plus four numbers: my height and my chest, waist and hip measurements.
Every frame is cut out automatically, and a standard body model (SMPL) is fitted to it using the silhouette and detected joints, nose, eyes and ears. The body's shape is fitted to views from all sides and to my measurements, and kept inside the outline of my clothes rather than inflated to fill it.
Following Gaussian Shell Maps (Abdal et al., CVPR 2024), five thin shells around the body each carry a texture of 3-D Gaussians. Their colour, opacity, size and orientation are adjusted until renders match the turn and the first half of the motion clip, which takes about two minutes on one consumer GPU.
Because the Gaussians ride on shells bound to the skeleton, any new pose moves them along. Turning, waving and squatting are all just new skeleton poses.
The first version tracked me well only while I faced the camera. Holding my position during the turn, fitting my shape from every side, and adding my real measurements cut the tracking error from about 42 to 9 pixels, so the sides and back are now learned too. They are still softer than the front.
Training on part of the waving clip fixed most of the streaks on raised arms, but the undersides of my arms and deep poses like a squat were never seen, so they are still blurrier. A longer capture with more varied movement is the next step.
Built on Gaussian Shell Maps (Abdal et al., CVPR 2024) and 3D Gaussian Splatting (Kerbl et al., SIGGRAPH 2023), with the SMPL body model (Loper et al., 2015), torchvision's Mask R-CNN and Keypoint R-CNN (He et al., 2017) and PyTorch3D. The fitting and tracking layer that turns two phone videos into this avatar is ours, written with help from Claude Code.