Scenes that Move

An ordinary scene is of something that holds still while one camera goes round it. A person, a pour or a flame cannot be done that way: by the time the camera has gone round, nothing is where it was. A scene that moves is filmed by several cameras at once, fixed in place, so that every instant is seen from every side.

A scene that moves, with its time bar

What it needs

  • A camera rig. Around twenty to thirty cameras, all filming the same thing at the same moment. Fewer cameras give a softer result.
  • Where the cameras are. The capture has to come with its cameras already solved. Two layouts are read:
    • a folder with one video for each camera and a poses_bounds.npy file, which is how the Neural 3D Video set is laid out;
    • a folder with train_meta.json and an ims folder of frames, which is how the PanopticSports set is laid out.

Open the folder as you would any other. The Cameras section says how many cameras and frames it has, and the Train section's settings change to what suits a rig.

A sample moving scene to open and play is on the website.

Training

Press Train. The program works in four stages and names each as it goes: the scene at rest, finding what moves, the motion, and scoring. While the motion trains, the view runs through the clip over and over, so you see it forming as movement.

Expect it to take longer than a still scene: tens of minutes for a few seconds of film.

Playing

Under the view is a bar with time on it.

  • Space plays and pauses. The slider can be dragged, and the bar shows which frame is showing of how many.
  • Time runs smoothly, not frame by frame: paused between two frames, things are where they were at that moment.
  • The speed can be as filmed, or a half, a quarter or an eighth of that.
  • The camera is free throughout. Orbit, fly, the turntable and a camera path all work while time runs.

File > Export video films the scene moving, at the speed the bar is set to. Export… in the Deliver section writes the moment showing, since a file for other programs holds no time.

What to expect

  • It is a little softer than a still scene, because the cameras are few.
  • Something that stays in one place for most of the clip is taken for part of the room.
  • Very fast, small things, a thrown ball, are the hardest.
  • Tidying the scene (Remove faint, Crop to view, Reduce to) drops its motion; Undo brings both back.