Stitching 360° Video

For the time-being, footage filmed on most 360° cameras cannot be directly edited and uploaded for viewing immediately after capture. Different cameras have different methods of outputting footage, but usually each camera lens corresponds to a separate video file. These video files must be combined using “video stitching” software on a computer or phone before the video becomes one connected, viewable video. Garmin and other companies have recently demonstrated interest in creating cameras that stitch on board, meaning that this step will likely soon be technologically obsolete, but for now, we have to figure out a way to blend the lenses into one video on our own.

Most 360 cameras have proprietary stitching software that stitch automatically in most cases. The Gear 360, for example, comes with software called “Gear 360 Action Director.” Importing footage from the camera into this software automatically stitches the video for you, no additional work necessary. This automation makes life remarkably easier for generating a vast amount of 360 content quickly, but comes at the cost of inexact stitching.

Stitch Lines

“Stitch lines” in 360 are the areas of overlap between the lenses that have been stitched together, and appear as disconnected lines that are clearly meant to be continuous. The areas on either side of stitch lines might look a bit discolored in contrast to one another, because the cameras were automatically set to different light settings. In the picture below, the subject was standing directly in the overlap between two lenses, and his face got caught in a stitch line.

There are two situations, then, where you’d want to stitch manually using a more sophisticated software: Your camera doesn’t come with an automatic stitching software, or you want to have more control over making a video appear cohesive by hiding particularly tricky stitch lines.

Autopano

Kolor’s Autopano is the most popular comprehensive program to stitch with manually. There are others, but they tend to be more intensive and difficult to use for little increase in quality, so we’ll be talking about Autopano for the remainder of this guide.

The first thing to do is look at what kind of unstitched footage your camera produces. Camera rigs like the GoPro Omni or the Z Cam S1 will produce multiple mp4 files per take; one for each lens, as in the image on the left. The Gear 360 and other few-lens cameras produce just one mp4 file. Viewing this file in a media player like VLC shows one oval corresponding to each lens, like the image on the right.

GoPro Omni footage (multiple files)
GoPro Omni footage (multiple files)
Vuze 3D 360 footage (one file)
Vuze 3D 360 footage (one file)

Using Autopano with the GoPro Omni

The GoPro Omni produces the easiest footage to stitch by default. Drag the footage from all 6 cameras into the main window of AVP, and select Synchro at the top.

You likely want to synchronize by sound rather than by motion. Sound will be more accurate if you did not take any preemptive stitching measures at the time of filming. To get the best effects here, a loud clap in the beginning of filming will be good to sync by audio, or twisting the camera before filming will be good to sync by motion.

Next, under the Stitch panel, make sure the lens preset is set to the correct model of GoPro camera used in the Omni Rig, and then select Stitch. For the most part, this will produce a clean video.

This will create an initial stitch of your video, providing a panorama preview in the right-hand window if you click a particular moment in the bottom panel’s timeline. This image will likely look a little wonky, so you probably want to click Edit beneath it to make things look like intended.

You’ll be prompted to open up Autopano Giga at this point. Go ahead and do so. You should see an image resembling the screenshot below, where the panoramic image from the previous step is clearly visible. If you were working with multiple panoramic images at once, you’d be able to see them here, but we’re not in this case, so just the one is fine. Once again, hit Edit right next to the image to bring up more options.

There’s a bunch to point out on this next page.

On the left side of the screen, you’ll see a number called the Global RMS Curve. RMS stands for “Root Mean Square,” for the mathematically inclined. In a perfect world, this number would be 0, meaning that the geometric equations running in the background to blend these two images found no differences between the same object in two different views. Realistically, that won’t happen, as viewing the same image from different angles results in distortions due to parallax. The important thing to remember is that a lower number means better stitching. Aim for below 3.5.

If you’re significantly above 3.5, you want to consider selecting a different snapshot from your video in the AVP timeline to edit in Giga. Images with more light or fewer objects-in-motion close to the camera will be the best for this purpose.

If you still can’t get that number low enough, you’ll want to edit your control points, which will be the button between the two red rectangular boxes at the top. I won’t cover this idea in too much detail here, as you are unlikely to have RMS issues if you’re using GoPro cameras. In general, the idea is to place control points on pixels that are exactly the same in both images presented to you. If you want to know more, Autopano has good info in their documentation.

The two tools in the red boxes are the more important to focus on. The one on the left, Move images mode, allows you to make the horizon lines more accurate so that your image doesn’t come out looking strangely distorted. Click it to get a new window. Here, you can manipulate the image by dragging it around. Focus on the straight lines farther away from the camera, like the horizon, making them appear straight.

After the image looks like you’d expect it to in real life in terms of the geometry, you may still have some subjects that you’d like to have be crisp throughout the video hanging out in the stitch lines for too long. For the most part, the stitch lines are small, so they probably won’t be there for a problematic amount of time. For the moments where your subject is in an inconvenient stitch, however, and their face appears misshapen or blurred, there’s a useful tool to help figure that out.

Head over to the other boxed tool, the mask tool. If you mouse over the different parts of the image, you should be able to see the areas of overlap between lenses from the perspective of just one.

Here, you can add the left side green markers for images you want to keep from this lens, which will take priority over other lenses that have captured the same image. You can also place red markers for objects you’d like to remove from certain angles, if an arm appears to be shadowing, or appearing twice, from two different lenses.

You can use the markers on the right to apply this keep/remove effect to ALL images. This tool is useful if you want to remove a tripod from an image. To do this, place red markers in all camera angles that contain the tripod, while placing green markers on the surrounding background.

Once you’ve done all this, save this template as a .pano file, and head back over to Autopano Video Pro. Select the timeline of your footage you want all of these effects to apply to, and select Stitch as Pano, choosing the .pano file you just saved as your template.

Using Autopano with other 360 cameras

Using Autopano for another type of camera requires the creation of a stitching template (a .pano file) and a lens preset. The first step here is figuring out the exact lens specifications of a particular camera. Variance is possible between different instances of the same camera model. Close is good enough usually, but the best quality stitching will come from an instance-specific lens preset.

To create this lens preset, set the camera in a still area (nobody walking by) on a tripod. Set the camera to record. Allow it a moment to film, then rotate it ever so slightly and stop again, allowing it to film from its new orientation. Do this 12-15 times, creating one full rotation.

In VLC or a similar media player, take a still frame of each of the moments where the camera is at rest. Import these still frames into AVP and select image properties. Select the lens type of the camera, and a value around 7.00 mm for 35mm eq focal length. This is a middle, starting value, and does not need to be exact.

In the circular crop tab, fit the circle to the size of one of the lenses. Then launch detection. Play around with the focal length setting, launching detection with a few different values, and try to find the one with the lowest RMS curve value. Then, click edit on that panorama and accept the prompt to open Autopano Giga.

In Giga, drag the screen to create the best possible stitch–focusing on flat lines/edges and horizon lines to ensure they remain flat.

Create an XML file that looks like this:

Replace the lens type and focal values with the values selected earlier. K1, K2, and K3 should be replaced by their corresponding values in the following table.

Then, open up settings in AVP, go to the Edition tab, and find the path where Data is located. Drag your XML file into the appropriate folder at this path user/presets/camera.

From here, shoot a quick video on your camera to use to create a stitching template.

If your camera has multiple cameras placed next to each other in the video preview (i.e. your camera produces only one video file per take), then you must duplicate that file for as many times as there are lenses on the camera and drag all files into AVP. If there are multiple video files per take, drag them all into AVP and synchronize them. Duplicated files need not be synchronized.

In the Stitch tab, select the camera preset you just created, and then select stitch.

The next part is certainly the most difficult. After the initial stitch, select Edit to open up Autopano Giga, where you will have to use the move tool to make the image appear as normal as possible. More than likely, you’ll have to manually add and remove control points. Once again, Kolor has some nice documentation on how to do this, and there are plenty of tutorials on Youtube if you are still confused.

Once the footage looks good enough, save the project as a .pano file.

When stitching future footage, instead of selecting to stitch by a camera preset, choose Stitch as Pano and select this file. Edit the new video stitch in Giga if you need to, repeating this process.

When you’re all done with stitching and making your video just right, select Render at the top to export your file as an mp4 with the resolution specifications you’d like.

About the authors

Jake Daniels

Virtual Reality storyteller studying Radio/TV/Film, Computer Science, and Design in Northwestern University’s class of 2019.

About the project

Storytelling Layers on 360 Video

In this project, students will film 360 video and explore the best ways to add on that additional layer; students will finish the quarter with two videos, and will document their findings to make their storytelling methods more accessible to others.

More results from Storytelling Layers on 360 Video

  • An introduction to 360° video

    360° video is a fairly recent technology in which omnidirectional cameras are used to grab a spherical video capture of a space, rather than the rectangular capture in traditional videography. The perspectives of the omnidirectional cameras are then stitched together to generate an immersive experience for viewers to experience, placing the viewer within the context of a scene or event rather than presenting them as an outside observer, and giving the viewer the ability to...

    Continue Reading

  • Guide to shooting 360° video

    Before embarking on any 360° video project, we strongly encourage you to ask why. It is true that a spherical video can transport and immerse viewers in a way that is hard to replicate with traditional film, but which shouldn’t be used for the sake of creating 360° video. We encourage content creators to ask themselves the following questions before deciding to shoot in 360: Are we transporting the viewer to a place they cannot...

    Continue Reading

  • How to Edit 360 Video in Adobe Premiere Pro CC

    Before you start editing your footage, make sure you have stitched your footage. You can read our guide to stitching here. After you have completed stitching your footage, you can now “edit” them in traditional video editing software like Adobe Premiere Pro CC. Stitching is a common term that involves merging the separate camera inputs into single viewable format. On the other hand, editing is a broad umbrella term that we will use to refer...

    Continue Reading

  • Publishing your 360° content

    Publishing can be confusing for aspiring 360° video storytellers. The lack of public information on platform viewership makes it nearly impossible to know where you can best reach your intended viewers, or even how much time and effort to devote to the creation of VR content. Numbers are hard to come by, but were more available in the beginning of 2016. At the time, most viewers encountered 360° video on Facebook. In February 2016, Facebook...

    Continue Reading