The ability to recreate an object in 3D to be viewed and handled in virtual reality seems like something you would read in a science fiction novel.
In fact, as early as 1950, Ray Bradbury entertained the idea of a virtual reality nursery in “The Veldt” - his short story about two children who solve their anger with their parents by escaping to a simulated grassland.
Photogrammetry is the science of deriving measurements from photographs. Some of these techniques have been used since the earliest days of photography. In current usage, photogrammetry generally refers to the processing of still images to produce produce 3D models and assets for viewing in virtual reality.
With the growth of virtual and augmented realities for media and storytelling, 3D photogrammetry is drawing interest in newsrooms. As a new journalistic tool, it is important to begin documenting and establishing best practices based on testing and iteration early on. We see our exploration as a contribution to the journalism community and an early addition to important ethical dialogue.
In normal 2D photography, the photographer acts as the eyes for his audience and decides which elements in his environment are framed. At the other end of a continuum, in 360° video, the photographer gives up much of the control over composition. For 3D photogrammetry, composition is still important for making models. The photographer and technicians can achieve much better results by knowing how best to capture still images.
Photogrammetry software analyzes the photograph and constructs a “point cloud” of the position of elements recognized from one photo to another. These points are connected by the software to create “polygons” — usually small triangles — that define the model’s shape or “mesh.”
Photogrammetry is very flexible because it can be used to make models of people, things and places that have flat-matte surface (such as skin, cloth, paper or walls). It is also not ideal for moving subjects so we would suggest using 360° video for motion-oriented stories.
Authors of the study “Virtual Reality Journalism” by the Tow Center for Digital Journalism hypothesize that as visual media technologies progress, viewers become closer to the experience of subjects, and “the barriers between self and the other begin to erode.” While 3D photogrammetry models do not fully achieve the interactivity and presence offered by immersive headset VR experiences created with game engines, photogrammetry models empower audiences to virtually explore locations, objects and people via headset and browser.
As with more familiar formats, journalists generating 3D VR environments and assets using photogrammetry must conform to ethics codes from organizations like the Society of Professional Journalists and the National Press Photographers Association to ensure boundaries remain intact. It is important to remember virtual reality and 3D technologies have the capacity to reveal more in- depth information about a subject than 2D photography can. Plan your work carefully and with respect for your subjects. One of the NPPA ethical guidelines journalists risk violating is “Resist being manipulated by staged photo opportunities.” Photogrammetry requires a degree of staging and maintaining still subjects, and then usually removes those subjects from their original context. Because of this, journalists will have to be careful they are not intentionally or accidentally biasing interpretation with their setup. One way to avoid this is to photograph subjects in a neutral space where the environment is incidental not consequential.
Also be aware that what you capture or use in your photogrammetry models could be protected by copyright. Finally, be careful how you manipulate those models during creation as to avoid anything that strays from what was actually there.
From Story Conception and Beyond
The first question we advise journalists ask themselves when considering photogrammetry for storytelling is whether or not the entire story will be told using photogrammetry as the primary medium, such as virtual assets in a virtual location, or as a secondary medium where models enhance a text or video story.
The second question is whether “exploration” or “discovery” will be a key aspect of your story? Photogrammetry is an ideal way to empower audiences to explore objects of complex detail, foreign locales, or engulfing environments. Via a headset, it provides users a sense of presence within or alongside these models. You can then incorporate audio or other methods to narrator your story.
The third question to keep in mind: will what we model have visual/visceral impact while it is motionless? Photogrammetry (as it exists today) creates stills assets. While your narrative may encourage the audience to manipulate, explore and progress– your locations and model subjects will be frozen in time.
The final question, will you have the control to get your location and subject to remain motionless long enough to capture every image you need to reconstruct the model? Photogrammetry is like taking a portrait, it requires cooperation, but shouldn’t be staged. But while a single portrait photo takes less than a second, photogrammetry requires hundreds of photos from different camera positions. And if the subject moves, just like a blurry photograph, it will be distorted, or sometimes not rendered at all or rendered with significant artifacts.
If you think you could have a compelling photogrammetry story, we suggest incorporating photogrammetry in your earliest story planning to define its purpose and ensure adequate rendering/processing time is available.
Examples where photogrammetry could be useful:
A place of historical significance or natural wonder. (Chernobyl VR)
Stories that involve relevant objects like a historical artifact or complex tool. (National Geographic: Trajan Column)
Stories about people’s appearance, position or pose. (3D Portrait of President Obama)
The Presentation of Photogrammetry Stories
3D assets and environments like those produced with photogrammetry require unique considerations before inclusion in a story.
While digital photography and video are familiar narrative complements, most web-oriented content management systems lack simple tools for embedding and/or hosting 3D content. (If your CMS allows you HTML, CSS and other code editing capabilities, you can integrate objects using A-Frame or WebVR.) For this reason, two competing concerns when developing a photogrammetry story are technical accessibility for audiences and engaging presentation.
The simple solution for presenting photogrammetry-derived assets–as well as the presentation we chose for our subject-oriented photogrammetry exploration–is to display 3D models in an embedded viewer. Just like embedded photos or videos, viewers can open and operate in all browsers including those on mobile devices, and can be set up with basic animations like constant, slow rotation. They are easy for audiences to use with click-and-drag and scroll-to-zoom controls for free rotation and scaling (respectively), and sharing assets presented in a viewer is quick and simple. What viewers lack is the ability to translate the viewing perspective in 3D space, so while they are useful for viewing a solid asset from its exterior, you could not fully view a photogrammetry-derived environment from its interior with obstructions.
More immersive methods of photogrammetry presentation utilize virtual and augmented reality technologies. These can be as simple as Google’s Cardboard app, available for Apple and Android phones, or as complicated as an HTC Vive, which requires a slew of expensive hardware. Regardless, at this end of the technological spectrum, the expectation should be that photogrammetry presentation will be inaccessible to all but a particularly savvy few. Currently, genuine virtual reality content (i.e. not 360 video) exists almost exclusively on online digital distribution platforms, like app stores and software marketplaces (most notably the Steam Store by Valve). Requiring dedicated programs to run unique VR experience makes platforms like these ill-suited for serial photogrammetry presentation, but room-scale VR like these platforms provide allows for unmatched and nearly unrestricted interaction with photogrammetry-derived assets *and *environments.
Recently the advent of SketchFab - an 3D asset sharing platform provides a simple way to embed 3D models into webpages without using A-Frame or WebVR.