Mixed Reality

Virtual, Augmented... It's all Reality

In these projects we explore tools and topics around the quickly changing fields of virtual reality and augmented reality. We want to understand how these new tools fit into journalism and other kinds of storytelling, and the only way to learn is to try things.

Projects

A Tool for Creating Shareable, Embeddable WebVR Stories

This team is looking at how to accommodate image formats from a variety of popular 360-degree cameras, including iPhone panoramas. The goal is to make a tool not only for journalists but also for any storyteller looking to tell a story in VR.

Augmented Reality Visualization Tool

Based on the outcomes of our a Exploring AR Visualizations project in the Winter, this project will take the unique forms of visualizations discovered and develop a tool that makes it easy for storytellers to build and embed augmented reality visualizations in their stories and projects.

Resurrecting History for VR

Automated Photogrammetry. As AR and VR increasingly becomes the focus as we move away from smartphones, media organizations will have to find ways to produce content native to those mediums. These same organizations have a wealth of stories that continue to have value over time. Flat video and photo will become less desirable as we continue to move into these new spaces. Previous projects in the lab have surfaced an opportunity to take existing video and process it using photogrammetry software to produce a 3D model.

Exploring Augmented Reality Visualizations

An experimental design project that explores visualizing data in three dimensions for augmented reality. Visualizations that can be examined and inspected by physically getting closer or understood by walking around them, open up exciting possibilities for how we communicate complex ideas and data that reveals hidden truths.

Exploring Data Visualization in VR

An experimental design project, in which we students are analyzing methods for communicating data visually and exploring how those principles might be transferred and transformed in a 360 environment.

Information Spaces in AR/VR

An experimental design project that explores an emerging concept of information spaces. This concept is behind Microsoft’s pivot to create future Windows operating systems that exist in mixed reality headsets. Students will explore the concept of a news/information app that exists in AR and/or VR. For example, a political news feed might exist on a wall in your living room, and breaking news would appear on a coffee table, twitter reactions surround the coffee table on the floor. Students will also explore the same concept in a virtual environment.

Designing Information Spaces for Augmented Reality

Immersive technology allows creators to engage users in new and novel ways, many of which can make the interactions users have with information easier or more meaningful. This project will look at a four different storytelling formats that exist today (a cooking blog, a sports broadcast, a web interactive and a podcast) and reimagine them for augmented reality using tools like Torch for iOS and Magic Leap’s Create tool.

Iterative Story Design Play-Testing Augmented Reality

This project will build on the work of the Location Based Storytelling Studio group from Fall 18 which produced and ran an initial play-test for an AR game about climate change. During Winter 19, we will continue play-testing and refining the game while gathering feedback from players through surveys and interviews. Can playing a game really impact attitudes about climate change? Which is more impactful--a game designed for “fun” or a game that’s more transparent about its educational goals?

Location-based Storytelling with Augmented Reality

Augmented Reality allows creators to tell stories in the places where they actually happened. For the purpose of this project, we’ll be working with the concept of physical accessibility on campus. We will use the Unity game engine and a framework developed at Knight Lab so that we can quickly experiment and refine our ideas. We’ll spend some time early in the quarter looking at AR experiences from museums and tourist attractions as well as games like Ingress and Pokemon Go. From there, we’ll build an app and start testing it with users, refining ideas as we go along. We’ll consider the differences between gamified apps that include scavenger hunts or challenges and AR applications that lay out information in a more traditional way.

Navigating Virtual Reality

Though many in journalism are excited about VR, few are addressing real issues with making it attractive and interactive for their audience. This story team will explore the idea of making multiple three-dimensional VR photos around a scene and linking them together so that the user can navigate it. They’ll be exploring complex VR design challenges, such as how to move around space without disorienting the user and how to easily author interactive environments.

Oscillations: Immersive Virtual Experiences in the Performing Arts

Based on the outcomes of Oscillations project in Winter quarter, this project will continue exploring using the latest VR production techniques to create engaging VR experiences. Students will record performances using motion capture and use machine learning to teach a computer to improvise a performance.

Oscillations: Immersive Virtual Experiences in the Performing Arts

Based on the outcomes of Oscillations project in Winter and Spring quarters, this project will continue exploring using the latest VR production techniques to create engaging VR experiences. Students will record performances using motion capture and use machine learning to teach a computer to improvise a performance.

Oscillations: Immersive Virtual Experiences in the Performing Arts

Recent breakthroughs in neuroscience and immersive technologies provide mechanisms for engineering an entirely new mode of performance art — one that engages audiences to unprecedented degrees.

Photojournalism in 3D for VR and Beyond

In this project, students will use modern approaches to making 3D images both with hardware and software processing.

Projection Mapping

Imagine that any wall, building floor or doorway could come alive and tell stories. Using projection mapping, many artists, advertisers and industries are already doing it. In this project, students will explore ways in which projection mapping is currently being used and adapt them for journalistic purposes. Students will build and prototype their adaptations.

Storytelling Layers on 360 Video

In this project, students will film 360 video and explore the best ways to add on that additional layer; students will finish the quarter with two videos, and will document their findings to make their storytelling methods more accessible to others.

Faculty and Staff Leads

Zach Wise

Associate Professor

Emmy winning interactive producer & Associate Professor @NorthwesternU, @KnightLab. Formerly of The New York Times. Creator of TimelineJS & StoryMapJS

Rebecca Poulson

Technical Program Manager, Emerging Platforms

Rebecca leads AR/VR projects at Knight Lab. In addition to being a software developer and playwright, Rebecca is an Oculus Launchpad Fellow and Mozilla Tech Speaker. She has taught WebVR workshops on three continents.