Augmented Reality Features with 3D Food Photography

As of 2020, consumers have already purchased more than 2 billion Augmented Reality (AR) capable smartphones. By 2022, that number will be 3.4 billion. This huge installed base presents news organizations with the best opportunity to explore spatial 3D media with their audiences.

In this project, Knight Lab students will make exploring mobile AR easier for small teams and individuals, by developing and publishing a series of guides on how to make AR features using open source and low cost software.

Students will learn by doing, developing skills along the whole production pipeline, from asset creation, to development and design, to deployment. We will produce at least three AR projects over the course of the quarter. Each project will contain at least one 3d model of a food item produced by photogrammetry, the process of constructing a 3d model from 2d photos.

We chose food as the theme because it taps into an existing culture of sharing tabletop photos on social media and requires a lot of detail in order to be captured in a compelling way. We’re also interested in the potential mobile AR has to bring food journalism into our audience’s homes in a much more literal way. Are you more likely to feel confident approaching a recipe if you see the ingredients laid out on your own countertop beforehand? Can a 3d model illustrate the difference between two kinds of tomatoes more effectively than a flat photograph? Would you try more unusual foods if you could inspect them close up in 3d on your phone first?

This is a great project for students with experience and interest in photography, 3D modeling, writing, graphic and interaction design. Students coming from a software background will have the opportunity to learn about graphics optimization, how AR technologies work, working with SDKs and deploying projects to the web. Tools we’ll explore will include Figma, Meshroom, Blender, Torch and the Spark AR Studio for creating Facebook and Instagram filters.

Faculty and Staff Leads

Rebecca Poulson

Engineer, 2016-2020

Rebecca leads AR/VR projects at Knight Lab. In addition to being a software developer and playwright, Rebecca is an Oculus Launchpad Fellow and Mozilla Tech Speaker. She has taught WebVR workshops on three continents.

Results

Mixed Reality > Augmented Reality Features with 3D Food Photography

Building AR Journalism Experiences to Educate and Inform

by Agnes Lee , Rui Xin Oh , Harrison Pearl , Carlyn Savino , Brian Sui

In Spring of 2020, one of the Knight Lab Studio teams focused on exploring how Augmented Reality can be used to create experiences and tools within journalism and food media. They identified three major categories of experiences. Later this summer, we’ll be sharing their work on experiences focused on entertaining and connecting audiences. This week we’re sharing what they learned while building experiences intended to educate and inform. Making Information Available in the Right Place...

Continue Reading

Project Details

2020 Spring

Important Questions

What’s the most effective way to capture food photographs for photogrammetry?
How do you build an efficient work-flow for a small team?
How can 2d assets and 3d assets be combined to create a meaningful AR experience?

Sample Milestones

Week One: Survey of the current state of AR for news, introduction to tools
Week Two: project ideation, creating and editing models
Week Three: Write up guides for creating and editing models. Choose model to work with for midterm.
Weeks 4-5: Design and build AR experience in Torch.
Weeks 5-7: Incorporating lessons learned, build a second experience in Torch. Write a guide for developing similar features in Torch
Week 8: Choose one of the previous experiences to make an Instagram or Facebook filter for.
Weeks 9-10: Continue working on filter and guides, create SparkAR guide, prepare for final presentation

Outcome

Students will develop skills along the full AR pipeline from project ideation to asset creation, experience design and deployment, documenting their process and creating guides for other journalists along the way.