Augmented Reality Features with 3D Food Photography

As of 2020, consumers have already purchased more than 2 billion Augmented Reality (AR) capable smartphones. By 2022, that number will be 3.4 billion. This huge installed base presents news organizations with the best opportunity to explore spatial 3D media with their audiences.

In this project, Knight Lab students will make exploring mobile AR easier for small teams and individuals, by developing and publishing a series of guides on how to make AR features using open source and low cost software.

Students will learn by doing, developing skills along the whole production pipeline, from asset creation, to development and design, to deployment. We will produce at least three AR projects over the course of the quarter. Each project will contain at least one 3d model of a food item produced by photogrammetry, the process of constructing a 3d model from 2d photos.

We chose food as the theme because it taps into an existing culture of sharing tabletop photos on social media and requires a lot of detail in order to be captured in a compelling way. We’re also interested in the potential mobile AR has to bring food journalism into our audience’s homes in a much more literal way. Are you more likely to feel confident approaching a recipe if you see the ingredients laid out on your own countertop beforehand? Can a 3d model illustrate the difference between two kinds of tomatoes more effectively than a flat photograph? Would you try more unusual foods if you could inspect them close up in 3d on your phone first?

This is a great project for students with experience and interest in photography, 3D modeling, writing, graphic and interaction design. Students coming from a software background will have the opportunity to learn about graphics optimization, how AR technologies work, working with SDKs and deploying projects to the web. Tools we’ll explore will include Figma, Meshroom, Blender, Torch and the Spark AR Studio for creating Facebook and Instagram filters.

Faculty and Staff Leads

Rebecca Poulson

Engineer

Rebecca leads AR/VR projects at Knight Lab. In addition to being a software developer and playwright, Rebecca is an Oculus Launchpad Fellow and Mozilla Tech Speaker. She has taught WebVR workshops on three continents.

Project Details

2020 Spring

Important Questions
  • What’s the most effective way to capture food photographs for photogrammetry?
  • How do you build an efficient work-flow for a small team?
  • How can 2d assets and 3d assets be combined to create a meaningful AR experience?
Sample Milestones
  • Week One: Survey of the current state of AR for news, introduction to tools
  • Week Two: project ideation, creating and editing models
  • Week Three: Write up guides for creating and editing models. Choose model to work with for midterm.
  • Weeks 4-5: Design and build AR experience in Torch.
  • Weeks 5-7: Incorporating lessons learned, build a second experience in Torch. Write a guide for developing similar features in Torch
  • Week 8: Choose one of the previous experiences to make an Instagram or Facebook filter for.
  • Weeks 9-10: Continue working on filter and guides, create SparkAR guide, prepare for final presentation
Outcome

Students will develop skills along the full AR pipeline from project ideation to asset creation, experience design and deployment, documenting their process and creating guides for other journalists along the way.

Apply to Project