The Hammer Without A Nail

Oscillations is a New Art Form Making Meaningful Impact

Imagine a classroom of elementary-aged students. They appear to be sitting cross-legged on the floor or walking around the room, but in reality, they’re dancing. Dancers flash across the screens of the virtual reality headsets the students wear. The performers jump and turn just inches from the young audience members’ eyes. The children can see the subtle flexibility of the dancers, connecting the shifts of movement to the music’s tempo. Looking up, down and side to side, the students see a world that’s unlike their reality and a source of endless curiosity.

Oscillations provides accessibility to a new art experience for young audiences. The dancers don’t have permanent studios filled with a stretching prima ballerina or clapping artistic directors. The performances don’t happen in big theatres to the sound of an orchestra. It’s dance, but in the mind. The responsive virtual reality experience is accessible without regard to the setting. Oscillations is a responsive VR immersion that challenges sensual perception. As audiences watch the performance, sensors in the VR headset detect the attentive state of the brain, altering  visual and auditory effects to capture the mind’s maximum focus.

For several quarters, the Knight Lab Studio class has worked with Oscillations, exploring a variety of topics and technologies. While previous cohorts focused on the VR experience, a group had yet to focus on the on-boarding experience for new users. Finding the initial audience for Oscillations started with the question of accessibility. Which users lacked access to performance art and were likely to engage with an experience such as Oscillations?

For this research, accessibility was defined as the ability to experience arts education and interpret meaning from the piece. On October 9th, 2018, we spoke with Oscillations’ Co-Founders Ilya Fomin and Danielle Perszyk. We raised the question of how this virtual reality performance can be accessed to bring meaningful impact to a specific audience, but the creators struggled to identify its target consumers. We met with Northwestern University Performance Studies Librarian Greg MacAyeal, who led us toward resources for background research, consisting of a deep background check into the accessibility of art. Who lacked  accessibility? Why was accessibility even important?

Young people of low socioeconomic status who engaged in the arts had better grades in school and higher rates of college enrollment.

According to a 2012 NEA study, young people with intensive arts experiences demonstrate higher levels of volunteering and civic engagement. The study showed that young people of low socioeconomic status who engaged in the arts had better grades in school and higher rates of college enrollment. Students who engaged in dance programs displayed improvements in social competence, (prosocial behavior and cooperation,) especially children from lower-SES families.  We established three potential audience segments for Oscillations to target: “The Art Enthusiast,” “The Elementary Student,” and “The Tech Fan.” Based on information of who attends performances, we ultimately decided to focus on “The Elementary Student” because of the proven potential of children to learn through art.

Access to early arts education is often limited by (1) distance to venues, (2) affordability of performance and (3) availability of shows. Oscillations provides an experience that allow users to interact with the performance happening before their eyes. This tool can overcome obstacles limiting the accessibility of performing arts in classrooms. An early age virtual reality experience will expose young people to a new form of technology and art, producing a  response similar to receiving arts education in a traditional classroom setting.

In user-centered design, “affordances” refer to the benefits that extend beyond the core deliverable of a product. A 2011 study established  the psychodynamic approach, a framing tool for cognitive growth. According to this  framework,the brain expands its creativity because of “tension between conscious reality and subconscious drives.” Oscillations is both a conscious and subconscious experience, testing the expectations of the user with their awareness. As a student’s mind explores their reality, a new form of understanding is formed, “giving tolerance for ambiguity…and mental freedom to find problems,” according to the study. Oscillations is not only entertaining, but also engaging for the mind.

I can’t imagine that buying a headset would be less expensive than sending the student to a show. You have a hammer without a nail.

Dr. Eleanor O’Rourke, NU Computer Science

Oscillations, though, is not without drawbacks. “I can’t imagine that buying a headset would be less expensive than sending the student to a show,” said Northwestern University Computer Science Professor Eleanor O’Rourke. “You have a hammer without a nail.” O’Rourke added that competitive options exist in the market for giving elementary students access to art, and should be a major consideration for Oscillations. In-classroom movies, in-theatre performances, student theatre performances, children’s education mobile apps and museum visits are alternatives with specific ramifications.

The Oscillations difference is what the brain looks like on art. A 2017 study by the Association of Art Historians highlights the idea of neural plasticity, or how the brain’s neurons are impacted by experiences: “The study of no other subject can introduce a pupil so directly to the problems of his sensory relation to his environment.” Oscillations literally shapes the minds of young audiences members by giving the brain’s sensory cortex function a new set of experiences. Our audience recommendations are consistent with how the brain’s plasticity decreases as the mind ages. Elementary students represent the population with the most plasticity, and are therefore able to benefit the most from a perceptual VR experience.

We recommend that Oscillations approach elementary students using the  “4Ps”: product, price, place and promotion. The neuro-social impact of Oscillations is likely to create  positive reinforcement in students and teachers – the more students learn, the more frequently consumers will use the tool, providing feedback and contributing a more dynamic experience. As Oscillations grows more stable, the 4P approach may be adapted for the secondary and tertiary segments, “Tech Fans” and “Art Enthusiasts.” A consistent audience presence will create long-term growth for Oscillations.

What sets Oscillations apart from other virtual reality performances is the meaningful impact it can bring to public education. Not only will it reach the usual “Tech Fans” but, by exploring its use as a tool for teaching and learning, elementary students will receive art experiences that were previously inaccessible. By shifting focus to creating interactive performances appropriate for all audiences, Oscillations has the potential to become a leading virtual reality experience used for social impact.

About the project

OscillationsImmersive Virtual Experiences in the Performing Arts

Advancements in neuroscience and immersive technologies offer mechanisms for engineering an entirely new mode of performance art one that engages audiences to unprecedented degree. Using the latest VR production techniques, students used motion capture and machine learning to teach a computer to improvise a performance, creating an engaging VR experience.

About the authors

Nicole Fallert

Janet Lee

More results from Oscillations

  • Oscillations Audience Engagement Research Findings

    During the Winter 2018 quarter, the Oscillations Knight Lab team was tasked in exploring the question: what constitutes an engaging live movement arts performance for audiences? Oscillations’ Chief Technology Officer, Ilya Fomin, told the team at quarter’s start that the startup aims to create performing arts experiences that are “better than reality.” In response, our team spent the quarter seeking to understand what is reality with qualitative research. Three members of the team interviewed more...

    Continue Reading

  • Comparing Motion Capture Techniques for Movement Art

    With Oscillations’ connection to the movement arts, it made sense to experiment with existing motion capture technology to find accurate, consistent, and scalable ways to obtain three-dimensional motion data for purposes such as animation or machine learning to augment performances in virtual reality. An additional motivation to learn more about motion capture was connected to our early experiments with spatial audio (read more about them in our spatial audio blog post). Apart from using ambisonic...

    Continue Reading

  • Prototyping Spatial Audio for Movement Art

    One of Oscillations’ technical goals for this quarter’s Knight Lab Studio class was an exploration of spatial audio. Spatial audio is sound that exists in three dimensions. It is a perfect complement to 360 video, because sound sources can be localized to certain parts of the video. Oscillations is especially interested in using spatial audio to enhance the neuroscientific principles of audiovisual synchrony that they aim to emphasize in their productions. Existing work in spatial...

    Continue Reading